Search is Easy; Data is Hard

The New York Times magazine has just published a fascinating article about Google, discussing whether Google has become an aggressive monopolist in the area of search, and if so, whether or not it needs to be broken up under anti-trust law. The article, which is well worth a read, cites case after case where Google ostensibly derailed other companies that had seemingly developed better search tools than Google.

Better search tools than Google? Is that even possible? That’s where I take some slight exception to the article. Possibly in order to make this topic more accessible to a mass audience, it labels all these competitive search providers as “vertical search” companies.

Those of us with some history in the business remember back to around 2006 when “vertical search” was a thing, a thing that has long since faded. At the time, the concept of vertical search was a full-text search engine, much like Google, but one that was focused on a single vertical market or specific topic area. The thinking was that if publishers curated the content that was being indexed, the search results would be stronger, more accurate, more contextual and more precise. A prime example at the time was the word “pumps.” As a search term, it’s a tough one – the user could be looking for a device that moves fluids … or shoes. A vertical search engine, which would be oriented towards either equipment or fashion, would reliably return more relevant responses. Vertical search failed as a business not because it was a bad idea, but because (and let’s be honest here) most of the publishers rushing to get into it were too lazy to do the up-front curation work. And without quality, up-front curation, vertical search quickly becomes just plain bad search.

Vertical search as used in the article really refers to vertical databases. The difference is important because the article also states that parametric search is hard. That statement is simply more proof that vertical search as used in this article means databases. Parametric search is not hard: collecting and normalizing data so it can be searched parametrically is hard. Put another way, searching a database is easy, providing there is a database to search.

Google never wanted to do the work of building databases. It sometimes bought them (example: a $700 million acquisition of airline data company ITA) or “borrowed” them (pulled results from third-party databases into its own search results, effectively depriving the database owner of much of its traffic – think Yelp). What Google did instead was devote unfathomable resources to develop software code to try to make unstructured data as searchable as structured data. While it made some impressive strides in this area, overall Google failed.

With this context, you can clearly see why data products are so important and valuable. Data collection is hard. Data normalization is hard. But there’s still no substitute for it, something Google has learned the hard way. It may be disheartening to see survey after survey where we learn that users turn to Google first for information. But this is the result of habituation, not superior results. For those who need to search precisely, and for those who really depend on the information they get back from a search, data products win almost every time … provided that users can find them. Read this article and judge for yourself  just how evil Google may be…



Looking for New Product Ideas? They're Not All in Your Head

Part Three.

For many information and media companies, the preferred way to develop new product ideas is to brainstorm them internally. Get your best minds in a room and talk about the industry and its needs. You can conduct these sessions in a highly structured way or make them completely freewheeling and open-ended. Good, solid ideas can result.

Brainstorming sessions are both convenient and efficient. And if your staff is deeply engaged in your market, bringing them together to discuss new product concepts can yield powerful, even electrifying results. That’s because your staff is essentially reporting back what it is hearing and seeing in the marketplace. Synthesizing their different inputs, finding themes and conceptualizing solutions to problems is a great group activity, and resulting new product ideas can be very strong indeed.

Contrast that with companies that aren’t close to their markets. Their group brainstorming sessions will yield bigger product concepts (arguably bigger opportunities, but also riskier and harder to execute), incomplete concepts (based on lack of detailed market knowledge), and little certainty about market appetite. Perhaps most significantly, these product concepts, because they tend to be bigger, somewhat amorphous and without clarity as to market need, rarely get developed further.

My bottom line view of new product brainstorming is that it works, but the output can’t ever be better than the input. If your staff knows your market, they can effectively act as customer proxies, and the results can be compelling. If your staff doesn’t, brainstorming results in pipe dreams.


Looking for New Product Ideas? Can We Talk?

Part Two.

As I explained in Part 1, the most dependable new product ideas are totally organic in origin, meaning they are originated by people who want the new product as much for their own use as for others. The best ideas come from real personal need, not concepts or abstractions. To this end, I am surprised so few publishers encourage people to bring them their new product ideas: it’s free market research, and the really good ideas tend to be easy to spot.

Of course, you can’t depend entirely on a passive source like this. That’s why many publishers make an effort to talk to their customers. It doesn’t take a lot of conversations to start hearing about marketplace needs and opportunities. While the idea of talking to customers for new product ideas is well-known, your success depends in large part on how you go about it.

It’s surprisingly difficult to get productive conversations going with your customers. First, you have to get a meaningful amount of time from them,  which gets harder every day. Second, you have to enter the conversation without preconceived notions or biases. Third, the conversation needs to be open-ended to allow the customer to take it in any direction. When a customer volunteers something like “but what I could really use is …” you have struck gold. You can have conversations by phone, though in-person conversations are always the best. And please don’t think that sending out an online survey in any way substitutes for customer conversations.

The good news is, yes, customers will tell you what they want, and they’ll do it happily. If multiple customers suggest the same new product idea, you’ve probably got a winner.



Looking for a New Product Idea? Just Ask.

(Part One- Continues Next Week)

Where do really good ideas for new data products come from? Not surprisingly, I am asked this question a lot. Perhaps surprisingly, the answer isn’t all that complicated.

The best ideas for new data products almost invariably come from personal need. History shows that the data products that succeed most readily tend to be highly specialized in terms of content and user base – and they were typically surfaced by people who would use such a data product themselves, if someone else produced it. The person who sees the opportunities knows just how useful and valuable the new product would be, that nothing else like it currently exists in the market, and that there are many other people in similar roles in other companies who would benefit from it. Right there, you have all the ingredients for a winning data product, and I have seen dozens of them over the years, in almost every case started by someone with no data publishing experience, but who did have a deep understanding of the need for the data. As just one example, a recent news article talks about a professor, frustrated by the lack of information on sustainable building products manufacturers, decided to compile his own directory. Despite being published as a print directory, it’s already in its second edition – the need was out there for this information.

Why did a professor of architectural technology and building science decide to become a publisher? Likely because he didn’t feel he had any options. And that’s not surprising. For despite the intense interest of B2B media companies in new data products, not one that I know tries to reach out to its audience for new product ideas. That’s a shame, because in my experience it’s mid-level executives buried deep in large organizations who are the best source of these new opportunities. All you have to do is ask. 

North Korea Sparks a Trip Down Memory Lane

The latest news from North Korea should make us grateful we are not in the business there.  On word that several North Korean phone directories had been smuggled outside of the country, the country’s leader, Kim Jung-un, ordered that ALL phone numbers in the country be changed … randomly and without warning!

Here’s some nostalgia to put this in perspective. Here in the US, it began with  something called the “fax machine.” This was a device that scanned documents and then transmitted them via phone lines to a distant location. Faxes were the email of their day, but to get the real-time delivery benefits of faxing, you needed a separate phone line for your fax machine so that it was always available to send and receive. This created a huge jump in demand for new phone lines, and thus, new phone numbers.

If fax machines weren’t enough, we also had the advent of mobile phones, each of which demanded its own phone number. Phone companies ran out of available phone numbers in existing area codes, and begin the seemingly endless process of introducing new area codes (73 in just the past ten years), creating endless amounts of new work for data publishers in the process.

Those of you in the trenches for all this fun may also recall that the phone companies initially favored the dreaded area code “splits,” where half the people in an existing area code would be assigned a new area code. After much complaining, particularly from businesses that had to change signage, stationery and more, the phone companies moved to “overlay” area codes, where all new phone number requests in an existing area code simply received numbers with the new area code.

That’s another quaint aspect of area codes in the old days – they used to define specific geographies. But with the growth of both toll-free numbers, VOIP phones and number portability, your phone number no longer necessarily ties you to any geographic area.

Of course, for all the angst and additional work these changes have caused, at least they were systematic.  And if you are looking for expansion opportunities in 2018, North Korea appears wide open.