The Fragmentation of Search

The New Year is traditionally a time for predictions, so I will dutifully take my turn. I am only offering one, but it's a big one: in 2006, we will begin to see a meaningful shift in the current GYM (Google, Yahoo, Microsoft) search hegemony.

Am I predicting the implosion or demise of one or more of the big three search engines? Not at all. Instead, I am predicting that the search business will begin to evolve in ways that don't play well to the strengths of the big general search engines and that we will see concrete evidence of this by the end of 2006. I also believe that we're starting to see the big three getting distracted, likely to the detriment of their core search business. Yahoo is increasingly emphasizing content, and has previously de-emphasized search to chase other opportunities. Google, for all its technological prowess and business success, is rapidly enmeshing itself in deals that will distract it, if not alter its very essence (think AOL). And at this late date, Microsoft is still trying to figure out what it wants to be when it grows up. You can never count Microsoft out of the game, but they're clearly running out of time and options if they truly want to be a major player in search.

Perhaps more importantly though, users are raising their expectations with regard to search. It was a huge technological achievement to get so much of the Web under one index, and I never ceased to be amazed by the new search tricks and innovations that are being rolled out on a regular basis. Yet, I believe the game is changing, and what users -- particularly business users -- hunger for now is not the most answers, but the "right" answer. To get to the "right" answer demands sophisticated filtering and true domain expertise, something that simply doesn't fit within the one size fits all framework of the general search engines.

If I am right (and at worst I may be off on my timing), we are going to see two types of search rapidly getting both attention and market traction: vertical search and paid search.

Vertical search is not a new concept, but it's had difficulty gaining mainstream acceptance and momentum, both of which seem imminent. While definitions vary a bit, to me vertical search means presenting a deep, highly filtered and very intelligent entry point into a specific market. Vertical search demands true domain expertise, because you can't filter intelligently unless you truly understand the information needs of a market, and you are experienced and confident enough to vet information sources to present the best as opposed to the most. Vertical search is most likely going to be advertising supported, but don't rule out paid search.

Paid search sounds a bit crazy, with GYM offering so much for free. Of course, people also said cable television would never succeed with broadcast television offering so much for free. I believe the marketplace increasingly wants both deep and dependable sources of information. This is where many of the content aggregators are playing right now, but most are hanging their hats on having the biggest collection of stuff, as opposed to the most comprehensive collection of stuff in a single area.

So while there will continue to be a large role for GYM to meet general and unsophisticated search needs, the future is about specialty search providers leveraging expertise in specific areas to become the search engines of choice for specific audiences. So while we are celebrating New Year's messages of unity in other contexts, in 2006 the data publishing industry should be welcoming -- and profiting from -- fragmentation.