I am just now reading a new book called Click by Bill Tancer, who is head of research at web analytics company Hitwise. The subtitle of the book, "What Millions of People Are Doing Online and Why it Matters," gives you a good sense of what the book is about. The underlying premise: that people's online search patterns are not only telling, but are a far more accurate way to get at people's real interests and activities than conventional surveys.
The book and its premise build on a termed coined by John Battelle way back in 2003 called "the database of intentions." Batelle defines it as:
The aggregate results of every search ever entered, every result list ever tendered, and every path taken as a result. It lives in many places, but three or four places in particular hold a massive amount of this data (ie MSN, Google, and Yahoo). This information represents, in aggregate form, a place holder for the intentions of humankind - a massive database of desires, needs, wants, and likes that can be discovered, supoenaed, archived, tracked, and exploited to all sorts of ends.
As someone who has spent more time examining server log files than I care to admit, I know first-hand the power of this concept. It is possible to glean from search activity what users, individually and in the aggregate, are thinking and doing. That's not as scary as it might sound, because online searching is still largely anonymous, and search patterns are often more useful in aggregate.
We're going to see a lot more activity around competitive intelligence built on analysis of search activity, and it's not too early to think about your own site traffic as a source of such intelligence. Consider as just one example Kelly Blue Book, which in a presentation at the 2006 InfoCommerce Conference detailed how it had built a substantial subscription business selling to auto manufacturers data on how many consumers had looked at what car models in what geographies.
And if you're interested in a quick demonstration of the power of "search intelligence," pay close attention to the Google search box. Depending on the search term you enter, you'll see not only similar search terms entered by others, but a count of the number of times each search term was entered. Want to assess a market? Fine-tune your paid search program? See how many people are querying a competitor's website? It's all there for you, fast, free and at your fingertips.
As the example above shows, you don't have to search at all for these search insights.