Survey Says ... It Depends

The data for data products can come from a wide array of sources. Traditionally, datasets were compiled through primary research, usually via questionnaires or by phone. There is alsosecondary research, where staff gathers data using online sources. There are also public domain databases that can be leveraged. We have also seen a rise in technologically-driven data gathering, such as web harvesting. And a growing number of data publishers license third-party data to augment their data gathering. Almost anything goes these days, and the savviest data publishers are mixing and matching their collection techniques for maximum effectiveness. (a topic that will be addressed at the Business Information and Media Summit in November. )

This brings me to a question I have been asked more than a few times: can survey data be turned into a data product? When I talk about surveys, I mean the types of surveys most of us do routinely: you ask, say, 20,000 restaurant owners to answer questions about their businesses and the market generally, and if you’re lucky, you’ll get 1,000 responses. My take? While a survey does in fact generate data, I don’t think a survey automatically qualifies as a commercial data product. The reason is subtle, but important.

Much of the value of a data product is in its granularity and specificity. Typically, a data product focuses on organizations, individuals or products and attempts to collect as much detail as possible on each unit of coverage, as comprehensively as possible. Most surveys, by contrast, are anonymous by nature and hit-and-miss in coverage. Using our earlier example, a survey of restaurants might well be useful and valuable if it didn’t get any response from Taco Bell operators. A restaurant database without any listings of Taco Bell locations would have no credibility.  Since most surveys promise anonymity to increase survey participation rates, only aggregate reporting is possible. From my perspective, surveys of this type are useless as data products.

But not all surveys are the same. Some surveys ask respondents to list the vendors they use, or which of a specified set of companies they like the most and the least. Surveys where you ask the anonymous respondent to list or opine on specific companies or products actually can yield a very compelling type of commercial data product. That’s because the companies or products that come out of the survey effort are not anonymous. If the owner of the Blue Duck restaurant tells you that she likes National Restaurant Supply, you’re developing lots of valuable data about National Restaurant Supply that you can publish, even while keeping Blue Duck restaurant anonymous. Your survey data can report on attitudes or adoption or market share of specific products or firms and compare them and rank and rate them. That’s very valuable because the data are highly proprietary, difficult to collect and actionable.

My bottom line on surveys is that “traditional format” surveys with anonymous submissions and aggregate reporting are truly surveys, not data products. But if your survey asks respondents to tell you how much they use or like specific companies or products – you’ve got yourself the makings of a data product!

Bigger Is Not Always Better

One key dynamic of the data business is that the strongest businesses serve single, tightly-defined markets, typically a single vertical market. The result is that the market opportunity tends to be smaller, but it is much easier to stay close to and defend.

The problem for data publishers attempting to build products with horizontal coverage across multiple markets, or who want to play in large consumer markets, comes down to a very simple reality: it’s hard to be everything to everybody.

It’s instructive to look at some of the reasons why it’s so hard to achieve long-term success with broad-based data products:

Lowest common denominator: In order to operate efficiently, broad-based data publishers typically have to collect fairly standardized and fairly shallow data across multiple vertical markets. This creates an opportunity for other data publishers to “slice and dice” these publishers, peeling off the largest and most profitable vertical sub-markets, and serving the same need with deeper and more tailored data.

Greater incentive for competitors: If you achieve any level of success with a horizontal, broad-based data product, you’ve not only identified a big market need, you’ve identified a big market opportunity as well. That means it may well be worth it for a competitor to invest significantly to steal market share or push you out entirely. Contrast this with successful vertical market data publishers, where the small scale of the market is one of their best protections. Competitors typically can’t financially justify trying to push their way into small vertical markets.

Turning an ocean liner: In addition to being a juicy competitive target, an established broad-based data publisher typically succeeds because it has built an operation that over time becomes very difficult to change for technical and business reasons. That means it will be at the mercy of such forces as new technology, shifts in user preferences and new business models, and just a few competitive successes can break the momentum and market dominance of the incumbent data provider. Moreover, the incumbent data responder is only able to react slowly, if it can react at all.

Too cool for school: While some broad-based data publishers become exposed because they can’t react quickly, others expose themselves by innovating so aggressively they get ahead of their markets and their customers. In a relentless quest to stay relevant and ahead of the competition, these publishers roll out features and functionality that their customer often don’t understand or even want, adding complexity to the user experience while muddying the core value proposition.

Platform envy: Perhaps encouraged by the spectacular success of Amazon, it’s easy to take the view that your data product can become a data platform, a way to distribute all kinds of data, products, whatever. That’s a big leap technologically, and while platforms are enticing to publishers, they almost inherently mean diffused focus, thus opening opportunities for competitors to enter the market with more focused products.

The most successful data publishers and products I see these days tend to serve one market and serve it extremely well. As long as these businesses stay close to their customers, evolve their products regularly and prudently, and offer good customer support and fair pricing, they can be enormously profitable while remaining largely immune to competition. That’s why in the data business at least, bigger isn’t always better.

The Roomba Ruckus

Roomba, the robot vacuum cleaner that took advanced technology and applied it to the consumer market by trying to eliminate the lowly task of vacuuming, has been in the news recently. Apparently, its devices suck up more than the dirt in your home: they are sucking up data about your home as well. And Roomba is starting the think about selling this trove of data.

There are several aspects to this development that merit discussion. First, of course, there’s the privacy issue. Roomba was forward-thinking to the extent that it buried appropriate language in its privacy agreement that allows it to do pretty much anything to the data it collects. However, that language wasn’t prominent and was written in legalese. In short, while it may be legal for Roomba to sell customer data, it wasn’t up-front and transparent with its customers.

Right now, most pundits are saying that convenience trumps privacy every time. That may be true currently, but I expect consumer attitudes will begin to shift as the nature and extent of furtive data collection fully penetrate the collective conscience.

Exactly what data does Roomba collect and how valuable is it? I have said many times that not all data are valuable, and while Roomba certainly has a trove of data, I am not convinced it is a treasure trove of data. Many articles on the subject talk breathlessly about this goldmine of “room geometry” data. Specific potential uses (of which very few are mentioned – a big clue right there) are such things as designing speaker systems. Sounds legit, but can Roomba tell you the ceiling height of the room? Can it tell you what rooms play music now? There are lots of clues that these data may not in reality be all that useful.

And who would buy these data? The articles are equally breathless on this subject, suggesting that of course Amazon would want it. Others suggest Apple will snap it up, and perhaps Home Depot as well. If you step back, all you see is a list of big companies with products for the home.

The increasingly common view that every company, including manufacturers, is expected to have a data strategy, is trendy, silly and will ultimately collapse. Not all data are valuable, and having huge quantities of not-valuable data doesn’t change that fact. And when you consider that to gather these data you risk a privacy backlash and reputational damage, companies (and those who fund them) will ultimately start to realize that not all data are created equal. Only a fortunate few can casually generate high-value datasets, and even then, it’s not cost or risk free. My prediction: Roomba won’t be cleaning up with data anytime soon.

Good Databases Are More Than Just Good Data

We can look to the UK for a case study of how a government agency, after several tries, couldn’t build a user-friendly data product, creating a giant opportunity for a for-profit data company.

The story begins with a regulatory agency called the Financial Conduct Authority (FCA) that among other duties, registers and regulates financial advisors and advisory firms. The FCA has a searchable database on its website, but like so many government websites, it is optimized for one purpose: checking the registration status of a known individual or firm. As a tool to assist you in identifying an advisor to help you with your investments, it’s pretty useless.

In recognition of this shortcoming, the FCA called on a quasi-governmental organization called the Money Advice Service (MAS) to help build a better adviser database, and MAS accepted the challenge. I took a look at this website when it first launched, and though I saw some design issues, it had potential.

But even though MAS nominally had the freedom to build a creative database with almost any business model behind it, its need to avoid controversy ultimately resulted in a very limited and timid product. And when, unsurprisingly, there wasn’t a lot of revenue to be had with such a product, MAS buried the database three levels down on its website and moved on to greener pastures.

With two free databases of financial advisers out there, you think there wouldn’t be much opportunity left for anyone. However, a company called Unbiased saw things differently, and said there was indeed an opportunity … for the right product.

Unbiased has been a big hit in the marketplace, and the way it differentiated itself from the free government services with the same basic listing data holds lessons for us all

• Greater visibility – Unbiased wants to be found because its business model depends on driving lots of traffic to its participating advisers
• Deeper data – ratings, discount offers and detailed profiles
• Strong user interface – clean, inviting design and both parametric search and a custom matching service         

If you have ever wondered how you could compete against a free, government online database, Unbiased provides the answer: data presentation can be as valuable as the underlying data itself, particularly if you are serving a consumer market. And aggressive promotion of your online database will let you run circles around government agency databases, that are generally hard to find in addition to being hard to use. 

Inferring Intent

Today’s Gartner blogpost points to some interesting limitations and opportunities surrounding intent data. Let’s start at the beginning by defining what it is.

Simply put, intent data is an indication that an individual or organization is actively interested in purchasing a specific product or service. You may already be familiar with sales triggers. One classic sales trigger is so-called “new move” data. It’s valuable to know when a company moves offices because it is highly likely that the company will likely make lots of new purchases such as office furniture and the like. Think of intent data as a more sophisticated cousin of the sales trigger.

Media companies are in a great position to generate sales intent data, because much intent data is generated by watching what a person reads and does online. If a reader looks at five articles on 3-D printers in a short period of time, those actions can be viewed as indicating an intention to purchase a 3-D printer. Intent data can get a lot more sophisticated than that, but this gives you the general idea.

You might think that if a sales organization has intent data available to it, that’s probably all the data it needs. After all, intent data is like mind-reading: it’s identifying people who are likely to be purchasing a product before they purchase it. What could be better?

Well, as the Gartner blogpost points out, many companies are filtering sales leads based on intent data with something called “fit analysis.” This is an automated attempt to evaluate if the company is a likely buyer. If your company typically sells to larger, multi-office organizations, a fit analysis will filter out smaller, single location companies because they represent lower grade prospects.

Further, the Gartner blogpost notes that companies selling highly specialized products or brand-new technologies often can’t get enough intent-based sales leads or they get leads that are weak because the intent indicators aren’t sufficiently granular. Finally, some sales departments don’t like intent-based sales leads because they identify prospects too early in the sales process. As you can see, sales leads based on intention are still fairly rudimentary, and there is lots of opportunity to refine them.

But what’s most worthy of note is that Gartner believes that most intent-based sales lead data is focused on the technology industry. But there is no reason that it should. Technology sellers just happen to be free-spending early adopters. I have long preached the virtues of what I call “inferential data,” a term that includes both intent and sales trigger data. I firmly believe that many data publishers have opportunities in this area, and if they happen to be part of larger media companies, they are even greater. In fact, data publishers are natural providers of fit analytics as well. If you look at your data creatively and read between the lines you can make some very lucrative connections.