Data Flipping

One of the best things above government databases is that even when the government agency makes the database available on its website for free, it isn’t very useful. That’s because government agencies put these databases online for regulatory or compliance reasons.  They’re designed to search for known entities because the expectation is that you are checking the license status of a company, or perhaps its compliance history.

Occasionally, a government agency will get ambitious and permit geographic searches, but in these cases, there are real limitations. That’s because the underlying data were collected for regulatory, not marketing purposes. So, for example, a manufacturer with 30 plants around the country may only appear in one ZIP code because the government agency wants filings only from headquarters locations.

Taking a regulatory database and changing it into, say, a marketing database, is something I call “flipping the file,” because while the underlying data remains the same, the way the database is accessed is different. Sometimes this is as simple as offering more search options; sometimes it involves normalizing or re-structuring the data to make it more useful and accessible. As just one example, a company called Labworks built a product called the RIA Database. It started with an  investment advisor database that the SEC maintains for regulatory purposes, and then flipped the file to make the same database useful to companies that wanted to market toinvestment advisors.  There are hundreds of data publishers doing this in different markets, and as you might expect, it’s a very attractive model since the underlying data can be obtained for free.

In addition to simply flipping a file, you can also enhance a database. The shortcoming of many government databases is that they focus on companies, not people, so while there may be a wealth of information on the company, data buyers typically want to know the names of contacts at those companies. Companies such as D&B and ZoomInfo do a brisk business licensing their contact information to be appended onto government databases of company information.

This is one of the truly magical aspects of the data business. Databases built for one reason can often be re-purposed for an entirely different use. And re-purposing can involve something as little as a new user interface. This magic isn’t limited to government data of course. Another great place to look for flipping opportunities is so-called “data exhaust,” data created in the course of some other activity, and thus not considered valuable by the entity creating it. You can even license data from other data providers and re-purpose it. There are a number of mapping products, for example, that take licensed company data and essentially create a new user interface by displaying data in a map context.

Increasingly, identifying the data need is as important as identifying the data source. With data, it’s all in how you look at it. 



Searching for a Better Recommendation Engine

My first experience with recommendation engines was with Amazon in its early days. Then, when you bought a book, Amazon would tell you that people who bought the same book had also bought these other books. It was simple, brilliant, and most importantly, it worked. When Amazon later started selling CDs, the recommendation engine worked even better. I got to enjoy music I never knew existed, and Amazon sold more CDs. It’s a classic win-win, and you would think Amazon would put its substantial resources into making its recommendations even better. 

But apparently not. After buying an introductory book on Photoshop a while back, the recommendation engine started showing me every Photoshop book ever written (there appear to be hundreds of them), and crowded out every other book recommendation for nearly a year. These were lazy recommendations, and disproportionate to the one book I bought – ever – on a specific topic. And Amazon recommendations have gotten even lazier since then.

You may also recall the Netflix Prize, announced with great fanfare back in 2008. A $1 million prize was given to anyone who could improve the efficacy of the Netflix recommendation engine. It was an impressive commitment by Netflix, and it showed they deeply understood the importance and value of recommendations to their business. Fast forward to today. Having watched every single episode of Arrested Development on Netflix, how did I learn about the arrival of new episodes? I read about it in the newspaper. Has Netflix brought these new episodes to my attention? Not yet. Somewhere along the way, Netflix seems to have stopped caring about the quality of its customer recommendations.

Move over to the search engines – all of them. You may know that you can force a search engine to search for a specific phrase by putting quote marks around it. Typically, your first search results will be web pages containing that exact phrase. But then the search engines actually remove the quote marks and toss in results that have the requested search terms, but not necessarily together. Then they toss in pages that have some but not all of your search terms. Since I didn’t ask for these search results, I think it’s fair to consider them as recommendations. And they are (predictably) lousy. It’s as if the search engines assume I don’t know what I am doing, so they give me every possible type of result. Yes, more is better with search engines, but only if they are giving me more of what I want. 

Contrast this with the music service Pandora that I’ve been raving about since 2007. Despite a tough revenue model, Pandora has not forgotten that it lives and dies by the quality of its recommendations, and it’s built to over $1 billion in annual revenue by staying focused. Hopefully they'l maintain that focus as it continues to grow.

When companies get big, it’s very easy for them to get distracted and lose interest in what made them big in the first place. There are more voices now saying that Google search quality is in decline. And remember when Yahoo got bored with search and decided to outsource search while it chased bigger dreams? These distractions create opportunities for smaller players to do search better, and some are finding success.  


Being in the Middle of a New Data Product

I’ve written before about the application model called the “Closed Data Pool.” In this model, companies (and many times they are competitors) contribute proprietary data to a central, neutral data company. The data company aggregates the data and sells aggregate views of the data back to the very companies that contributed it. Madness you say? Not really, because these companies get great benefit from those aggregated views (think market share, average pricing and other vital business metrics). It’s the neutral, trusted data provider in the middle who makes it possible. 

But there is another twist on the closed data pool that represents an even more profitable business for the data provider in the middle. Consider a company called The Work Number.

The Work Number came into being because a lot of credit grantors need to be able to quickly verify employment status and income. At the same time, companies hated getting an endless stream of calls from creditors seeking to verify employment data. The Work Number came up with an ingenious solution. It went to big companies and said that they could outsource all these nuisance calls to The Work Number. All the company had to do was supply a feed of its payroll data. 

The Work Number then went to major credit grantors such as banks and said that instead of those painful verification calls they were making, credit grantors could just do a lookup on The Work Number website and instantaneously get the exact data they needed.

The best part? The Work Number was able to charge credit grantors for access to the database because of the big productivity gains it offered. But The Work Number was also able to charge the companies supplyingthe data because it increased their productivity as well by eliminating all these annoying verification calls. Yes, The Work Number charges both to collect the data and provide access to it!

If this sounds like an interesting but one-off opportunity to you, it’s not. Opportunities exist in vertical markets as well. Consider National Student Clearinghouse, which does the same thing as The Work Number, only with college transcripts.

Is there an opportunity in your market? Look for areas where relatively important or high-value information is being exchanged by phone or one-off emails or even by fax. If the information exchange constitutes a serious pain point or productivity drag for either or both parties, you’ve probably got a new data product. 

Standard Stuff Is Actually Cool

In the not-too-distant past, there was something close to an agreed-upon standard for the user interface for software applications. Promoted by Microsoft, it is the reason that so much software still adheres to conventions such as a “file” menu in the upper left corner of the screen.

The reason Microsoft promoted this open standard is that it saw clear benefit in bringing order out of chaos. If most software functioned in largely the same way, users could become comfortable with new software faster, meaning greater productivity, reduced training time and associated cost, and greater overall levels of satisfaction.

Back up a bit more and you can see that the World Wide Web itself represented a standard – it provides one path to access all websites that function in all critical respects in the same way. Before that, companies with online offerings had varying login conventions, different communications networks, and totally proprietary software that looked like nobody else’s software. Costs were high, learning curves were steep and user satisfaction was low.

There are clear benefits to adhering to high-level user interface standards, even ones that bubble up out of nowhere to become de facto standards. Consider the term “grayed out.” By virtue of this de facto user standard, users learned that website features and functions that were “grayed out” were inaccessible to them, either because the user hadn’t paid for them, or because they weren’t relevant to what the user was currently doing within the application. Having a common understanding of what “grayed out” meant was important to many data publishers because it was a key part of the upsell strategy.

That’s why I am so disappointed to see the erosion of these standards. On many websites and mobile apps now, a “grayed out” tab now represents the active tab the user is working in, not an unavailable tab. And virtually all other standards have evaporated as designers have been allowed to favor “pretty” and “cool” over functional and intuitive. I could go on for days about software developers who similarly run amok, employing all kinds of functionality mostly because it is new and with absolutely no consideration for the user experience. What we are doing is reverting to the balkanized state of applications software before the World Wide Web.

And while I call out designers and developers, the fault really lies with the product managers who favor speed above all, or who themselves start to believe that “cutting edge” somehow confers prestige or competitive advantage. Who’s getting left out the conversation? The end-user customer. What does the customer want? At a basic level the answer is simple: a clean, intuitive interface that allows them to access data and get answers as quickly and painlessly as possible. Standard stuff, and the best reason that being different for the sake of being different isn’t in your best interest.

The Low Hanging Fruit Hiding in Plain Sight

One of the unintended consequences of the rapid shift to sales force automation tools, CRM systems and large-scale lead generation campaigns is that things only work well when you target prospects and they respond to your promotions. It’s an outbound world now. Pity the poor prospect who unprompted calls you to buy something!

I have recently been in that position, having to make sales inquiries to data companies on behalf of clients. At first, I simply bemoaned the quality of salespeople these days. But then I realized it wasn’t the salespeople who were the problem; it was me! None of these companies had put any thought into how to handle an unsolicited lead, probably because they assumed it was a non-issue. But it’s a big issue. I consistently fell through the cracks because none of these companies had made any provision to deal with me. I didn’t fit their workflow.

The first thing you learn about being a buyer in this situation is that you better not be in a hurry. Callbacks to unsolicited leads in my recent experiences ranged from two to four days. And when I did get a response, it was often by a screener, charged with determining if my business was worth a salesperson’s time. Indeed, after being screened by one major data provider, I received a surprisingly curt email informing me that the size of my potential order didn’t merit their attention, but that my name had been passed along to one of their distributors, and I would hear from them in due course. I’m still waiting after three weeks.

I’ve also learned that using the phone doesn’t accelerate the buying process at all. In fact, it makes things worse. Two of the data companies I contacted had automated attendants that would helpfully connect me … but only if I already knew who I wanted to talk to. In one case, I actually reached a live person who answered the company’s main number. When I asked to speak to someone in Sales, I got the response I hear nearly 100% of the time: there are no salespeople in the office. When I asked to leave a message for someone in Sales, I got a long pause, followed by a very hesitant and somewhat dubious “sure, if you really want to.” One receptionist actually made the mistake of connecting me to someone in the sales department. I say “mistake” because the person answering the phone said he “wasn’t allowed to talk to me,” but he’d have someone call me back. When I said I needed some basic product information first, he did in fact provide it, after swearing me to secrecy because “I could get in a lot of trouble for doing this.”

Since companies have clearly abandoned the telephone as means of inbound contact, you think they would pay close attention to incoming leads by email. If only that was true! After submitting my sales inquiries to three companies via the ever popular “contact us” form, proving that I was not a robot, and in some cases being asked the size of my budget (required field), I sat back and waited. And then waited some more. One company responded fairly quickly, but the salesperson was apparently so incredulous that a sales lead would be unsolicited that I had to submit to a grilling via email to confirm my interest and my bona fides.

The second company responded three days later, and apologetically asked for lots of information about my product requirements and me so that he could “get me in the system.” Once properly in the company’s lead stream, I had a satisfactory buying experience.

The third company? Three weeks and I am still waiting on a response.

You surely know where I am going with this: with so much technology and so many resources being devoted to lead cultivation, generation and management, we seem to have forgotten about the most valuable sales lead of all: the unsolicited inquiry. There is apparently no place for them in our automated workflows.

Not your problem? I challenge you: complete the form on your own company’s “contact us” page and sit back and wait, not with a stopwatch but with a calendar. If you want an even more dismal experience, call your own company’s main number and ask to speak to a salesperson. Yeah, it’s that bad ... which means the opportunity for quick increased revenue is that good!