Open Data Opens New Competitive Front

Recently signed into law, the Foundations for Evidence-Based Policymaking Act is going to have a big impact on the data business. It contains within it provisions to open up all non-sensitive databases, and make them easily available in machine-readable, non-proprietary formats. Moreover, every federal agency is now obliged to publish a master catalog of all its datasets in order to make them more readily accessible.

Federal government databases are the gift that keeps on giving. Because they are generally the result of regulatory/compliance activity by the government, they are quite complete, and the data quite trustworthy. Moreover, the great shift online has made it easier for government agencies to require more frequent data updates. And with more data coming to these agencies electronically, the notoriously bad government data entry of years past has largely disappeared. Best of all, you can obtain these databases at little or no charge to use as you please.

However, this new push for open formats and is a two-edged sword. Many of the great data companies that have been built in whole or in part on government data got significant advantage from the complexity and obscurity of that data. Indeed, government data has been open for decades now – you just needed to know it existed, what it was called and who to talk to in order to get your hands on it. This was actually a meaningful barrier to entry for many years.

While it won’t happen overnight, increased data transparency and availability is likely to create a new wave of industry disruption. These government datasets are catnip to Silicon Valley start-ups because these companies develop software and don’t have the skills or interest to compile data. “Plug and play” data will assuredly attract these new players, and they will cause havoc with many established data providers.

How do you fight back against this coming onslaught? The key is to understand the Achilles heel of these companies. Not only don’t these companies tend to understand data, most of them actively dislike it. That means that you can find competitive advantage by augmenting your data with proprietary data elements or even other public data that might need to be cleaned and normalized. Think about emphasizing historical data, which is often harder for new entrants to obtain. These disruptive players will win every time if the battlefield is around the user interface or fancy reports. Change the battlefield to the data itself, and the advantage shifts back to you.

Marketing for Dummies

The composition of my email inbox has changed dramatically over the last several months, and it’s given me fresh insight into how data is being used by marketers. Apparently, contact data has found increased importance as the raw material needed to power marketing automation software.

Every day now, I am accosted not with simple email solicitations, but email campaigns, all relentlessly determined either to trick me into a conversation with a salesperson, or turn me into a customer by grinding me into submission through endless messaging. Marketing automation technology is widely being used as a “fire and forget” weapon. Load in a series of messages, load in a mailing list, and watch the leads roll in.

Marketing automation platforms do in fact offer a sophisticated new approach to marketing. But where things go wrong is that customers are expected to supply the sophistication, not the software. The two main areas of abuse:

  • Trying to fake a relationship in order to encourage a response. You’ve probably seen them: the carefully worded emails written to imply you’ve had previous contact with the sender. Should you fail to respond, you keep getting more emails (each with the full email chain), all written to make you feel as if you dropped the ball at some point, with the hope that concern, confusion or guilt will push you to engage. I have just one question about this: have you really created a qualified prospect by getting someone to contact you under false pretenses? And since for this deception to work, the emails need to look personal, that means no CAN-SPAM compliant opt-out link. You’re going to receive these emails until the sender gets tired of sending them. 

  • Blasting out repeated messages to an unqualified list. Do I really need to repair the roof on my office building? There are plenty of clues (starting with my industry classification code) to suggest you are wasting your time. Ditto that for robots to automate my factory. Offer the average marketer 100 perfectly qualified in-market leads or 10,000 lightly qualified contacts, and the sad fact is that the majority will take the big list every time.

My simple point in all this is that even with vastly improved data and state-of-the-art tools, most marketing people use it only to push more stuff out faster. Yes, even in 2019, marketers still talk targeting but buy volume, and this translates to their data buying practices as well. As an industry, we can offer our customer so much more. Unfortunately,  there are still too many people doing marketing for dummies. 


Nice Try, Moody's

For over a decade I have watched with interest as a company called CoStar became the largest player in commercial real estate data. It achieved this feat – and a market cap of over $13 billion – by old-fashioned data compilation, well-timed acquisitions, and aggressive litigation to keep competitors at bay.

What resulted is an effective monopoly in the commercial real estate space. CoStar achieved this by never cutting corners on primary data collection. As just one example, at one point it had a fleet of trucks snapping photos of every commercial property in the country. CoStar was never shy about getting on the phone too, collecting valuable leasing data from brokers and property owners nationwide. Marry all that proprietary data with public records data and a strong online platform, and you have a business that is highly profitable and nearly impregnable.

 Data companies in this privileged position do sometimes suffer at the hands of competitors, but nin times out of ten, it’s because of self-inflicted damage. Companies that become data monopolies have to be endlessly vigilant about not becoming arrogant or charging extortionate prices, because being hated by your customers provides an opening for competitive players. So too does complacency, and a failure to invest in the business to enhance the value of its products and keep up with changing market needs.

It doesn’t seem that CoStar has made any of these mistakes, but it is feeling new competitive heat anyway from another information giant, Moody’s (market cap $31 billion).

Moody’s (through its Moody’s Analytics division) has never been a big player in commercial real estate data, but having decided it wants a piece of this market, it has been spending heavily on acquisitions to buy its way in. The centerpiece of its acquisitions was the $278 million purchase of commercial real estate analytics firm REIS last year. Moody’s also made “strategic investments” in a number of other industry data providers.

So is it curtains for CoStar? I think not. Moody’s has spent huge amounts of money to position itself to compete for only a small portion of the market CoStar serves (think banks and real estate investors). Moreover, Moody’s will be in large part dependent on data it doesn’t own, sourced from companies selling into the same market, meaning that a lot of the data Moody’s will offer will come heavily restricted. 

Perhaps most importantly, CoStar’s proprietary data (commercial real estate inventory and listings data) remains proprietary and untouchable. My take is Moody’s has over-spent for the opportunity to enter a bruising battle with an established company whose smarts and street-fighting skills are well established. Moody’s will build a business here, but it will be one much smaller than its ambitions, and one that will take relatively little revenue from CoStar. Data franchises are strong and it usually takes more than a large checkbook to bring them down.

Choose Your Customer

From the standpoint of “lessons learned,” one of the most interesting data companies out there is TrueCar.

Founded in 2005 as, TrueCar provides consumers with data on what other consumers actually paid for specific vehicles in their local area. You can imagine the value to consumers if they could walk into dealerships with printouts of the lowest price recently paid for any given vehicle. 

The original TrueCar business model is awe-inspiring. It convinced thousands of car dealers to give it detailed sales data, including the final price paid for every car they sold. TrueCar aggregated the data and gave it to consumers for free. In exchange, the dealers got sales leads, for which they paid a fee on every sale.

 Did it work? Indeed it did. TrueCar was an industry disruptor well before the term had even been coined. As a matter of fact, TrueCar worked so well that dealers started an organized revolt in 2012 that cost TrueCar over one-third of its dealer customers.

The problem was with the TrueCar model. TrueCar collected sales data from dealers then essentially weaponized it, allowing consumers to purchase cars with little or no dealer profit. Moreover, after TrueCar allowed consumers to purchase cars on the cheap, it then charged dealers a fee for every sale! Eventually, dealers realized they were paying a third-party to destroy their margins, and decided not to play any more.

TrueCar was left with a stark choice: close up shop or find a new business model. TrueCar elected the latter, pivoting to a more dealer-friendly model that provided price data in ways that allowed dealers to better preserve their margins. It worked. TrueCar re-built its business, and successfully went public in 2014.

A happy ending? Not entirely. TrueCar, which had spent tens of millions to build its brand and site traffic by offering data on the cheapest prices for cars, quietly shifted to offering what it calls “fair prices” for cars without telling this to the consumers who visited its website. Lawsuits followed.  

There are four important lessons here. First, you can succeed in disrupting an industry and still fail f you are dependent on that industry to support what you are doing. Second, when it comes to B2C data businesses, you really need to pick a side. Third, if you change your revenue model in a way that impacts any of your customers, best to be clear and up-front about it. In fact, if you feel compelled to be sneaky about it, that’s a clue your new business model is flawed. Fourth, and I’ve said it before, market disruption is a strategy, not a business requirement. 

Getting From A to B

When I started in the data publishing business decades ago, information products were largely paper-based (think directories), and the selling of information products was largely paper-based as well (think direct mail). Fast forward to today, and now we’re mostly selling online subscriptions via online marketing, and everyone is better off for it, or so it would seem.

Yet in the great shift from offline to online marketing, what didn’t seem to shift over were all the people who really understood offline marketing. These people tended to know their stuff, for the simple reason that direct mail was expensive. Too many mistakes and you would be out of a job … or out of business.

As a result, the development of online marketing canon was a tabula rasa exercise.  I still vividly remember sitting in a seminar for online marketers in 1999 as the speaker described an extraordinary new marketing concept: in order to find the best price for his product, he had split his list in two and sent each half the same offer but with different price points. He said the concept could be used dozens of different ways, and because it was new there wasn’t even a name for it. As dozens of online marketers from household name companies furiously scribbled notes, I remember thinking that one possible name the group might want to consider was “A/B testing.” These young marketers were so convinced that what they were doing was so new and so different it never occurred to them to explore what had been learned before they arrived on the scene.

Sure, online marketing has come a long way in the last 20 years, and there are now aspects of online marketing that don’t have any offline parallel. But the basics live on.

In talking to the pricing research experts at TRC, folks whose deep knowledge of market research never fails to impress, I learned of a recent study conducted by researchers at Stanford and the University of Chicago. It sought to quantify the value of adding personalization to email messages. The results were stunning: the research found a 21% lift in email opens, a 31% lift in the number of inquiries, and as a bonus, a 17% drop in the number of unsubscribes. Online gold! But, just for the record, personalization delivered magical results in offline direct mail as well, so while these research results are good news, at the same time they’re not really new news. 

Yet, one recent study finds that while 81% of online marketers claim they send personalized email, only 3% of consumers feel they regularly receive personalized email. The discrepancy comes from the difference between personalizing an email and effectively personalizing an email. The best online marketers know that there’s more to it than just dropping a name in the email somewhere.

How do you figure out what’s effective? Testing, endless testing, having a good research methodology (such as not testing multiple things in one email), and monitoring and recording results carefully. Not sure where to start? Well, you might consider this new thing — it’s called an A/B test.