This Score Doesn't Compute

This week the College Board, operators of the SAT college admissions tests, made a very big announcement: in addition to its traditional verbal and mathematic skills measurement scores, it will be adding a new score, which it is calling an “adversity score.”

In a nutshell, the purpose of the adversity score is to help college admissions officers “contextualize” the other two scores. Primarily based on area demographic data (crime rates, poverty rates, etc.) and school-specific data (number of AP courses offered, etc.) this new assessment will generate a score from 1 to 100, with 100 indicating that the student has experienced the highest level of adversity.

Public reaction so far has been mixed. Some see it as an honest effort to help combat college admission disparities. Other see it is a desperate business move by the College Board, which is facing an accelerating trend towards college adopting test-optional admission policies (over 1,000 colleges nationwide are currently test-optional).

I’m willing to stipulate that the College Board had its heart in the right place in developing this new score, but I am underwhelmed by its design and execution.

My first concern is that the College Board is keeping the design methodology of the score secret. I find that odd since the new score seems to rely on benign and objective Census and school data. However, at least a few published articles seemed to suggest that the College Board has included “proprietary data” as well. Let the conspiracy theories begin!

Secondly, the score is being kept secret from students for no good reason that I can see. All this policy does is add to adolescent and parental angst and uncertainty, while creating lots of new opportunities for high-priced advisors to suggest ways to game the score to advantage. And the recent college admissions scandal shows just how far some parents are willing to go to improve the scores of their children.

My third concern is that this new score is assigned to each individual student, when it is in reality a score of the school and its surrounding area. If the College Board had created a school scoring data product (one that could be easily linked to any student’s application) and sold it as a freestanding product, there would likely be no controversy around it. 

Perhaps most fundamentally though, the new score doesn’t work to strengthen or improve the original two scores. That’s because what it is measuring and how it measures is completely at odds with the original two scores. The new score is potentially useful, but it’s a bolt-on. Moreover, the way this score was positioned and launched opens it up to all the scrutiny and criticism the original scores have attracted, and that can’t be what the College Board wants. Already, Twitter is ablaze with people citing specific circumstances where the score would be inaccurate or yield unintended outcomes.

Scores and ratings can be extremely powerful. But the more powerful they become, the more carefully you need to tread in updating, modifying or extending them. The College Board hasn’t just created a new Adversity Score for students. It’s also likely to have a caused a lot of new adversity for itself.



On May 4, 2019 it’s official: connect is shutting down. You may remember connect in its original incarnation as acquired Jigsaw in 2010, paying huge dollars to kick-start an ambitious plan to not only be a software platform to manage sales activities, but to help companies maintain and grow their sales leads as well.

Lest you think Salesforce was lacking in ambition, it then acquired the domain name for $1.5 million. Jigsaw moved over to, and Salesforce began to execute on its vision of a data marketplace, where its software users could discover, purchase and seamlessly import third-party data into Salesforce. It was a big, slick and arguably brilliant idea.

But an idea falls far short of a successful strategy, and never appeared to be much more than an idea, or more accurately, a series of ideas. And Salesforce, for all its success, never figured out decisively what it wanted to be when it grew up. Add in competing corporate strategies, office politics, a high-growth core business and a go-go culture, and it’s perhaps not surprising that quickly became a corporate orphan.

 More fundamentally though, we see once again that software companies – despite lots of brave talk – just don’t “get” data. In particular, a good database needs care and feeding using processes and techniques that are messy, imperfect, never-ending and perhaps most importantly of all, impossible to simply automate and forget.

 Jigsaw probably looked like a light lift to Salesforce. After all, the brilliance of Jigsaw was it was crowd-sourced data. The people using the data committed to correcting it and adding to it. On the surface, it probably looked like a perpetual motion machine to Salesforce. But that perception couldn’t be farther from the truth. Crowdsourcing is an intensely human activity, because you have to motivate and incent users to keep working on the database. You have to construct a structure that rewards top producers and pushes out bad actors. You have to relentlessly monitor quality and comprehensiveness. It’s endless fine-tuning, lots of trial and error, and a deep understanding of how to motivate people.

This is where Salesforce failed. It either didn’t understand the commitment required or didn’t want to do the work required. And just as a crowdsource database can grow quickly, it can also decline quickly.

I’ve said it before: I see more success among data providers that develop software around their data than software companies trying to develop their own databases.

1 Comment

Does B+C=E?

When Steve Lucas talks, people listen. After all, Steve was the founder of Marketo, which was sold to Adobe last year for $4.8 billion. Now the SVP of Digital Experience at Adobe, Steve gave a talk at the recent Adobe Summit in Las Vegas to describe his vision of the future, which revolves around B2B and B2C merging to become B2E – Business to Everyone.

 Steve acknowledged that B2C is all about individuals in their personal buying capacities, and that B2B is about accounts – typically groups of buyers and influencers within an organization, so he understands where and how B2B and B2C differ. But he then cites Amazon as a company that is both a B2C seller (think Whole Foods) and a B2B seller (think Amazon Web Services). Steve thinks there are lots of B2C/B2B sellers out there, and that there is a great unmet need for a single, integrated marketing platform. May I politely disagree?

 Sure, there are always a few hubristic companies that are flush with cash and feeling their oats that will be willing to chase this inchoate vision of the future. But are there many of these companies? Most companies are still trying to get their B2B or B2C marketing straight. B2B marketers are only a few years into their embrace of Account-Based Marketing (a new name for a very old concept, but that simply reinforces my point) and B2C marketers are warily picking their way through a growing minefield of privacy and regulatory rules that actually creates a bias against being too cutting edge.

As to the information need, a simple question: why? In a B2E world, Amazon would indeed be able to know that, for example, the VP of Purchasing at Boeing likes organic lemons. But how does this knowledge advance the sale of either Amazon cloud services or Whole Food produce sales? Indeed, B2E creates the opportunity to build such detailed dossiers that the end results will toggle between scary and silly.

There are already B2B data companies that build detailed personal profiles of top corporate executives to help with high-level selling. That makes sense. Trying to do the same thing with even more granular detail and at scale doesn’t make sense.

B2E reeks of “wouldn’t it be cool if…” syndrome. Making databases bigger and more complex doesn’t inherently make them more powerful or useful. I remember a marketing database at AT&T many years back. It was state of the art for its time, centrally tracking every purchase of AT&T equipment, services and merchandise for 20 million customers. Then one day, an AT&T executive had an epiphany: not only should the database track what customers bought, it should also track everything they were offered but didn’t buy. Everyone thought the concept was brilliant, though at the same time nobody could articulate how all this new information might be used. The initiative involved building a huge new data center, and the sheer volume of data being collected and stored brought the system to its knees. The system became so complex and so slow that the marketing organizations stopped using it … for anything. The whole database was quietly abandoned a few years later. 

I can understand why Adobe would find B2E appealing, but it’s a mystery to me why anyone else would.

1 Comment

Sell Your Smarts

In the bygone days of print directories, one sales challenge stood out above all others: convincing buyers to regularly buy the new edition of the directory. The issue was that directory buyers weren’t easily convinced that enough had changed in the course of a year to justify purchasing the new edition. Everyone in the business back then knows about the nemesis that was the every-other-year buying pattern.

 The marketing remedy for this problem (a problem that sounds positively quaint today) was to “quantify change.” This typically meant using actual database counts to prove that enough had changed to justify purchase of a new edition. Rather than saying “thousands of updated addresses,” a vague and not very compelling statement, publishers would say, “23,418 updated addresses.” By quantifying change, publishers offered specificity to describe their updating efforts, something that proved both credible and compelling to data buyers.

 Fast forward to today when the marketing challenge has almost been completely reversed. Data buyers have radically increased their expectations, based on the belief that all online databases are updated in real-time and are completely current. It’s a belief based on wishful thinking and a fundamental misunderstanding of what the word “online” means. Of course, many data providers encouraged that thinking by saying that their databases were “updated daily.” That meant that they were making changes every day. To many data buyers, it meant that the entire database was refreshed and updated daily. It’s a disconnect, and it’s a big one.

 These raised expectations and technological misunderstandings have complicated the marketing strategies of most data publishers. After all, it’s hard to sell your database on the basis of it being complete and current when that’s a basic expectation of your prospects. For that reason, many data publishers now sell on the basis of the qualityof their data. But that’s a tough slog, because as I have said many times, quality is easy to claim and hard to prove. In a crazy way, hedge funds have it right: they buy databases like candy. They try them for a year or two, and if they can’t make money off them, they move on to the next database. That’s easy for hedge funds to do, but not the average business. Selling on quality is further complicated by the fact that some publishers don’t want to talk about the source of their data (for reasons good and bad), and some publishers who license data from reputable sources are contractually prohibited from disclosing this information. 

What’s a publisher to do? I think marketing strategy today has to be based on deep market knowledge. Too many data purveyors these days (especially the disruptive variety) are aggregators or packagers. Aggregators tend to focus on building the biggest datasets, and usually emphasize quantity over everything else. Packagers take a dataset (typically public domain data) and create a fancy user interface to add value. What’s lacking in both cases is the market knowledge that would allow them, for example, to confidently drop out unqualified records as opposed to blindly selling whatever they can get their hands on. It’s also about building user interfaces that meet real industry-specific needs as opposed to generic searching and reporting.

Business is getting more complex and specialized all the time. And I see time and again that the most successful data publishers come out of the industries they serve and build databases to solve real and painful business problems that they experienced themselves. Artificial intelligence and fancy algorithms are all well and good, but their answers can’t be any better than the underlying data on which they operate. Think of data companies such as PitchBook and CompStak.

To succeed in the data business today, you need to sell your smarts. That means you have to demonstrate you know your market and the needs of your market better than anyone, and that you have built a dataset uniquely capable of addressing those needs. Fancy sales triggers are wonderful, but if they are monitoring the wrong companies or the wrong events, they’ll produce more noise than signal, defeating their purpose.

Going forward, it won’t be about having the most companies. It will be about having the rightcompanies. It won’t be about having the most data elements. It will be about having the rightdata elements. And you determine both by deeply understanding how your market works and what business problems need to be solved. Determine how to demonstrate your market knowledge and you’ll have a winning marketing strategy that will be effective for years to come. 


Open Data Opens New Competitive Front

Recently signed into law, the Foundations for Evidence-Based Policymaking Act is going to have a big impact on the data business. It contains within it provisions to open up all non-sensitive databases, and make them easily available in machine-readable, non-proprietary formats. Moreover, every federal agency is now obliged to publish a master catalog of all its datasets in order to make them more readily accessible.

Federal government databases are the gift that keeps on giving. Because they are generally the result of regulatory/compliance activity by the government, they are quite complete, and the data quite trustworthy. Moreover, the great shift online has made it easier for government agencies to require more frequent data updates. And with more data coming to these agencies electronically, the notoriously bad government data entry of years past has largely disappeared. Best of all, you can obtain these databases at little or no charge to use as you please.

However, this new push for open formats and is a two-edged sword. Many of the great data companies that have been built in whole or in part on government data got significant advantage from the complexity and obscurity of that data. Indeed, government data has been open for decades now – you just needed to know it existed, what it was called and who to talk to in order to get your hands on it. This was actually a meaningful barrier to entry for many years.

While it won’t happen overnight, increased data transparency and availability is likely to create a new wave of industry disruption. These government datasets are catnip to Silicon Valley start-ups because these companies develop software and don’t have the skills or interest to compile data. “Plug and play” data will assuredly attract these new players, and they will cause havoc with many established data providers.

How do you fight back against this coming onslaught? The key is to understand the Achilles heel of these companies. Not only don’t these companies tend to understand data, most of them actively dislike it. That means that you can find competitive advantage by augmenting your data with proprietary data elements or even other public data that might need to be cleaned and normalized. Think about emphasizing historical data, which is often harder for new entrants to obtain. These disruptive players will win every time if the battlefield is around the user interface or fancy reports. Change the battlefield to the data itself, and the advantage shifts back to you.