Models for Being the Best

There is endless innovation and variety in what I call the “best guides” segment of the market. These are guides, print and online, that help consumers find and select the best of something – from hotels to restaurants to contractors to consumer electronics.

It’s a huge market segment. Consider such vast scale businesses as Yelp and TripAdvisor that largely focus on restaurants and hotels respectively, though you can find on their platforms crowd-sourced reviews for just about anything. If you have huge ambitions, it’s pretty well established that the fastest way to develop massive amounts of content is via crowd-sourcing. If you build it, the crowd will comment on it and rate it for you. The downside to crowdsourcing has always been lack of control. Too many people posting ratings and reviews have malign motives. More fundamentally, two people can honestly have diametrically opposed opinions about a restaurant, for example, and it’s close to impossible to reflect this in an overall rating. Crowdsourcing depends on sheer volume for accurate reviews to drown out inaccurate and biased reviews. It works, more or less.

Knowing the limitations of crowdsourcing, there have been many who have tried to refine the concept. Angie’s List was an early pioneer, aggregating reviews but only fromits members and only forits members, what we call a “closed pool” model in our Business Information Framework. The concept worked, but was difficult to scale, in large part because members didn’t want to pay for ongoing memberships. Angie’s List has since shifted to a lead generation model. I also wrote in 2016 about a company called BestPickReports.com, that’s building its business both online and with expensive print guides mailed to consumers. It appears to be a long-term play, and is backed, somewhat surprisingly, by EBSCO.

 We all know about Consumer Reports. For decades, Consumer Reports was the first place to check before making a major consumer purchase such as a car or a dishwasher. Consumer Reports did all the testing and rating itself, and understood that its reputation was everything, so much so that it prohibited manufacturers from citing its reviews, and the owner was a non-profit organization. In the days of print, Consumer Reports did very well selling subscriptions to its print magazine. It wasn’t an ideal way to distribute information (how do you buy that new car in March when the new car reviews didn’t come out until the May issue?) but it worked for a long time because there weren’t a lot of options. That’s why Consumer Reports had some struggles when it moved online because consumers didn’t want an online subscription as much as they wanted to be able to buy just dishwasher reviews and only when they were in the market for dishwashers. Consumer Reports continues to flourish, the result of momentum, its pristine reputation and quality reviews, but it’s quite possible its business model will come under increasing pressure for the same reason as Angie’s List: selling a continuous information service to consumers who don’t continuously need information is just plain hard.

Finally, let’s look at the original arbiters of what’s good, better and best: newspapers and magazines. 

Hearst Magazines has a review site called BestProducts.com. While the name might imply product testing, the site recommendations appear to be closer to the traditional “editor’s picks.” There is heavy use of the phrase “what welike,” and the site overall seems to be much more about informed personal preferences of the writer – more taste-making than research. Indeed, aside from a great (and arguably misleading) domain name, these are product recommendations that would not look out of place as print magazine articles from ten or twenty years ago. Online forced a change in business model, however. Hearst links to vendors of all the items it recommends, hoping to profit from online referral fees.

 The New York Times blends a few models together through its Wirecutter.com site, a business it acquired in 2016. Wirecutter offers much more than the personal opinion model of Hearst, but less than the rigorous product testing of Consumer Reports. It walks a middle ground, doing real product research, but not actual product testing. In terms of business model, Wirecutter follows Hearst, generating revenue from product referral fees.

 Depending on product referral fees is a risky business because of “leakage.” Simply put, it’s too easy to take your recommendation but not click your link. When that happens, the business generates no revenue. The only real solution to the leakage problem is sheer traffic volume, something both Hearst and the New York Timesalready have and can easily leverage. The New York Times, for example, is increasingly citing Wirecutter in its own news stories, albeit with full disclosure of its ownership.

 There is no single best model, and here’s a rundown of the tradeoffs. Crowdsourcing works, and it is cheap, but the quality of the content is uneven. Closed pool crowdsourcing yields a huge step-up in quality, but it’s a tough model to execute. 

 You can generate your own reviews to guarantee the quality, but you have to fight the trend towards unbundling. Consumers will pay for reviews and recommendations, but only the ones they want when they want them. It’s tough to generate adequate revenue on that basis.

 Online referral fees are an inherently dicey business because it’s too hard to mask the name of the manufacturer, and there are far too many sellers, all a click away. You can make it work if you have gobs of traffic, and this is even a better business if you can leverage your existing traffic and not start from scratch.

 If I was trying to build a “best guide” site, I’d select Wirecutter as my starting point. It has the benefit of offering true product research without the huge testing costs incurred by Consumer Reports. It totally controls both the research process and resulting recommendations. It can leverage the brand and traffic of its parent, the New York Times. What would I change? First, I’d see if I could sell recommendations on an a la carte basis. Buying a dishwasher? Then buy our dishwasher reviews. I might also be able to generate some additional revenue from national retailers or manufacturers who could offer special deals along with the recommendations, though I would need to be careful to make it clear nobody had paid for a preferential rating. I’d ditch the referral fee model because it’s catnip for free-riders. Finally, I’d wrap the New York Timesbrand more aggressively around Wirecutter to reinforce the quality of the recommendations. I understand why the New York Times is moving cautiously here, but at some point, if you want to be in this business, you need to be in this business. You can’t hold it at arms-length. 

If you’re considering getting into the business, leverage your strengths, choose the right content and business model, and plan for the worst and hope for the best, because as of yet, there is no clear pathway to success in this huge and tantalizing area.

When Algorithms and Advertising Collide

You may remember when real estate listings firm Zillow first burst on the scene back in 2006. While there are many online real estate listings sites, Zillow distinguished itself with its “Zestimates,” an algorithmically-derived valuation for every house in the United States. Many Americans amused themselves throughout 2006 checking Zestimates for their own homes, as well as the homes of neighbors and friends.

Zestimates were never intended to be appraisals. After all, Zillow has no idea what is on the inside of any home. But the Zestimate algorithm does use many of the same approaches as appraisers use, including comparisons of recent sale prices of similar houses and historical sales trends. To the average consumer, they sure looked and felt like appraisals, and in a sense, that’s what really matters.

While Zestimates were unquestionably a brilliant way to launch a new website in a crowded vertical (Zillow become one of the highest traffic websites virtually overnight), Zestimates have always been an awkward fit with the Zillow business model. That’s because Zillow is an advertising-based business.

Think about it from the perspective of the real estate agent – the advertising buyer. The agent is attracted by Zillow’s huge traffic numbers and pays for an enhanced listing to get even more prominence. But Zillow automatically (and prominently) displays its Zestimate right near the asking price. Imagine asking $1 million for a home when the seemingly authoritative Zestimate pronounces the value of the home to be $700,000. As an agent, you’re not going to be happy.

Zillow’s stance is basically, “hey, it’s just an objective data point.” But advertisers don’t want to hear it. And that’s the essence of several recent lawsuits. In one lawsuit, the plaintiff argues that Zillow damaged her selling prospects by posting a lower Zestimate near her asking price and doing so without her permission. Another lawsuit goes further, saying that Zillow agreed with certain real estate agents to “de-emphasize” (read: hide) the Zestimate within the listing, meaning that some agents were getting a more attractive listing presentation, and those that didn’t pay an advertising fee were being disadvantaged.

This may sound like a problem peculiar to Zillow but it’s not. Yelp has dealt with a similar issue for years. In short, Yelp is finding it hard to sell advertising to customers whose listings are chock full of negative reviews. Yelp has been repeatedly accused of “de-emphasizing” (read: hiding) these negative reviews to satisfy advertisers.

The simple lesson here is that objective data and advertising don’t always mix, and that creates complexity and legal exposure unless you are aware of the issue and identify a solution that works for everybody. Those solutions can be hard to find.

Reviewing the Reviewers

You are likely familiar with Yelp, the local business ratings and review platform. It's been a phenomenal success, but it also has a large number of detractors, in large part the businesses that are the subject of those ratings and reviews. Yelp is a business that, if examined dispassionately, really should never have succeeded. The primary reason was its business model: let anonymous users pick apart businesses in published reviews, then try to sell advertising to those same businesses. Yelp made this challenged business model even tougher by introducing a secretive filtering algorithm that would decide what reviews got published. The objective was to weed out spam, but all it did instead was to spur conspiracy theories among businesses that felt good reviews were being swallowed while bad reviews always seemed to get published.

Tough business, right? Well it gets tougher, particularly because Yelp showed little interest in mediating disputes (for example, there are documented cases of restaurants getting bad reviews on dishes they have never offered), essentially admitting it was too much work. Mix into this the inexperienced sales force Yelp fielded, giving rise to stories of reps offering to make bad reviews disappear  in exchange for advertising, with a raft of lawsuits claiming extortion quickly following.

Things seem to have calmed down for Yelp in the last year or so, but it's hard to imagine that the rift between the business community and Yelp has fully mended. Yelp is more powerful than ever, and can make or break a business. Yet it maintains as a core principle that it is a consumer empowerment tool, even though Yelp generates no revenue from consumers.

That's why I find it surprising that Yelp just announced the acquisition of Eat24, a service that lets people order food for home delivery. Yes, the company that controls the reputation and success of restaurants now wants to control their order flow as well. I see nothing to suggest that Yelp has become a friendly, trusted brand to the average local restaurateur. Yelp brings scale, but a lot of baggage as well.

What is the correct business model for a ratings and review business? There is no easy answer, especially as the consumers who typically provide the reviews show little appetite to pay to access them. One exception is Angie's List, which sells subscriptions, but even Angie's List now makes more money from advertising than subscriptions. Fortunately, Angie's List found a middle path that allowed this revenue pivot without compromising its credibility and integrity.

TripAdvisor is another reviews site with many of the same issues as Yelp. But TripAdvisor makes most of its money by selling eyeballs, a traditional media model. This means its doesn't have to rely on hotels for revenue, though it recently started to push in this direction.

The real estate website Zillow posts its estimate of a home's value right next to the (almost always higher) asking price. One can presume that's not helpful to making the sale. Awkward? Well, Zillow now asks consumers to rate and review the real estate agents to whom it sells advertising.

In many respects, the jury is still out on what does and doesn't work for review sites. What we've seen to date is that if you can build a big enough audience, the advertising dollars will follow, no matter how upside-down your business model. But just because they pay you doesn't mean they have to like you.  And this may come back to haunt these companies, if not in their core business, then by ultimately limiting their growth and expansion potential.

How Do Your Customers Rate?

It’s recently become known that Uber not only allows its customers to rate its drivers; it also allows its drivers to rate its customers. If you’re loud, abusive, demanding or otherwise unpleasant, you might find yourself silently boycotted by Uber drivers, none of whom are obliged to pick you up. Is this scary, unfair, an exercise in pure democracy or a wake-up call to consumers, who have to date exercised sometime life-and-death power over all types of small businesses with their comments, ratings and reviews, often furnished anonymously? It’s too early to know if this move by Uber foretells a trend, but it’s worth exploring.

Spend just a few minutes online, and you’ll quickly learn of the outrage of small businesses against rating and review websites such as TripAdvisor and Yelp, not to mention the periodic lawsuits. These businesses see an issue of basic fairness: why should unknown strangers (who may even be my competitors pretending to be customers) determine the success of my business in a manner in which I cannot even defend myself?

On the flip side, should businesses be empowered to rate their customers? In some cases, it’s simply not possible: customer transactions are largely anonymous. But for big dollar B2B transactions, rating and review platforms already exist.

An early example of this is a site called TheFunded.com. It had the audacity to let entrepreneurs rate venture capitalists, anonymously. It created a firestorm in the industry, with venture capitalists up in arms about the unfairness of anonymous reviews. In fact, the outrage really stemmed from the upending of the power dynamic in that business. Suddenly, entrepreneurs were no longer supplicants.

Business credit website Cortera has an interesting approach, creating online forums for credit managers in specific market areas to exchange information on companies. It’s a good concept, but one where it’s difficult to get a critical mass of interactions.

The idea of businesses rating customers is not completely new. Indeed, companies like ChexSystems operate “bad customer” databases used by banks to judge whether or not they want to do business with you. And in the apartment rental industry, numerous databases exist to report bad tenants, some with catchy names like badtenantslist.net and donotrento.com. These are not credit rating databases as much as they are places to report poor behavior.

So maybe widespread customer ratings will come along faster than we think. And if they do, a nifty data opportunity will arise: aggregate the ratings of customers that can then be used to weight the ratings these consumers assign to businesses. In algorithms, veritas.