Three discussions in one week on the topic of user-generated reviews is a sure sign that publishers are on the bandwagon. But what these discussions helped underscore is that user-generated reviews are largely unexplored territory, meaning unanticipated outcomes in some cases.
One publisher, after pushing a user review feature on his site for well over a year, is finally getting a decent volume of reviews, but they are distributed very unevenly. Some companies have garnered dozens of reviews, while the majority still have none. The publisher was considering a promotion to the companies with no reviews, urging them to encourage their customers to post comments. At the same time, the publisher was aware this might inspire some of these companies to game the system by reviewing themselves or only encouraging their most satisfied customers to post reviews.
Another publisher was having the exact opposite issue: too many reviews for anyone to possibly read, and often wildly contradictory to boot. Was he simply aggregating "gigabytes of garbage" he wondered? Could anyone find value in information that was routinely all over the map?
A third publisher, trying to avoid some of the issues of free-form commentary, has instituted a user rating system. His concern? Virtually every organization in his database was scoring between 7 and 8 on a 1 to 10 scale. Beyond the curiosity of it all was the problem that ratings aren't very useful if everyone has essentially the same rating!
There are no easy answers to these situations, but I would offer this basic guidance: if we institute user ratings, we need to work with and trust what we get. If we view ratings as a value-add to our core dataset, then it's clearly not our problem if not every company in our database has a rating. Similarly, numerous and contradictory comments about a company may not offer a clear answer, but if we help the user to conclude "where there's smoke, there's fire" or "proceed with caution," we've still delivered real value. As to the problem of similar ratings, remember that we are still creating useful differentiation from those companies with no ratings. TripAdvisor is trying some innovative things to summarize user comments to make them more actionable, so it's possible that technology may ultimately allow us to smooth out some of the issues relating to user comments and ratings. In the short-run, however, other than working to try to keep user posts useful and honest, don't get hung up trying to perfect this content, which is inherently messy and imprecise. User-generated content is simply color commentary that enhances what really matters: your underlying database.