Sentiment analysis represents a real opportunity for many data publishers to add new, high-value, proprietary and even real-time insight to their data products. But sentiment analysis has inherent strengths and weaknesses you need to appreciate when considering if there is a sentiment analysis opportunity for your data products.
A wonderful example of sentiment analysis at work is represented in a new service called the Twitter Political Index. Working with two respected political polling organizations, Twitter analyzes over 400 million tweets per day, to determine how people are feeling about the two presidential candidates. The key word here is "feeling," because that's what sentiment analysis is all about -- guessing how people feel about a topic. It's not easy, and it is particularly complex for tweets, which are short and often lack context. Moreover, most sentiment analysis tools go well beyond simply binary like/dislike assessments and try to gauge the degree of like or dislike. It's tricky stuff, but the potential applications are numerous and exciting.
This is often the point where many people will start questioning such issues as whether or not Twitter offers a representative sample, and what level of precision and confidence sentiment analysis can offer. These are valid questions, but be careful not to fall into the trap described by research industry veteran Ray Poynter, who notes that all new research methodologies are invariably measured against the standard of perfection. This implies that all current research metholodogies are perfect, which is far from the case. When using tools such as sentiment analysis, you need to consider the application, then pick the methodology, seeking the best fit possible.
That's why data publishers should be thinking about sentiment analysis. You don't need to analyze every tweet; indeed you don't necessarily need Twitter at all. Sentiment analysis can be applied to research reports, blog posts and press releases. And if you can help your
customers better understand how the world currently views a company or product, for example, you can deliver a useful new layer of insight that differentiates you from the competition, makes your products more valuable, and can be acquired and implemented fairly quickly and economically.
And particularly with new research methodologies, I think it's useful to remember the saying, "don't let the perfect become the enemy of the good." Building powerful new data analysis tools is a long journey, one that both publisher and customer are taking together.