Thumbs up? Sentiment Classification using Machine Learning Techniques

4 A Closer Look At the Problem

1. How did Pang et al. test whether hand-picked sentiment words suffice?
   Two computer science graduate students independently chose positive and negative indicator words; a document was classified by counting proposed positive vs negative words, on balanced data (chance = 50%). The lists reached 58% and 64% accuracy, with high tie rates (75% and 39%).
   • Pang et al., 2002, Thumbs up? Sentiment Classification using Machine Learning Techniques (Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)), §4, pp. 80–81
2. What did the third, “introspection plus corpus statistics” baseline show?
   Seven positive and seven negative items picked by glancing at corpus counts (including test data) reached 69% with a 16% tie rate. It included items people wouldn’t propose by introspection, like “?” (“What was the director thinking?”) and “still” (“Still, though, it was worth seeing”). So corpus-based feature selection beats intuition.
   • Pang et al., 2002, Thumbs up? Sentiment Classification using Machine Learning Techniques (Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)), §4, pp. 80–81