Thumbs up? Sentiment Classification using Machine Learning Techniques

7 Discussion

1. What were Pang et al.’s overall conclusions about classifiers and features?
   Machine learning beats human-chosen word lists; Naive Bayes tended to be worst and SVMs best, but differences were small. Unigram presence was the most effective feature, and none of the alternatives consistently improved on it. Accuracy stayed well below topic categorization.
   • Pang et al., 2002, Thumbs up? Sentiment Classification using Machine Learning Techniques (Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)), §7, pp. 84–85
2. What is the “thwarted expectations” problem in sentiment classification?
   Authors set up a contrast to earlier text: many sentences praise (or attack) something before the verdict flips, e.g. “This film should be brilliant… However, it can’t hold up.” A human sees the true sentiment, but a bag of features sees many words of the opposite polarity.
   • Pang et al., 2002, Thumbs up? Sentiment Classification using Machine Learning Techniques (Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)), §7, pp. 84–85
3. What future direction did Pang et al. propose to handle thwarted expectations?
   Some discourse analysis, or at least determining each sentence’s focus (whether it is about the film itself, i.e. on-topic, a kind of co-reference problem), e.g. for “a good actor trapped in a bad movie”. As Turney put it, “the whole is not necessarily the sum of the parts”.
   • Pang et al., 2002, Thumbs up? Sentiment Classification using Machine Learning Techniques (Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)), §7, pp. 84–85