Thumbs up? Sentiment Classification using Machine Learning Techniques

6.1 Experimental Set-up

1. What data and evaluation protocol did Pang et al. use for their classifiers?
   700 positive and 700 negative reviews, three-fold cross-validation with balanced folds (few folds because MaxEnt training was slow). Rating indicators were removed and punctuation kept as tokens; no stemming or stop lists.
   • Pang et al., 2002, Thumbs up? Sentiment Classification using Machine Learning Techniques (Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)), §6.1, pp. 82–83
2. How did Pang et al. model negation in a bag of words?
   Adapting Das and Chen (2001), they prefixed NOT_ to every word between a negation word (“not”, “isn’t”, “didn’t”…) and the next punctuation mark, so “good” and “not very good” give different features: not very good . → not NOT_very NOT_good .
   • Pang et al., 2002, Thumbs up? Sentiment Classification using Machine Learning Techniques (Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)), §6.1, pp. 82–83
3. Which features did Pang et al. use, and why were they limited?
   Unigrams (with negation tags) occurring at least 4 times (16165) and the 16165 most frequent bigrams (each occurring at least 7 times), because MaxEnt training is expensive in the number of features.
   • Pang et al., 2002, Thumbs up? Sentiment Classification using Machine Learning Techniques (Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)), §6.1, pp. 82–83