Thumbs up? Sentiment Classification using Machine Learning Techniques

5.2 Maximum Entropy

1. Give the MaxEnt model used by Pang et al., including its feature functions.
   \(P_{\text{ME}}(c \mid d) = \frac{1}{Z(d)} \exp\big(\sum_i \lambda_{i,c} F_{i,c}(d, c)\big)\), where \(F_{i,c}(d, c') = 1\) iff \(n_i(d) > 0\) and \(c' = c\). A large \(\lambda_{i,c}\) makes \(f_i\) a strong indicator of class \(c\). Weights were trained with 10 iterations of improved iterative scaling and a Gaussian prior.
   • Pang et al., 2002, Thumbs up? Sentiment Classification using Machine Learning Techniques (Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)), §5.2, p. 82
2. What is MaxEnt’s potential advantage over Naive Bayes for text classification?
   It makes no assumptions about relationships between features, so it may do better when conditional independence fails, e.g. with overlapping unigram and bigram features.
   • Pang et al., 2002, Thumbs up? Sentiment Classification using Machine Learning Techniques (Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)), §5.2, p. 82