A large annotated corpus for learning natural language inference

3.2 Lexicalized Classifier

1. What features did the SNLI lexicalized classifier use?
   Unlexicalized: BLEU of hypothesis vs premise, length difference, and word overlap counts. Lexicalized: hypothesis unigrams and bigrams, cross-unigrams (pairs of words across premise and hypothesis sharing a POS tag), and cross-bigrams (pairs of bigrams whose second words share a POS tag).
   • Bowman et al., 2015, A large annotated corpus for learning natural language inference (Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP)), §3.2, p. 637
2. What did the lexicalized classifier show on SNLI?
   With enough data, a simple lexicalized model with no alignment or tree transformations is strong: 78.2% test accuracy (3-class), vs 71.6% with unigrams only and 50.4% unlexicalized. It nearly memorized training data (99.7%). Cross-bigrams help learn structures like negation and adjective modification.
   • Bowman et al., 2015, A large annotated corpus for learning natural language inference (Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP)), §3.2, p. 637