A large annotated corpus for learning natural language inference

3.4 Analysis and discussion

1. What did learning curves on SNLI show?
   The corpus size is crucial for both the LSTM and the lexicalized model, and more data would likely help both; the LSTM’s steeper slope hints it may overtake the constrained lexicalized model on still larger data. The lexicalized model beat the unlexicalized one with only 100 examples.
   • Bowman et al., 2015, A large annotated corpus for learning natural language inference (Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP)), §3.4, pp. 638–639
2. What kinds of errors did all SNLI models share?
   Misanalysed lexical relations even for frequent pairs (beach/surf, sprinter/runner), phrasal semantics, world knowledge (“driver leaps from a burning car” → “escaping danger”), and shortcuts through lexical cues and expected parallel structure: all models predicted entailment for “A man bit a dog” from a premise where a dog bites a man.
   • Bowman et al., 2015, A large annotated corpus for learning natural language inference (Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP)), §3.4, pp. 638–639