Annotation Artifacts in Natural Language Inference Data

3.2 Sentence Length

1. How does hypothesis length relate to the NLI label in SNLI?
   Neutral hypotheses tend to be long (median 9 tokens), entailments short (60% have 7 tokens or fewer). Half of hypotheses with at least 12 tokens are neutral, and about half of those with 5 or fewer are entailments, so length alone is a useful feature.
   • Gururangan et al., 2018, Annotation Artifacts in Natural Language Inference Data (Proceedings of the 2018 Conference of the North American Chapter of ACL: Human Language Technologies, Volume 2 (Short Papers)), §3.2, p. 109
2. What does word containment reveal about how entailed hypotheses were written?
   As bags of words, 8.8% of entailed SNLI hypotheses are fully contained in their premise, versus 0.2% of neutrals and contradictions, suggesting many entailments were written by deleting words from the premise.
   • Gururangan et al., 2018, Annotation Artifacts in Natural Language Inference Data (Proceedings of the 2018 Conference of the North American Chapter of ACL: Human Language Technologies, Volume 2 (Short Papers)), §3.2, p. 109