learn.ifs.name

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin et al., 2019, BERT (Proceedings of the 2019 Conference of the North American Chapter of ACL: Human Language Technologies (NAACL-HLT 2019))

Subscriber deck — free during early access

1 cards in “4.3 SQuAD v2.0” · whole deck

4.3 SQuAD v2.0

  1. How does BERT handle unanswerable questions in SQuAD 2.0?
    No-answer is treated as a span starting and ending at [CLS], with score \(s_{\text{null}} = S \cdot C + E \cdot C\). A non-null answer is predicted only if the best span score exceeds \(s_{\text{null}} + \tau\), with \(\tau\) tuned on dev for F1. This gave 83.1 test F1, +5.1 over the previous best.
    • Devlin et al., 2019, BERT (Proceedings of the 2019 Conference of the North American Chapter of ACL: Human Language Technologies (NAACL-HLT 2019)), §4.3, p. 4177

Source

  • Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019), pp. 4171–4186. License: CC BY 4.0. Flashcards adapted from this work. doi:10.18653/v1/N19-1423 https://aclanthology.org/N19-1423/