learn.ifs.name

SQuAD: 100,000+ Questions for Machine Comprehension of Text

Rajpurkar et al., 2016, SQuAD (Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP))

Subscriber deck — free during early access

1 cards in “6.2 Human Performance” · whole deck

6.2 Human Performance

  1. How was human performance on SQuAD measured, and what was it?
    Treat the second answer to each question as the human prediction and the others as gold: 77.0% exact match and 86.8% F1 on test. Mismatches were mostly about including or excluding non-essential phrases, not about the answer itself.
    • Rajpurkar et al., 2016, SQuAD (Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP)), §6.2, p. 2389

Source

  • Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev and Percy Liang. 2016. SQuAD: 100,000+ Questions for Machine Comprehension of Text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2383–2392. License: CC BY 4.0. Flashcards adapted from this work. doi:10.18653/v1/D16-1264 https://aclanthology.org/D16-1264/