learn.ifs.name

GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

Wang et al., 2019, GLUE (7th International Conference on Learning Representations (ICLR 2019))

Subscriber deck — free during early access

1 cards in “3.2 Similarity and Paraphrase Tasks” · whole deck

3.2 Similarity and Paraphrase Tasks

  1. Why are MRPC and QQP scored with both accuracy and F1?
    Their classes are unbalanced (MRPC 68% positive, QQP 63% negative), so accuracy alone is misleading; STS-B, a regression task predicting 1–5 similarity scores, uses Pearson and Spearman correlation.
    • Wang et al., 2019, GLUE (7th International Conference on Learning Representations (ICLR 2019)), §3.2

Source

  • Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy and Samuel R. Bowman. 2019. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In 7th International Conference on Learning Representations (ICLR 2019). https://openreview.net/forum?id=rJ4km2R5t7