learn.ifs.name

GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

Wang et al., 2019, GLUE (7th International Conference on Learning Representations (ICLR 2019))

Subscriber deck — free during early access

2 cards in “6 Benchmark Results” · whole deck

6 Benchmark Results

  1. What were the main GLUE baseline results?
    Multi-task training beat single-task training for models with attention or ELMo; the best was multi-task BiLSTM + attention + ELMo with a 70.0 average. ELMo helped consistently, especially on single-sentence tasks; no model beat the majority class on WNLI (65.1%).
    • Wang et al., 2019, GLUE (7th International Conference on Learning Representations (ICLR 2019)), §6
  2. How did pretrained sentence encoders compare with models trained on GLUE tasks?
    Gains rose from CBoW to Skip-Thought, InferSent and GenSen, and GenSen beat all but the two best GLUE-trained models. Sentence encoders did badly on CoLA but best on STS-B.
    • Wang et al., 2019, GLUE (7th International Conference on Learning Representations (ICLR 2019)), §6

Source

  • Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy and Samuel R. Bowman. 2019. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In 7th International Conference on Learning Representations (ICLR 2019). https://openreview.net/forum?id=rJ4km2R5t7