Deep contextualized word representations
Peters et al., 2018, Deep contextualized word representations (Proceedings of the 2018 Conference of the North American Chapter of ACL: Human Language Technologies (NAACL-HLT 2018))
Subscriber deck — free during early access
1 cards in “5.4 Sample efficiency” · whole deck
5.4 Sample efficiency
- How does ELMo affect sample efficiency?Strongly: the SRL model without ELMo peaked after 486 epochs, while with ELMo it passed that score at epoch 10 (98% fewer updates). Gains were largest for small training sets; with 1% of the SRL data, ELMo matched the baseline trained on 10%.
- Peters et al., 2018, Deep contextualized word representations (Proceedings of the 2018 Conference of the North American Chapter of ACL: Human Language Technologies (NAACL-HLT 2018)), §5.4, p. 2234
Source
- Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2018), pp. 2227–2237. License: CC BY 4.0. Flashcards adapted from this work. doi:10.18653/v1/N18-1202 https://aclanthology.org/N18-1202/