A Structural Probe for Finding Syntax in Word Representations
Hewitt & Manning, 2019, A Structural Probe for Finding Syntax in Word Representations (Proceedings of the 2019 Conference of the North American Chapter of ACL: Human Language Technologies (NAACL-HLT 2019))
Subscriber deck — free during early access
1 cards in “3 Experiments” · whole deck
3 Experiments
- Which baselines did Hewitt and Manning compare ELMo and BERT against?LINEAR (a left-to-right chain), ELMo0 (non-contextual character embeddings, no position information), DECAY0 (exponentially distance-weighted averages of ELMo0 vectors), and PROJ0 (ELMo0 contextualized by an untrained random BiLSTM of ELMo’s size): baselines with useful features that shouldn’t be able to parse.
- Hewitt & Manning, 2019, A Structural Probe for Finding Syntax in Word Representations (Proceedings of the 2019 Conference of the North American Chapter of ACL: Human Language Technologies (NAACL-HLT 2019)), §3, p. 4131
Source
- John Hewitt and Christopher D. Manning. 2019. A Structural Probe for Finding Syntax in Word Representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019), pp. 4129–4138. License: CC BY 4.0. Flashcards adapted from this work. doi:10.18653/v1/N19-1419 https://aclanthology.org/N19-1419/