A Structural Probe for Finding Syntax in Word Representations
Hewitt & Manning, 2019, A Structural Probe for Finding Syntax in Word Representations (Proceedings of the 2019 Conference of the North American Chapter of ACL: Human Language Technologies (NAACL-HLT 2019))
Subscriber deck — free during early access
1 cards in “2.2 Properties of the structural probe” · whole deck
2.2 Properties of the structural probe
- Why is the structural probe a strong claim about a representation space?It asserts that one inner product on the space makes squared distance, a global property, encode the syntactic distance between every pair of words, not just which word heads which. Probing distance rather than headedness also avoids arbitrary headedness conventions (PPs, determiners, auxiliaries).
- Hewitt & Manning, 2019, A Structural Probe for Finding Syntax in Word Representations (Proceedings of the 2019 Conference of the North American Chapter of ACL: Human Language Technologies (NAACL-HLT 2019)), §2.2, p. 4130
Source
- John Hewitt and Christopher D. Manning. 2019. A Structural Probe for Finding Syntax in Word Representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019), pp. 4129–4138. License: CC BY 4.0. Flashcards adapted from this work. doi:10.18653/v1/N19-1419 https://aclanthology.org/N19-1419/