A Structural Probe for Finding Syntax in Word Representations

2 Methods

1. Why do tree distances and depths capture a parse tree?
   Words at tree distance 1 are neighbours, which recovers the tree’s edges, and of two neighbours the one with greater depth is the child, which recovers edge direction. Distances also explain hierarchical behaviour, e.g. subject–verb agreement over attractor nouns: the verb is closer in the tree to its subject than to the attractors.
   • Hewitt & Manning, 2019, A Structural Probe for Finding Syntax in Word Representations (Proceedings of the 2019 Conference of the North American Chapter of ACL: Human Language Technologies (NAACL-HLT 2019)), §2, pp. 4129–4130
2. In the tree “the dog chased a cat” (chased is the root; dog and cat depend on chased; the on dog; a on cat), what are \(d(\textit{the}, \textit{cat})\) and the depths of the and chased?
   \(d(\textit{the}, \textit{cat}) = 3\) (the → dog → chased → cat). Depth counts edges to the root: \(\|\textit{chased}\| = 0\), \(\|\textit{the}\| = 2\).
   • Hewitt & Manning, 2019, A Structural Probe for Finding Syntax in Word Representations (Proceedings of the 2019 Conference of the North American Chapter of ACL: Human Language Technologies (NAACL-HLT 2019)), §2, pp. 4129–4130