learn.ifs.name

Context-Free Grammars and Constituency Parsing

Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 19

7 cards in “19.7 Span-based neural parsing” · whole deck

19.7 Span-based neural parsing

  1. What problem of CKY does span-based neural constituency parsing solve?
    CKY enumerates all parses but doesn’t say which is correct. A neural classifier scores each labeled span, and a modified CKY combines span scores to find the best-scoring tree (“neural CKY”).
  2. How does the span-based parser of Kitaev et al. (2019) differ from classic CKY?
    It doesn’t use a hand-written grammar to constrain which constituents combine; it relies on learned span representations to encode likely combinations.
  3. How are words represented in a span-based parser built on BERT?
    Run the tokens through a pretrained model like BERT, map subwords to words (e.g. use the first subword), optionally add transformer layers, giving one output per word.
  4. How is a span \((i, j)\) represented and scored?
    Split each word output into a leftward and a rightward half, representing fencepost endpoints; the span vector concatenates the differences of the endpoint halves. An MLP gives a score per label: \(s(i, j, \cdot) = W_2\,\text{ReLU}(\text{LayerNorm}(W_1v(i, j)))\).
  5. How is a whole tree scored in span-based parsing?
    A tree is a set of labeled spans \(\{(i_t, j_t, l_t)\}\); its score is the sum of their span scores, and the parser picks the tree with the maximum score.
  6. Why not simply pick the best label for each span independently?
    The greedy choice isn’t guaranteed to form a valid tree (though it usually does), so a CKY variant is used.
  7. Give the CKY recursion used in span-based parsing.
    \(s_{best}(i, i+1) = \max_l s(i, i+1, l)\); for longer spans, \(s_{best}(i, j) = \max_l s(i, j, l) + \max_k[s_{best}(i, k) + s_{best}(k, j)]\), with no grammar constraints: the network learns them.

Source