Accurate Unlexicalized Parsing
Klein & Manning, 2003, Accurate Unlexicalized Parsing (Proceedings of the 41st Annual Meeting of ACL)
Subscriber deck — free during early access
2 cards in “5 What is an Unlexicalized Grammar?” · whole deck
5 What is an Unlexicalized Grammar?
- Where do Klein and Manning draw the line between unlexicalized and lexicalized grammar?At function words vs content words: annotating phrasal nodes with closed-class distinctions (like CP[for], PP[to], or VP finiteness) is standard linguistic practice and allowed; using open-class words (NP[stocks]) for monolexical or bilexical probabilities is not.
- Klein & Manning, 2003, Accurate Unlexicalized Parsing (Proceedings of the 41st Annual Meeting of ACL), §5, p. 428
- Why must state splits in an unsmoothed PCFG be frequent and coarse?With pure maximum-likelihood estimation and no interpolation to unsplit categories, detailed lexical splits make many sentences fail to parse or get weird parses; only a few distinctions with major distributional impact pay off. Even after lexeme splits the grammar had only 9255 states vs 7619 in the baseline.
- Klein & Manning, 2003, Accurate Unlexicalized Parsing (Proceedings of the 41st Annual Meeting of ACL), §5, p. 428
Source
- Dan Klein and Christopher D. Manning. 2003. Accurate Unlexicalized Parsing. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pp. 423–430. https://aclanthology.org/P03-1054/