Accurate Unlexicalized Parsing

Introduction

1. What is the main claim of Klein and Manning’s “Accurate Unlexicalized Parsing”?
   An unlexicalized PCFG can parse far more accurately than previously shown, using simple, linguistically motivated state splits that break down false independence assumptions of the vanilla treebank grammar: 86.36% F1, better than early lexicalized PCFGs (Magerman 1995, Collins 1996) and close to the state of the art.
   • Klein & Manning, 2003, Accurate Unlexicalized Parsing (Proceedings of the 41st Annual Meeting of ACL), Introduction, pp. 423–424
2. What evidence questioned how much lexicalization helps PCFG parsers?
   Johnson (1998) improved an unlexicalized treebank PCFG enormously just by parent annotation, and Gildea (2001) found that removing bilexical probabilities from a good lexicalized parser cost at most 0.5% in-domain and nothing out of domain.
   • Klein & Manning, 2003, Accurate Unlexicalized Parsing (Proceedings of the 41st Annual Meeting of ACL), Introduction, pp. 423–424
3. Why are bilexical dependencies of limited use when trained on the Penn Treebank, according to Klein and Manning?
   Sparseness: a million words isn’t enough. Even for central WSJ topics, plausible dependencies occur once (stocks stabilized) or never (stocks skyrocketed).
   • Klein & Manning, 2003, Accurate Unlexicalized Parsing (Proceedings of the 41st Annual Meeting of ACL), Introduction, pp. 423–424
4. What practical advantages does an unlexicalized PCFG have over lexicalized ones?
   It is more compact, easier to replicate, interpret and improve; parsing is simpler, better understood and asymptotically faster (\(O(n^3)\) vs \(O(n^5)\) naively, or \(O(n^4)\) with Eisner and Satta 1999), with smaller grammar constants and easier optimization and pruning.
   • Klein & Manning, 2003, Accurate Unlexicalized Parsing (Proceedings of the 41st Annual Meeting of ACL), Introduction, pp. 423–424
5. Did Klein and Manning argue against lexicalized parsing?
   No: lexical dependencies are useful for major ambiguity classes and parsers should use them. They argued that unlexicalized structural context was underexploited, giving a much stronger lower bound and showing that lexicalization’s benefit had been overestimated.
   • Klein & Manning, 2003, Accurate Unlexicalized Parsing (Proceedings of the 41st Annual Meeting of ACL), Introduction, pp. 423–424