learn.ifs.name

Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms

Collins, 2002, Discriminative Training Methods for Hidden Markov Models (Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002))

Subscriber deck — free during early access

4 cards in “4 Experiments” · whole deck

4 Experiments

  1. On which tasks and data did Collins test the structured perceptron?
    POS tagging on the Penn WSJ treebank (train sections 0–18, dev 19–21, test 22–24), with Ratnaparkhi’s (1996) MaxEnt tagger features, and base NP chunking on Ramshaw and Marcus’s data (sections 15–18 train, 20 test), measured by F-measure.
  2. How was NP chunking cast as tagging in Collins (2002)?
    As a three-tag task: B begins a chunk, I continues it, O is outside. Every chunk starts with B whether the previous word is O or I. Features combine the current chunk tag \(t_i\) with nearby words and POS tags (from Brill’s tagger), up to two positions away, plus bigram and trigram combinations.
  3. What did development experiments show about averaging and feature cut-offs?
    Averaging improved the perceptron significantly and made it much more stable across iterations. Including all features (count cut-off 0) helped the perceptron but hurt the MaxEnt model, which did better with a cut-off of 5. The perceptron also needed far fewer training iterations (about 6–39 vs 100–900).
  4. How did the averaged perceptron compare with the MaxEnt tagger on test data?
    POS tagging: 2.89% error vs 3.28% (an 11.9% relative error reduction). NP chunking: F-measure 93.63% vs 93.29% (a 5.1% relative error reduction).

Source

  • Michael Collins. 2002. Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002), pp. 1–8. https://aclanthology.org/W02-1001/