Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms
Collins, 2002, Discriminative Training Methods for Hidden Markov Models (Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002))
Subscriber deck — free during early access
1 cards in “2.5 Averaging Parameters” · whole deck
2.5 Averaging Parameters
- What is the averaged perceptron?Instead of the final parameters \(\alpha_s^{T,n}\), output the average of the parameter vectors after every training example in every pass: \(\gamma_s = \sum_{t=1}^{T} \sum_{i=1}^{n} \alpha_s^{t,i} / nT\). It is simple to maintain during training and performed significantly better than the final parameters.
- Collins, 2002, Discriminative Training Methods for Hidden Markov Models (Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)), §2.5, p. 4
Source
- Michael Collins. 2002. Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002), pp. 1–8. https://aclanthology.org/W02-1001/