Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms

2.2 Local and Global Feature Vectors

1. What is a history in a MaxEnt-style tagger, and what is a local feature vector?
   A history is \(\langle t_{-1}, t_{-2}, w_{[1:n]}, i \rangle\): the previous two tags, the whole sentence and the current position. A local representation \(\phi: \mathcal{H} \times \mathcal{T} \to \mathbb{R}^d\) maps a history/tag pair to features, typically indicators, e.g. \(\phi_{1000}(h, t) = 1\) iff the current word is the and \(t = \text{DT}\).
   • Collins, 2002, Discriminative Training Methods for Hidden Markov Models (Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)), §2.2, pp. 2–3
2. How does Collins define global features, and what are they when local features are indicators?
   \(\Phi_s(w_{[1:n]}, t_{[1:n]}) = \sum_{i=1}^{n} \phi_s(h_i, t_i)\): local features summed over all positions. For indicator features, global features are counts, e.g. how often the is tagged DT in the sentence.
   • Collins, 2002, Discriminative Training Methods for Hidden Markov Models (Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)), §2.2, pp. 2–3