Sequence Labeling for Parts of Speech and Named Entities

Historical notes

1. What architecture did the earliest POS taggers (TDAP, CGC, TAGGIT) share?
   Two stages: a dictionary (plus morphological analysis) assigned each word its set of possible tags, then hand-written disambiguation rules winnowed them to one per word.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 18, Historical notes, p. 422
2. Which was probably the first implemented HMM tagger?
   Church’s PARTS tagger (1988), described correctly in Church (1989); CLAWS and DeRose’s tagger were earlier approximations.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 18, Historical notes, p. 422
3. How did log-linear and neural sequence labelers develop for POS and NER?
   Ratnaparkhi’s MXPOST (a maximum entropy Markov model, MEMM, 1996), then CRFs (Lafferty et al., 2001) applied to NER; neural NER from Collobert et al. (CRF over a CNN), then biLSTMs with word and character embeddings, then transformers like BERT.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 18, Historical notes, p. 423
4. What did Manning (2011) find about the remaining errors of a top POS tagger?
   A third to half came from errors or inconsistencies in the training data, about a third might be solvable with richer linguistic models, and the rest reflect an underspecified or unclear task.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 18, Historical notes, p. 423