Sequence Labeling for Parts of Speech and Named Entities

18.2 Part-of-speech tagging

1. Why is POS tagging a disambiguation task?
   Words are ambiguous between parts of speech (book as verb or noun, that as determiner or complementizer), and the tagger must choose the right one in context.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 18, §18.2, p. 403
2. How accurate is POS tagging?
   Very accurate: for English (and many UD languages) taggers of all kinds (HMM, CRF, BERT) reach about human-level accuracy, in the high 90s percent.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 18, §18.2, p. 404
3. How ambiguous is English POS tagging by types versus tokens?
   Most word types are unambiguous, but the ambiguous ones are very frequent, so a majority of word tokens in running text are ambiguous (that, back, down, put, set).
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 18, §18.2, p. 404
4. Give several different parts of speech of back.
   Adjective (a back seat), noun (in the back), verb (senators back the bill), particle (buy back debt), adverb (back then).
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 18, §18.2, p. 404
5. What is the most frequent class baseline, and why use it?
   Assign each token the tag it occurred with most often in training. It is surprisingly strong (only a few points below state of the art for English POS tagging), so classifiers should always be compared against at least this baseline.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 18, §18.2, p. 404