Sequence Labeling for Parts of Speech and Named Entities

18.6 Evaluation of NER

1. How are POS taggers and NER systems evaluated?
   POS taggers by accuracy. NER by precision, recall and F1 over entities (the entity, not the word, is the unit of response), with paired bootstrap or randomization tests for significance.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 18, §18.6, p. 419