Masked Language Models

9.5 Sequence labeling: named entity recognition

1. What is sequence labeling?
   Assigning each token a label from a small fixed set, e.g. named entity recognition or part-of-speech tagging.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 9, §9.5, p. 238
2. What is a named entity, and what is NER?
   Roughly anything referred to by a proper name: a person, location, organization. Named entity recognition finds the spans of text that are proper names and tags their type. The term is often extended to dates, times and prices.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 9, §9.5, p. 238
3. What are the four most common named entity types?
   PER (person), LOC (location), ORG (organization), GPE (geo-political entity).
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 9, §9.5, p. 238
4. What is NER useful for?
   Linking text to knowledge sources like Wikipedia, measuring sentiment toward particular entities, and anonymizing text for privacy.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 9, §9.5, p. 238
5. Why is NER difficult?
   Segmentation ambiguity (which tokens form an entity span, when most words aren’t entities) and type ambiguity: Washington can be a person, a sports team (ORG), a city (LOC) or a government (GPE).
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 9, §9.5, pp. 238–239
6. What is BIO tagging?
   A way to turn span recognition into per-token labeling: B marks the first token of a span, I a token inside a span, O a token outside any span, with B and I tags per entity type (\(2n + 1\) tags for \(n\) types).

   word	BIO	BIOES
   Jane	B-PER	B-PER
   Villanueva	I-PER	E-PER
   of	O	O
   United	B-ORG	B-ORG
   Airlines	I-ORG	E-ORG
   Chicago	B-LOC	S-LOC

   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 9, §9.5, p. 239
7. What are IO and BIOES tagging?
   IO drops the B tag, losing information about where adjacent spans begin. BIOES adds E for the end of a span and S for a single-token span.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 9, §9.5, p. 239
8. How does an encoder do NER as sequence labeling?
   Pass each token’s final output vector through a classifier \(y_i = \text{softmax}(h^L_iW_K)\) over the \(k\) tags and take the argmax per token, or pass the distributions to a CRF layer that models tag transitions.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 9, §9.5, pp. 239–240
9. How do you handle the mismatch between word-level BIO annotations and subword tokens?
   Training: give every subword token the tag of the word it came from. Decoding: use the tag predicted for the first subword of each word (or combine the subwords’ tag distributions).
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 9, §9.5, pp. 240–241
10. How is NER evaluated, and what is the unit of response?
    Precision, recall and F1 (with paired bootstrap tests for significance), counting entities, not words, as units: Jane Villanueva is one response.
    • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 9, §9.5, p. 241
11. Why does segmentation complicate NER evaluation?
    Labeling only Jane instead of Jane Villanueva as PER causes two errors (a false positive and a false negative), and training is per word while evaluation is per entity.
    • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 9, §9.5, p. 241