Coreference Resolution and Entity Linking

24.7 Entity linking

1. What are the two stages of entity linking?
   Mention detection and mention disambiguation.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.7, p. 558
2. What does the TAGME linker precompute from Wikipedia?
   A catalog of entities (pages) indexed for search, in-link counts in(e), and an anchor dictionary: for each page, its title and the anchor texts of links pointing to it, with each anchor’s frequency, link count and \(linkprob(a) = link(a)/freq(a)\).
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.7, p. 558
3. How does TAGME detect mentions?
   Look up every token sequence up to 6 words in the anchor dictionary, then prune, e.g. substrings with low linkprob (Ada Lovelace kept, Lovelace pruned, born not an anchor at all).
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.7, p. 559
4. What is the prior probability factor in TAGME disambiguation, and when does it fail?
   \(p(e \mid a) = \frac{count(a \to e)}{link(a)}\). In “What Chinese Dynasty came before the Yuan?” it picks the most common sense, the currency, missing Yuan dynasty.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.7, p. 559
5. How does TAGME measure relatedness between two Wikipedia entities?
   By shared in-links: \(rel(A, B) = \frac{\log\max(|in(A)|, |in(B)|) - \log|in(A) \cap in(B)|}{\log|W| - \log\min(|in(A)|, |in(B)|)}\).
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.7, p. 559
6. How does TAGME combine relatedness and prior?
   Each other anchor \(b\) votes for \(a \to X\) with \(\frac{1}{|E(b)|}\sum_{Y \in E(b)} rel(X, Y)p(Y \mid b)\); sum the votes, take the entities within \(\epsilon\) of the best relatedness, and choose the one with the highest prior. Finally prune pairs whose average of coherence and linkprob is below a threshold.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.7, p. 560
7. How do bi-encoder entity linkers work?
   Encode the candidate mention span and the entity separately and score them by dot product, so entity embeddings for the whole knowledge base can be precomputed and cached.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.7, p. 560
8. How does ELQ detect mention spans?
   Encode the question with BERT, then \(p([i, j]) = \sigma\big(s_{start}(i) + s_{end}(j) + \sum_{t=i}^{j} s_{mention}(t)\big)\), each score a learned vector dotted with token encodings.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.7, pp. 560–561
9. How does ELQ link spans to entities?
   Entity embedding: BERT[CLS] over the page title and first 128 tokens, [CLS] t(e) [ENT] d(e) [SEP]. Span embedding: the average of its token encodings. Score by dot product, softmax over entities.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.7, p. 561
10. How is ELQ trained?
    Fully supervised on questions with linked entity spans (e.g. WebQSP-EL, GraphQ-EL), jointly optimizing a binary cross-entropy mention detection loss and \(-\log p(e_g \mid [i, j])\) for the gold entity.
    • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.7, pp. 561–562