Coreference Resolution and Entity Linking

24.6 A neural mention-ranking algorithm

1. Outline the e2e-coref algorithm (Lee et al. 2017).
   Consider all spans up to about 10 words; score each as a mention; prune; for each remaining span \(i\), choose an antecedent \(y_i \in \{1, \ldots, i-1, \epsilon\}\) with \(P(y_i) = \frac{\exp s(i, y_i)}{\sum_{y'} \exp s(i, y')}\), where \(s(i, j) = m(i) + m(j) + c(i, j)\).
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.6, p. 555
2. Why is \(s(i, \epsilon)\) fixed to 0 in e2e-coref?
   So if any real antecedent scores positive the model picks the highest-scoring one, and if all are negative it abstains (no antecedent).
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.6, p. 555
3. How is a span represented in e2e-coref?
   \(g_i = [h_{START(i)}, h_{END(i)}, h_{ATT(i)}]\): the encoder outputs (e.g. BERT) of the first and last tokens plus an attention-weighted sum over the span’s tokens that learns to find the head word.

   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.6, pp. 555–556
4. How are the mention score \(m(i)\) and antecedent score \(c(i, j)\) computed?
   By feedforward networks: \(m(i) = w_m \cdot FFN_m(g_i)\), \(c(i, j) = w_c \cdot FFN_c([g_i, g_j, g_i \circ g_j])\), where \(\circ\) is elementwise multiplication (a similarity feature). \(m\) also prunes spans at inference.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.6, p. 556
5. What loss does e2e-coref train with?
   Since the antecedent is latent, maximize the total probability of all gold antecedents before \(i\): \(L = \sum_i -\log \sum_{\hat{y} \in Y(i) \cap GOLD(i)} P(\hat{y})\) (with \(GOLD(i) = \epsilon\) if \(i\) is in no gold cluster).
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.6, pp. 557–558