Coreference Resolution and Entity Linking

24.10 Gender bias in coreference

1. How does WinoBias test gender bias in coreference?
   Sentences with a stereotypically male and a stereotypically female occupation and a pronoun that grammar and gender can’t disambiguate: “The secretary called the physician and told him/her about a new patient.” Biased systems are more accurate on pro-stereotypical cases.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.10, pp. 564–565
2. How did Zhao et al. reduce coreference gender bias?
   Gender-swap all entities in OntoNotes to create a second dataset, train on original plus swapped, and use debiased embeddings (later, retrain ELMo with augmentation); WinoBias bias disappeared without hurting OntoNotes accuracy.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.10, p. 565
3. What is GAP?
   A gender-balanced corpus of Wikipedia sentences with an ambiguous gendered pronoun and two same-gender names, for Gendered Pronoun Resolution; systems do worse on feminine pronouns.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 24, §24.10, p. 565