Logistic Regression and Text Classification

4.12 Avoiding harms in classification

1. What are representational harms? Give an example from sentiment analysis.
   Harms caused by systems that demean a social group, e.g. by perpetuating stereotypes. Most of 200 sentiment systems assigned more negative sentiment to sentences that differed only by having a common African American first name instead of a European American one.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 4, §4.12, pp. 119–120
2. How can toxicity classifiers themselves cause harm?
   They falsely flag non-toxic sentences that mention identities (women, blind people, gay people) or use features of varieties like AAVE, which can silence discourse by or about these groups.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 4, §4.12, p. 120
3. What are possible sources of bias in classifiers?
   Training data (systems replicate and amplify its biases), labels (biased annotators), resources (lexicons, pretrained embeddings), and even model architecture or training objective.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 4, §4.12, p. 120
4. What is a model card?
   Documentation released with each model version: training algorithms and parameters, training and evaluation data (sources, motivation, preprocessing), intended use and users, and performance across demographic or other groups and situations.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 4, §4.12, p. 120