Logistic Regression and Text Classification

Introduction

1. What is text categorization? Give examples of text classification tasks.
   Assigning a label or category to a text or document. Examples: sentiment analysis (positive/negative orientation toward something), spam detection, language identification, and authorship attribution.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 4, Introduction, p. 92
2. In what sense is language modeling a classification task?
   Each word of the vocabulary is a class: predicting the next word means classifying the context-so-far into one of \(|V|\) classes.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 4, Introduction, p. 92
3. Why is logistic regression important beyond being a classifier?
   A neural network can be seen as stacked logistic regression classifiers; it introduces the sigmoid, softmax, logit and gradient descent that underlie LLMs; and it is a key analytic tool in the social and natural sciences.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 4, Introduction, pp. 92–93