learn.ifs.name

Introduction

Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 1

14 cards in “1.9 Evaluating large language models” · whole deck

1.9 Evaluating large language models

  1. What is a test set, and why must it be unseen?
    A set of data used to evaluate a system (e.g. questions with human-labeled answers; accuracy = % correct). It must be unseen, i.e. not used in training, or the score would overstate performance.
  2. What are LLM benchmarks?
    Large evaluation test harnesses, often factual or reasoning questions (multiple-choice or free-answer) drawn from human tests in educational or professional contexts.
  3. Name some commonly used LLM evaluation datasets and what they test.
    MMLU (knowledge and reasoning questions across 57 areas such as medicine, law, CS), BigBench Hard (hard reasoning tasks), GSM8k (grade-school math word problems), AIME (competition math).
  4. What is data contamination, and how can it be mitigated?
    When test data makes its way into the training set (likely, since LLMs train on the web where benchmarks live), inflating scores. Mitigation: release the exact training data, or at least report overlap with test sets.
  5. How can we evaluate a language model by its word prediction?
    By how much probability it assigns to the actual next words of unseen text: a better LM gives the correct words higher probability. In practice we use perplexity, which normalizes for text length.
  6. What makes a task “subjective” for evaluation purposes? Give examples.
    It has many possible correct answers, so there is no single right output: judging whether a chatbot is better, how good a translation is, or how sycophantic a response is.
  7. On what two criteria might human raters judge a translation?
    Faithfulness (does it convey the same meaning as the source?) and fluency (does it read as well-formed in the target language?).
  8. What is LLM-as-a-judge, and how do we make sure it is trustworthy?
    Using LLMs instead of humans to evaluate LLM outputs (cheaper, faster, and spares people from emotionally damaging content like toxicity). The judge prompt must be written carefully and its judgments compared to expert humans on a sample.
  9. What is the difference between single and pairwise evaluation?
    Single evaluation assigns a score to one output; pairwise evaluation takes two outputs and decides which is better.
  10. What is a proxy metric? Give examples.
    A formal metric that is easy to compute automatically in place of human judgment, e.g. token overlap with a human reference translation. Examples: chrF, BLEU and ROUGE for MT and summarization; word error rate for speech recognition.
  11. State Goodhart’s Law and its consequence for proxy metrics.
    “When a measure becomes a target, it ceases to be a good measure.” Tuning a system to a metric like chrF may optimize incidental properties (word overlap) rather than the real goal (good translations).
  12. Besides accuracy, what other factors matter when evaluating language models?
    Model size, training and inference time, memory, energy use (kWh or kg CO₂ emitted), safety and bias, and fairness across groups.
  13. What do benchmarks like StereoSet, RealToxicityPrompts and BBQ measure?
    Bias and toxicity: how strongly a model generates stereotypes or toxic language.
  14. What does it mean for an evaluation to be fair in a Rawlsian sense?
    It maximizes the welfare of the worst-off group, rather than average performance.

Source