A Watermark for Large Language Models

A hidden pattern in generated text that humans can’t perceive but an algorithm can detect, marking the text as machine-generated; Kirchenbauer et al. aim to detect it from short spans (as few as about 25 tokens) with statistically improbable false positives.

Detectable without the model’s parameters or API (so the detector can be public and cheap); applicable to a standard model without retraining; detectable from any contiguous portion of the text; not removable without changing a large fraction of tokens; and with a rigorous statistical confidence measure.

When the first tokens nearly determine the rest (e.g. a famous pangram or a standard for-loop), humans and models produce the same completion, so the source can’t be told apart; and forcing different tokens would make the text unnatural.

At each step, hash the previous token to seed a random generator, use it to split the vocabulary into a green and a red list, and sample the next token only from the green list. A detector that knows the hash can rebuild each red list and count violations.

Under the null hypothesis that the text was written without knowing the red list rule, the green-token count \(|s|_G\) of \(T\) tokens has mean \(\gamma T\) and variance \(T\gamma(1-\gamma)\); compute \(z = (|s|_G - \gamma T)/\sqrt{T\gamma(1-\gamma)}\) and flag the text if \(z\) exceeds a threshold, e.g. \(z > 4\) (false-positive rate \(3 \times 10^{-5}\)).

16: if every token is green, \(z = 2(T - T/2)/\sqrt{T} = \sqrt{T}\), which reaches 4 at \(T = 16\).

Each changed token can create at most two red-list violations (itself and the next token, whose list it seeds). Even a maximally adversarial edit of 200 of 1,000 tokens leaves 600 green tokens, \(z \approx 6.3\); removal needs changing roughly a quarter of the tokens, and an attacker who doesn’t know the lists does far worse.

It forbids red tokens even when they are the only sensible continuation (e.g. “Obama” after “Barack”), degrading low-entropy text.

Instead of forbidding red tokens, it adds a constant \(\delta\) to the logits of green-list tokens (a fraction \(\gamma\) of the vocabulary) before the softmax. Near-certain tokens stay likely even if red, while in high-entropy steps the bias strongly favours green tokens.

\(\gamma\) is the green-list fraction of the vocabulary, \(\delta\) the logit bias (hardness). Smaller \(\gamma\) and larger \(\delta\) give a stronger watermark from fewer tokens but distort the text more (higher perplexity); \(\delta \to \infty\) recovers the hard rule.

Detection uses the same z-test, but the green-token excess depends on entropy: high-entropy text is detected from few tokens, while low-entropy text (often memorized or near-copied human text) needs many tokens or isn’t detectable at all.

A measure of how spread out a distribution is, roughly a softened count of entries with probability above \(1/z\); minimal when all mass is on one token. It bounds how many green tokens a watermarked model is expected to produce, and so how detectable its output is.

At both extremes: if the model’s distribution is uniform, a random green list still gives uniform sampling; if all mass is on one token, the bias doesn’t change the choice. Perplexity is affected only for tokens of moderate entropy.

The red lists depend on a secret key behind an API, so attackers don’t know which tokens to replace; but detection then requires the API, and access must be monitored so that an adversary can’t probe it with many slightly varied texts.

With \(\gamma = 0.5\), \(\delta = 2\) and \(z = 4\) on about 200 generated tokens, 98.4% of watermarked texts were detected with multinomial sampling (99.6% with 4-way beam search), and essentially no human text was flagged.

It “irons in” the watermark: searching among likely continuations finds ones dense in green tokens, giving much higher z-scores at little perplexity cost; with 8 beams and \(\delta = 2\), average \(z > 5\) was reached from about 35 tokens.

The z-statistic depends only on \(\gamma\) and the hash function, not on \(\delta\) or how green tokens are favoured, so \(\delta\) can vary by context (e.g. prose vs code) or the generator can change without updating the detector.

Replacing random spans with T5-Large suggestions, without knowing the green lists, barely helped at a 10% budget (AUC dropped by 0.01); at 30% it degraded the watermark substantially, but tripled perplexity and needed many model calls.

Post-hoc detectors lose ground as models improve (GPT-2 detectors struggle with GPT-3), are vulnerable to adversarial attacks, and risk falsely flagging human text, especially from non-native speakers or writers using assistance tools.