A Watermark for Large Language Models

1 Introduction

1. What is a watermark for language model output?
   A hidden pattern in generated text that humans can’t perceive but an algorithm can detect, marking the text as machine-generated; Kirchenbauer et al. aim to detect it from short spans (as few as about 25 tokens) with statistically improbable false positives.
   • Kirchenbauer et al., 2023, A Watermark for Large Language Models (Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202), §1
2. What properties do Kirchenbauer et al. want from a watermark?
   Detectable without the model’s parameters or API (so the detector can be public and cheap); applicable to a standard model without retraining; detectable from any contiguous portion of the text; not removable without changing a large fraction of tokens; and with a rigorous statistical confidence measure.
   • Kirchenbauer et al., 2023, A Watermark for Large Language Models (Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202), §1