A Watermark for Large Language Models
Kirchenbauer et al., 2023, A Watermark for Large Language Models (Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202)
Subscriber deck — free during early access
1 cards in “1.2 A caveat: The difficulty of watermarking low-entropy sequences” · whole deck
1.2 A caveat: The difficulty of watermarking low-entropy sequences
- Why is low-entropy text hard to watermark?When the first tokens nearly determine the rest (e.g. a famous pangram or a standard for-loop), humans and models produce the same completion, so the source can’t be told apart; and forcing different tokens would make the text unnatural.
- Kirchenbauer et al., 2023, A Watermark for Large Language Models (Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202), §1.2
Source
- John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers and Tom Goldstein. 2023. A Watermark for Large Language Models. In Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202, pp. 17061–17084. https://proceedings.mlr.press/v202/kirchenbauer23a.html