A Watermark for Large Language Models
Kirchenbauer et al., 2023, A Watermark for Large Language Models (Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202)
Subscriber deck — free during early access
1 cards in “4 Analysis of the soft watermark” · whole deck
4 Analysis of the soft watermark
- What is spike entropy, and why do Kirchenbauer et al. use it?A measure of how spread out a distribution is, roughly a softened count of entries with probability above \(1/z\); minimal when all mass is on one token. It bounds how many green tokens a watermarked model is expected to produce, and so how detectable its output is.
- Kirchenbauer et al., 2023, A Watermark for Large Language Models (Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202), §4
Source
- John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers and Tom Goldstein. 2023. A Watermark for Large Language Models. In Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202, pp. 17061–17084. https://proceedings.mlr.press/v202/kirchenbauer23a.html