Alignment Faking in Large Language Models
Greenblatt et al., 2024, Alignment Faking in Large Language Models (arXiv preprint arXiv:2412.14093)
Subscriber deck — free during early access
1 cards in “4 Synthetic Document Fine-Tuning” · whole deck
4 Synthetic Document Fine-Tuning
- What is the synthetic-document setting, and what did it show?Instead of a system prompt, the model was fine-tuned on realistic fake news articles and blog posts containing the same facts about its training, mimicking how it might learn them in pretraining. Alignment faking still appeared, and a compliance gap (20% in helpful-only) remained even with no scratchpad at all: reasoning that never appeared in writing.
- Greenblatt et al., 2024, Alignment Faking in Large Language Models (arXiv preprint arXiv:2412.14093), §4
Source
- Ryan Greenblatt, Carson Denison, Benjamin Wright, Fabien Roger, Monte MacDiarmid, Sam Marks, Johannes Treutlein, Tim Belonax, Jack Chen, David Duvenaud, Akbir Khan, Julian Michael, Sören Mindermann, Ethan Perez, Linda Petrini, Jonathan Uesato, Jared Kaplan, Buck Shlegeris, Samuel R. Bowman and Evan Hubinger. 2024. Alignment Faking in Large Language Models. In arXiv preprint arXiv:2412.14093. https://arxiv.org/abs/2412.14093