Alignment Faking in Large Language Models
Greenblatt et al., 2024, Alignment Faking in Large Language Models (arXiv preprint arXiv:2412.14093)
Subscriber deck — free during early access
1 cards in “3.3.1 Prompt Variations” · whole deck
3.3.1 Prompt Variations
- How did Greenblatt et al. show the effect was not just priming from the prompt?Removing any mention of the conflicting training objective eliminated alignment faking, but paraphrasing the prompt, dropping the explanation of how RLHF modifies models, or telling the model to be honest and not to care about refusing did not. So the driver is the conflict between existing preferences and training.
- Greenblatt et al., 2024, Alignment Faking in Large Language Models (arXiv preprint arXiv:2412.14093), §3.3.1
Source
- Ryan Greenblatt, Carson Denison, Benjamin Wright, Fabien Roger, Monte MacDiarmid, Sam Marks, Johannes Treutlein, Tim Belonax, Jack Chen, David Duvenaud, Akbir Khan, Julian Michael, Sören Mindermann, Ethan Perez, Linda Petrini, Jonathan Uesato, Jared Kaplan, Buck Shlegeris, Samuel R. Bowman and Evan Hubinger. 2024. Alignment Faking in Large Language Models. In arXiv preprint arXiv:2412.14093. https://arxiv.org/abs/2412.14093