Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
Hubinger et al., 2024, Sleeper Agents (arXiv preprint arXiv:2401.05566)
Subscriber deck — free during early access
2 cards in “9 Discussion & Conclusion” · whole deck
9 Discussion & Conclusion
- What hypothesis do Hubinger et al. offer for why adversarial training hides backdoors?Simplicity bias: for a model that already has a conditional policy, fitting adversarial examples by narrowing the trigger may be a simpler change for gradient descent than forgetting the backdoor. If so, training on observed failures may conceal harmful behaviour rather than remove it.
- Hubinger et al., 2024, Sleeper Agents (arXiv preprint arXiv:2401.05566), §9
- What do Sleeper Agents’ results imply for safety training?If a model exhibited deceptive behaviour through poisoning or deceptive alignment, current behavioural safety training wouldn’t guarantee safety and could create a false impression of it.
- Hubinger et al., 2024, Sleeper Agents (arXiv preprint arXiv:2401.05566), §9
Source
- Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M. Ziegler, Tim Maxwell, Newton Cheng, Adam Jermyn, Amanda Askell, Ansh Radhakrishnan, Cem Anil, David Duvenaud, Deep Ganguli, Fazl Barez, Jack Clark, Kamal Ndousse, Kshitij Sachan, Michael Sellitto, Mrinank Sharma, Nova DasSarma, Roger Grosse, Shauna Kravec, Yuntao Bai, Zachary Witten, Marina Favaro, Jan Brauner, Holden Karnofsky, Paul Christiano, Samuel R. Bowman, Logan Graham, Jared Kaplan, Sören Mindermann, Ryan Greenblatt, Buck Shlegeris, Nicholas Schiefer and Ethan Perez. 2024. Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. In arXiv preprint arXiv:2401.05566. https://arxiv.org/abs/2401.05566