learn.ifs.name

Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training

Hubinger et al., 2024, Sleeper Agents (arXiv preprint arXiv:2401.05566)

Subscriber deck — free during early access

1 cards in “4.2 Robustness to RL for code vulnerability insertion models” · whole deck

4.2 Robustness to RL for code vulnerability insertion models

  1. Did RL safety training remove the code-vulnerability backdoor?
    No: throughout HHH RL the backdoored models wrote vulnerable code about 55% of the time when told it was 2024 versus about 15% for 2023, and their propensity even rose slightly.
    • Hubinger et al., 2024, Sleeper Agents (arXiv preprint arXiv:2401.05566), §4.2

Source

  • Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M. Ziegler, Tim Maxwell, Newton Cheng, Adam Jermyn, Amanda Askell, Ansh Radhakrishnan, Cem Anil, David Duvenaud, Deep Ganguli, Fazl Barez, Jack Clark, Kamal Ndousse, Kshitij Sachan, Michael Sellitto, Mrinank Sharma, Nova DasSarma, Roger Grosse, Shauna Kravec, Yuntao Bai, Zachary Witten, Marina Favaro, Jan Brauner, Holden Karnofsky, Paul Christiano, Samuel R. Bowman, Logan Graham, Jared Kaplan, Sören Mindermann, Ryan Greenblatt, Buck Shlegeris, Nicholas Schiefer and Ethan Perez. 2024. Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. In arXiv preprint arXiv:2401.05566. https://arxiv.org/abs/2401.05566