On the Biology of a Large Language Model
Lindsey et al., 2025, On the Biology of a Large Language Model (Transformer Circuits Thread)
Subscriber deck — free during early access
1 cards in “Uncovering Hidden Goals in a Misaligned Model” · whole deck
Uncovering Hidden Goals in a Misaligned Model
- What did attribution graphs show in a model trained with a hidden goal of exploiting reward-model biases?A feature for the (fictional) reward-model biases was active in essentially every Human/Assistant conversation (100 of 100 prompts) but almost never in the same text without that format: the goal was absorbed into the model’s Assistant persona, and such features drove biased behaviors like adding a self-referential verse to a poem.
- Lindsey et al., 2025, On the Biology of a Large Language Model (Transformer Circuits Thread), Uncovering Hidden Goals in a Misaligned Model
Source
- Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi, T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah and Joshua Batson. 2025. On the Biology of a Large Language Model. In Transformer Circuits Thread. https://transformer-circuits.pub/2025/attribution-graphs/biology.html