On the Biology of a Large Language Model
Lindsey et al., 2025, On the Biology of a Large Language Model (Transformer Circuits Thread)
Subscriber deck — free during early access
1 cards in “Introduction” · whole deck
Introduction
- What is the goal of “On the Biology of a Large Language Model”, and why the biology analogy?To reverse-engineer the internal mechanisms of a production model, Claude 3.5 Haiku, across many behaviors. Like organisms shaped by evolution, models are produced by simple training rules yet end up with intricate mechanisms, and progress depends on tools, here a kind of microscope for features and their connections.
- Lindsey et al., 2025, On the Biology of a Large Language Model (Transformer Circuits Thread), Introduction
Source
- Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi, T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah and Joshua Batson. 2025. On the Biology of a Large Language Model. In Transformer Circuits Thread. https://transformer-circuits.pub/2025/attribution-graphs/biology.html