On the Biology of a Large Language Model
Lindsey et al., 2025, On the Biology of a Large Language Model (Transformer Circuits Thread)
Subscriber deck — free during early access
1 cards in “Limitations” · whole deck
Limitations
- What are the main limitations of attribution graphs, according to Lindsey et al.?They gave satisfying insight for only about a quarter of prompts tried. They do not explain how attention patterns are computed; error nodes leave “dark matter” unexplained; inactive features and inhibition are hard to see; they work for short prompts (about a hundred tokens) and one output token at a time; and the replacement model may not be mechanistically faithful.
- Lindsey et al., 2025, On the Biology of a Large Language Model (Transformer Circuits Thread), Limitations
Source
- Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi, T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah and Joshua Batson. 2025. On the Biology of a Large Language Model. In Transformer Circuits Thread. https://transformer-circuits.pub/2025/attribution-graphs/biology.html