On the Biology of a Large Language Model
Lindsey et al., 2025, On the Biology of a Large Language Model (Transformer Circuits Thread)
Subscriber deck — free during early access
1 cards in “Chain-of-thought Faithfulness” · whole deck
Chain-of-thought Faithfulness
- What kinds of chain of thought did Lindsey et al. distinguish mechanistically?Faithful: for √0.64 the model really computes √64. “Bullshitting”: for cos(23423) it claims a calculation but the graph shows a guess. Motivated reasoning: given the user’s claimed answer, it works backwards from that hint to pick intermediate values that will reach it.
- Lindsey et al., 2025, On the Biology of a Large Language Model (Transformer Circuits Thread), Chain-of-thought Faithfulness
Source
- Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi, T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah and Joshua Batson. 2025. On the Biology of a Large Language Model. In Transformer Circuits Thread. https://transformer-circuits.pub/2025/attribution-graphs/biology.html