Towards Monosemanticity: Decomposing Language Models With Dictionary Learning
Bricken et al., 2023, Towards Monosemanticity (Transformer Circuits Thread)
Subscriber deck — free during early access
1 cards in “How much of the model does our interpretation explain?” · whole deck
How much of the model does our interpretation explain?
- How much of the MLP’s contribution do the features capture?Replacing the MLP activations with the 4,096-feature autoencoder’s reconstruction recovers 79% of the loss reduction the MLP provides; a 131,072-feature dictionary recovers 94.5%. The authors warn that a long tail of features likely remains.
- Bricken et al., 2023, Towards Monosemanticity (Transformer Circuits Thread), How much of the model does our interpretation explain?
Source
- Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nicholas L. Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E. Burke, Tristan Hume, Shan Carter, Tom Henighan and Chris Olah. 2023. Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. In Transformer Circuits Thread. https://transformer-circuits.pub/2023/monosemantic-features/index.html