Towards Monosemanticity: Decomposing Language Models With Dictionary Learning
Bricken et al., 2023, Towards Monosemanticity (Transformer Circuits Thread)
Subscriber deck — free during early access
2 cards in “Problem Setup” · whole deck
Problem Setup
- What problem does Towards Monosemanticity address?Neurons in language models are often polysemantic (they respond to several unrelated things), so they are poor units for reverse-engineering a model. The paper decomposes MLP activations into many more interpretable “features” than there are neurons.
- Bricken et al., 2023, Towards Monosemanticity (Transformer Circuits Thread), Problem Setup
- What model and what layer did Bricken et al. study?A one-layer transformer with a 512-neuron ReLU MLP trained on the Pile, “the simplest language model we profoundly don’t understand”: one-layer models are cheap to overtrain and their features’ effects on the output logits are nearly linear and easy to read.
- Bricken et al., 2023, Towards Monosemanticity (Transformer Circuits Thread), Problem Setup
Source
- Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nicholas L. Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E. Burke, Tristan Hume, Shan Carter, Tom Henighan and Chris Olah. 2023. Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. In Transformer Circuits Thread. https://transformer-circuits.pub/2023/monosemantic-features/index.html