Towards Monosemanticity: Decomposing Language Models With Dictionary Learning
Bricken et al., 2023, Towards Monosemanticity (Transformer Circuits Thread)
Subscriber deck — free during early access
1 cards in “Why not use architectural approaches?” · whole deck
Why not use architectural approaches?
- Why don’t Bricken et al. simply train models without superposition?Even with extreme activation sparsity (one-hot activations) neurons stayed polysemantic: under cross-entropy loss a neuron covering two features ambiguously (loss \(\ln 2 \approx 0.7\) in a toy case) beats one covering a single feature cleanly (\(\tfrac34 \ln 3 \approx 0.8\)). Polysemanticity can arise without superposition.
- Bricken et al., 2023, Towards Monosemanticity (Transformer Circuits Thread), Why not use architectural approaches?
Source
- Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nicholas L. Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E. Burke, Tristan Hume, Shan Carter, Tom Henighan and Chris Olah. 2023. Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. In Transformer Circuits Thread. https://transformer-circuits.pub/2023/monosemantic-features/index.html