Towards Monosemanticity: Decomposing Language Models With Dictionary Learning
Bricken et al., 2023, Towards Monosemanticity (Transformer Circuits Thread)
Subscriber deck — free during early access
1 cards in “Features as a Decomposition” · whole deck
Features as a Decomposition
- How is an activation vector decomposed into features?\(\mathbf{x}^j \approx \mathbf{b} + \sum_i f_i(\mathbf{x}^j)\, \mathbf{d}_i\): a bias plus a sparse weighted sum of feature directions \(\mathbf{d}_i\) (unit vectors, the decoder’s columns), with feature activations \(f_i(\mathbf{x}) = \mathrm{ReLU}(W_e(\mathbf{x} - \mathbf{b}_d) + \mathbf{b}_e)_i\) from the encoder.
Few features are active at once (highlighted); loss = reconstruction error + L1 on the features - Bricken et al., 2023, Towards Monosemanticity (Transformer Circuits Thread), Features as a Decomposition
Source
- Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nicholas L. Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E. Burke, Tristan Hume, Shan Carter, Tom Henighan and Chris Olah. 2023. Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. In Transformer Circuits Thread. https://transformer-circuits.pub/2023/monosemantic-features/index.html