Towards Monosemanticity: Decomposing Language Models With Dictionary Learning
Bricken et al., 2023, Towards Monosemanticity (Transformer Circuits Thread)
Subscriber deck — free during early access
2 cards in “Detailed Investigations of Individual Features” · whole deck
Detailed Investigations of Individual Features
- What five claims did Bricken et al. try to establish for features such as Arabic script, DNA, base64 and Hebrew?The feature fires specifically on its context; it fires whenever the context is present (sensitivity); it causes appropriate downstream behaviour; it doesn’t correspond to any single neuron; and a similar feature appears when dictionary learning is applied to a different model (universality).
- Bricken et al., 2023, Towards Monosemanticity (Transformer Circuits Thread), Detailed Investigations of Individual Features
- Why is the Arabic-script feature not visible in the neuron basis?Its direction is spread over many neurons: its three largest neuron coefficients are all negative and 27 neurons have coefficients of at least 0.1, so no individual neuron tracks Arabic text.
- Bricken et al., 2023, Towards Monosemanticity (Transformer Circuits Thread), Detailed Investigations of Individual Features
Source
- Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nicholas L. Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E. Burke, Tristan Hume, Shan Carter, Tom Henighan and Chris Olah. 2023. Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. In Transformer Circuits Thread. https://transformer-circuits.pub/2023/monosemantic-features/index.html