Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet
Templeton et al., 2024, Scaling Monosemanticity (Transformer Circuits Thread)
Subscriber deck — free during early access
2 cards in “Four Examples of Interpretable Features” · whole deck
Four Examples of Interpretable Features
- What evidence showed that features like the Golden Gate Bridge feature are genuinely about their concept?Strong activations were almost all on that concept (judged with automated rubric scoring by Claude 3 Opus); it fired on the bridge’s Wikipedia article in many languages and on images of the bridge, although training used only text.
- Templeton et al., 2024, Scaling Monosemanticity (Transformer Circuits Thread), Four Examples of Interpretable Features
- What is feature steering, and what happened when the Golden Gate Bridge feature was clamped high?Fixing a feature’s activation to an artificial value during the forward pass to test its causal role. Clamped to 10× its maximum, the Golden Gate Bridge feature made the model talk about the bridge and even identify itself as the Golden Gate Bridge.
- Templeton et al., 2024, Scaling Monosemanticity (Transformer Circuits Thread), Four Examples of Interpretable Features
Source
- Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L. Turner, Callum McDougall, Monte MacDiarmid, Alex Tamkin, Esin Durmus, Tristan Hume, Francesco Mosconi, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah and Tom Henighan. 2024. Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet. In Transformer Circuits Thread. https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html