Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet
Templeton et al., 2024, Scaling Monosemanticity (Transformer Circuits Thread)
Subscriber deck — free during early access
2 cards in “Feature Survey” · whole deck
Feature Survey
- What is the relationship between feature geometry and meaning?Nearby feature vectors (high cosine similarity) have related meanings: around the Golden Gate Bridge feature are Alcatraz and the Presidio, then Lake Tahoe and Yosemite, then tourist sites elsewhere. Features also split: a “San Francisco” feature in the 1M SAE became 2 in the 4M and 11 in the 34M SAE.
Larger dictionaries resolve one concept into finer, related features (counts from the paper; the grouping is illustrative) - Templeton et al., 2024, Scaling Monosemanticity (Transformer Circuits Thread), Feature Survey
- Did the 34M-feature SAE capture everything Sonnet knows?No: Sonnet could list all London boroughs, but features were found for only about 60% of them. Whether a concept gets a dedicated feature depends on its frequency in the training data; concepts rarer than about one in the number of alive features usually lack one.
- Templeton et al., 2024, Scaling Monosemanticity (Transformer Circuits Thread), Feature Survey
Source
- Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L. Turner, Callum McDougall, Monte MacDiarmid, Alex Tamkin, Esin Durmus, Tristan Hume, Francesco Mosconi, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah and Tom Henighan. 2024. Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet. In Transformer Circuits Thread. https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html