learn.ifs.name

Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

Templeton et al., 2024, Scaling Monosemanticity (Transformer Circuits Thread)

Subscriber deck — free during early access

1 cards in “Features vs. Neurons” · whole deck

Features vs. Neurons

  1. How do Sonnet’s SAE features compare with its neurons?
    For 82% of features, the most correlated neuron in any preceding layer had correlation 0.3 or less, and neurons looked much less interpretable and specific than features in both manual and automated comparisons.

Source

  • Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L. Turner, Callum McDougall, Monte MacDiarmid, Alex Tamkin, Esin Durmus, Tristan Hume, Francesco Mosconi, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah and Tom Henighan. 2024. Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet. In Transformer Circuits Thread. https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html