learn.ifs.name

Towards Monosemanticity: Decomposing Language Models With Dictionary Learning

Bricken et al., 2023, Towards Monosemanticity (Transformer Circuits Thread)

Subscriber deck — free during early access

1 cards in “Universality” · whole deck

Universality

  1. How universal were the learned features across two separately trained transformers?
    Features from model A had a median activation correlation of 0.72 with their closest feature in model B, far more than the closest neurons (0.46): features are more similar across models than to their own model’s neurons.

Source

  • Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nicholas L. Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E. Burke, Tristan Hume, Shan Carter, Tom Henighan and Chris Olah. 2023. Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. In Transformer Circuits Thread. https://transformer-circuits.pub/2023/monosemantic-features/index.html