A reading note on how SAEs decompose a model's activations into a dictionary of features: the superposition geometry that makes them necessary, the four ways the dictionary goes wrong (absorption, hedging, splitting, shrinkage), and the sanity checks SAE benchmarks have since had to survive.
The full note has two interactive figures.
The full note has the complete argument and its interactive figures.
Read the full note