Sparse Autoencoders (SAEs)
One hidden state mixes many ideas in a few numbers. A sparse autoencoder spreads it across thousands of features so only a few fire at once and you can read them.
Why Does This Exist?
You look at one direction inside a chat model and it fires for dogs, for the Eiffel Tower, for C++ brackets, and for sad goodbyes. That is not four jobs done well. It is one wire carrying four signals at once, so you cannot tell which idea drove the next word.
The name for that tangle is superposition: the model packs more ideas than it has numbers to hold them, so ideas share directions. You have felt the cost if you have tried to ask why a model refused, quoted a fact, or turned hostile. The raw numbers do not answer. They are readable as arithmetic and unreadable as reasons.
Here is the one idea you need first. An autoencoder squeezes an input through a narrow code and then rebuilds the input from that code. What survives the squeeze names the structure. A sparse autoencoder flips the squeeze. It spreads one dense model state across a far wider code, then forces only a few code numbers to fire at once. Few active numbers means each one can stand for one thing.
This page covers that spread, the sparsity rule, and the rebuild. It does not cover control or proof. For control, read activation steering, which adds a chosen direction for one pass. For causal proof, read activation patching, which swaps states between runs. For a quick peek at half formed next word odds, read logit lens.
Think of It Like This
One microphone for the whole choir
You record a school choir with one microphone hanging from the ceiling. Play it back and you hear everything and can name nothing. That single track is the dense state. Loud means many voices, not one voice you can point at.
Now you hand the tape to an engineer with a mixing board that has one fader per singer. She sets the board so only two faders move at any second and the mixed output still sounds like your tape. Those moving faders are the sparse features. You can finally say the altos carry this bar.
Where it stops: her faders are not the singers. A real feature can split one idea across two faders or merge two ideas into one when your sparsity setting is wrong. And a fader sitting at zero does not prove the idea is absent. It proves the board chose not to use it for this rebuild.
How It Actually Works
Take a dense activation , meaning one hidden state vector with numbers, so lives in . You learn a dictionary with features where is far larger than . Write for the encoder matrix with shape by , meaning it maps dense to wide. Write for its bias with numbers. Write for the sparse code with numbers. Write for the decoder matrix with shape by , whose columns are the feature directions. Write for the decoder bias with numbers. Write for the rebuild with numbers. Write for the sparsity price, a small positive number.
Encoding scores every feature, then keeps only a few:
Here holds the raw scores with numbers, one per dictionary feature. The classic choice applies , meaning negatives become zero and positives pass through. You then pay , meaning times the sum of absolute values, so small scores get pushed to exactly zero. The newer choice skips and keeps the top values of , meaning the largest scores survive and the rest become zero. Either way names how many features may fire, and says exactly entries are nonzero.
Decoding mixes only the survivors back to dense width:
Training keeps two goals at once:
In words: squared rebuild error plus sparsity price times code size. Columns of are kept at unit length during training, or one long direction would fake a small code value while carrying large meaning.
The parts and the loss
Think of as the dictionary and as which entries you looked up. If has entries , you used entry one with strength and entry three with strength . The rebuild is times column one plus times column three. The loss asks whether that short mix matches the full . Tighten sparsity and labels get cleaner while the rebuild gets worse. Loosen it and the rebuild improves while features turn polysemantic again. That tug is the whole trade.
Do not reach for this when you need proof of cause or a fix for a small model. An SAE suggests which directions to test. It does not prove they drive behavior. If your model fits on one screen, read its weights directly. And do not steer from a label you have not tested, because a nice name can hide a split or merged feature.
The bet underneath is that is close to a sparse sum of dictionary directions plus noise. When the model truly spreads one decision across hundreds of weak directions, or builds it from a hard nonlinearity the dictionary cannot copy, the bet fails. Then you see many tiny nonzero values, high rebuild error, or one idea shattered across lookalike features.
Worked example with tiny numbers
Keep dense numbers and dictionary features, with survivors. Take:
Use these five encoder rows and biases. Each row scores one feature:
Multiply and add. Row one gives . Row two gives . Row three gives . Row four gives . Row five gives . So:
ReLU zeroes the negative, giving . Top keeps and :
Let the first and third decoder columns be and , with . The other columns do not matter here because their code values are zero. Then:
Error by slot is . Squared error is . Code size . With the training loss is . You paid in lost detail to say this state with two named parts.
Code
This snippet repeats the worked example so you can check the arithmetic. It uses the same , , , , and basis decoder columns.
import numpy as np
def encode_topk(x: np.ndarray, w_enc: np.ndarray, b_enc: np.ndarray, k: int) -> np.ndarray: z: np.ndarray = w_enc @ x + b_enc f: np.ndarray = np.maximum(0.0, z) thresh: float = float(np.sort(f)[-k]) # kth largest value return np.where(f >= thresh, f, 0.0)
def decode(f: np.ndarray, w_dec: np.ndarray) -> np.ndarray: return w_dec @ f
x = np.array([0.8, -0.3, 0.5])w_enc = np.array([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0], [1.0, 1.0, 0.0], [0.0, -1.0, 1.0]])b_enc = np.array([-0.2, 0.0, -0.1, -0.4, -0.6])w_dec = np.array([[1.0, 0.0, 0.0, 1.0, 0.0], [0.0, 1.0, 0.0, 1.0, -1.0], [0.0, 0.0, 1.0, 0.0, 1.0]])
f = encode_topk(x, w_enc, b_enc, k=2)x_hat = decode(f, w_dec)err: float = float(((x - x_hat) ** 2).sum())loss: float = err + 0.05 * float(np.abs(f).sum())
print(f.tolist()) # -> [0.6, 0.0, 0.4, 0.0, 0.0]print(x_hat.tolist()) # -> [0.6, 0.0, 0.4]print(round(err, 2)) # -> 0.14print(round(loss, 2)) # -> 0.19The prints match the hand run: two survivors, rebuild , error , loss at .
Watch Out For
Most of the dictionary goes dead
You train, rebuild error looks fine, then you count firing rates and 8 in 10 features never fire. The cause is usually a high sparsity price or dead initialization: pushed small features to zero early, their decoder columns stopped getting gradients, and they never recovered.
You will see it as a flat histogram with a tall bar at zero firing plus a few overused features. Fix it by lowering or raising first, keeping decoder columns at unit length, and re seeding columns that fired on fewer than about one in a million inputs. Track firing rate per feature from the start, not just mean loss.
You trust the auto label
A feature fires on your ten dog sentences, the labeling model names it dogs, then you zero it and the model still completes the dog sentence. Or you clamp it high and get Eiffel Tower trivia. The label described your sample, not the feature. Common culprits are feature splitting, where dogs live across two lookalikes, and shrinkage, where L1 pulled strengths down so your clamp was too weak.
Treat every label as a guess. Test it on held out sentences, then confirm with activation patching or a small activation steering sweep before you claim meaning. If the effect does not move with the knob, rename or drop the feature.
The Quick Version
- A sparse autoencoder spreads one dense hidden state across a wide dictionary so only a few features fire per input.
- The loss is rebuild error plus a sparsity price, or a hard TopK rule that keeps exactly features alive.
- In the tiny run here, keeps alive and rebuilds with squared error .
- Tighter sparsity buys cleaner names at the cost of worse rebuilds, wider dictionaries buy room at the cost of dead features and compute.
- Labels are guesses until a patch or steer moves behavior, so test before you trust them.