Skip to content
AI360XpertBeta

Sparse Autoencoders (SAEs)

One hidden state mixes many ideas in a few numbers. A sparse autoencoder spreads it across thousands of features so only a few fire at once and you can read them.

A 3-number dense state expands to 5 features where only 2 fire, then rebuilds as [0.6, 0, 0.4] with error 0.14
A 3-number dense state expands to 5 features where only 2 fire, then rebuilds as [0.6, 0, 0.4] with error 0.14

Why Does This Exist?

You look at one direction inside a chat model and it fires for dogs, for the Eiffel Tower, for C++ brackets, and for sad goodbyes. That is not four jobs done well. It is one wire carrying four signals at once, so you cannot tell which idea drove the next word.

The name for that tangle is superposition: the model packs more ideas than it has numbers to hold them, so ideas share directions. You have felt the cost if you have tried to ask why a model refused, quoted a fact, or turned hostile. The raw numbers do not answer. They are readable as arithmetic and unreadable as reasons.

Here is the one idea you need first. An autoencoder squeezes an input through a narrow code and then rebuilds the input from that code. What survives the squeeze names the structure. A sparse autoencoder flips the squeeze. It spreads one dense model state across a far wider code, then forces only a few code numbers to fire at once. Few active numbers means each one can stand for one thing.

This page covers that spread, the sparsity rule, and the rebuild. It does not cover control or proof. For control, read activation steering, which adds a chosen direction for one pass. For causal proof, read activation patching, which swaps states between runs. For a quick peek at half formed next word odds, read logit lens.

Think of It Like This

One microphone for the whole choir

You record a school choir with one microphone hanging from the ceiling. Play it back and you hear everything and can name nothing. That single track is the dense state. Loud means many voices, not one voice you can point at.

Now you hand the tape to an engineer with a mixing board that has one fader per singer. She sets the board so only two faders move at any second and the mixed output still sounds like your tape. Those moving faders are the sparse features. You can finally say the altos carry this bar.

Where it stops: her faders are not the singers. A real feature can split one idea across two faders or merge two ideas into one when your sparsity setting is wrong. And a fader sitting at zero does not prove the idea is absent. It proves the board chose not to use it for this rebuild.

How It Actually Works

Take a dense activation xx, meaning one hidden state vector with dd numbers, so xx lives in RdR^d. You learn a dictionary with mm features where mm is far larger than dd. Write WencW_{\text{enc}} for the encoder matrix with shape mm by dd, meaning it maps dense to wide. Write bencb_{\text{enc}} for its bias with mm numbers. Write ff for the sparse code with mm numbers. Write WdecW_{\text{dec}} for the decoder matrix with shape dd by mm, whose columns are the feature directions. Write bdecb_{\text{dec}} for the decoder bias with dd numbers. Write x^\hat{x} for the rebuild with dd numbers. Write λ\lambda for the sparsity price, a small positive number.

Encoding scores every feature, then keeps only a few:

z=Wencx+bencz = W_{\text{enc}} x + b_{\text{enc}}

Here zz holds the raw scores with mm numbers, one per dictionary feature. The classic choice applies f=ReLU(z)f = \text{ReLU}(z), meaning negatives become zero and positives pass through. You then pay λ∥f∥1\lambda \lVert f \rVert_1, meaning λ\lambda times the sum of absolute values, so small scores get pushed to exactly zero. The newer choice skips λ\lambda and keeps the top kk values of zz, meaning the kk largest scores survive and the rest become zero. Either way kk names how many features may fire, and ∥f∥0=k\lVert f \rVert_0 = k says exactly kk entries are nonzero.

Decoding mixes only the survivors back to dense width:

x^=Wdecf+bdec\hat{x} = W_{\text{dec}} f + b_{\text{dec}}

Training keeps two goals at once:

L=∥x−x^∥22+λ∥f∥1L = \lVert x - \hat{x} \rVert_2^2 + \lambda \lVert f \rVert_1

In words: squared rebuild error plus sparsity price times code size. Columns of WdecW_{\text{dec}} are kept at unit length during training, or one long direction would fake a small code value while carrying large meaning.

The parts and the loss

Think of WdecW_{\text{dec}} as the dictionary and ff as which entries you looked up. If ff has entries [0.6,0,0.4,0,0][0.6, 0, 0.4, 0, 0], you used entry one with strength 0.60.6 and entry three with strength 0.40.4. The rebuild is 0.60.6 times column one plus 0.40.4 times column three. The loss asks whether that short mix matches the full xx. Tighten sparsity and labels get cleaner while the rebuild gets worse. Loosen it and the rebuild improves while features turn polysemantic again. That tug is the whole trade.

Do not reach for this when you need proof of cause or a fix for a small model. An SAE suggests which directions to test. It does not prove they drive behavior. If your model fits on one screen, read its weights directly. And do not steer from a label you have not tested, because a nice name can hide a split or merged feature.

The bet underneath is that xx is close to a sparse sum of dictionary directions plus noise. When the model truly spreads one decision across hundreds of weak directions, or builds it from a hard nonlinearity the dictionary cannot copy, the bet fails. Then you see many tiny nonzero values, high rebuild error, or one idea shattered across lookalike features.

Worked example with tiny numbers

Keep d=3d = 3 dense numbers and m=5m = 5 dictionary features, with k=2k = 2 survivors. Take:

x=[0.8,−0.3,0.5]x = [0.8, -0.3, 0.5]

Use these five encoder rows and biases. Each row scores one feature:

Wenc=[[1,0,0],[0,1,0],[0,0,1],[1,1,0],[0,−1,1]],benc=[−0.2,0.0,−0.1,−0.4,−0.6]W_{\text{enc}} = [[1, 0, 0], [0, 1, 0], [0, 0, 1], [1, 1, 0], [0, -1, 1]], \quad b_{\text{enc}} = [-0.2, 0.0, -0.1, -0.4, -0.6]

Multiply and add. Row one gives 0.8−0.2=0.60.8 - 0.2 = 0.6. Row two gives −0.3+0.0=−0.3-0.3 + 0.0 = -0.3. Row three gives 0.5−0.1=0.40.5 - 0.1 = 0.4. Row four gives 0.8−0.3−0.4=0.10.8 - 0.3 - 0.4 = 0.1. Row five gives 0.3+0.5−0.6=0.20.3 + 0.5 - 0.6 = 0.2. So:

z=[0.6,−0.3,0.4,0.1,0.2]z = [0.6, -0.3, 0.4, 0.1, 0.2]

ReLU zeroes the negative, giving [0.6,0,0.4,0.1,0.2][0.6, 0, 0.4, 0.1, 0.2]. Top 22 keeps 0.60.6 and 0.40.4:

f=[0.6,0,0.4,0,0]f = [0.6, 0, 0.4, 0, 0]

Let the first and third decoder columns be [1,0,0][1, 0, 0] and [0,0,1][0, 0, 1], with bdec=[0,0,0]b_{\text{dec}} = [0, 0, 0]. The other columns do not matter here because their code values are zero. Then:

x^=0.6[1,0,0]+0.4[0,0,1]=[0.6,0,0.4]\hat{x} = 0.6 [1, 0, 0] + 0.4 [0, 0, 1] = [0.6, 0, 0.4]

Error by slot is [0.2,−0.3,0.1][0.2, -0.3, 0.1]. Squared error is 0.04+0.09+0.01=0.140.04 + 0.09 + 0.01 = 0.14. Code size ∥f∥1=1.0\lVert f \rVert_1 = 1.0. With λ=0.05\lambda = 0.05 the training loss is 0.14+0.05=0.190.14 + 0.05 = 0.19. You paid 0.140.14 in lost detail to say this state with two named parts.

Code

This snippet repeats the worked example so you can check the arithmetic. It uses the same xx, WencW_{\text{enc}}, bencb_{\text{enc}}, k=2k = 2, and basis decoder columns.

import numpy as np

def encode_topk(x: np.ndarray, w_enc: np.ndarray, b_enc: np.ndarray, k: int) -> np.ndarray:    z: np.ndarray = w_enc @ x + b_enc    f: np.ndarray = np.maximum(0.0, z)    thresh: float = float(np.sort(f)[-k])  # kth largest value    return np.where(f >= thresh, f, 0.0)

def decode(f: np.ndarray, w_dec: np.ndarray) -> np.ndarray:    return w_dec @ f

x = np.array([0.8, -0.3, 0.5])w_enc = np.array([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0], [1.0, 1.0, 0.0], [0.0, -1.0, 1.0]])b_enc = np.array([-0.2, 0.0, -0.1, -0.4, -0.6])w_dec = np.array([[1.0, 0.0, 0.0, 1.0, 0.0], [0.0, 1.0, 0.0, 1.0, -1.0], [0.0, 0.0, 1.0, 0.0, 1.0]])
f = encode_topk(x, w_enc, b_enc, k=2)x_hat = decode(f, w_dec)err: float = float(((x - x_hat) ** 2).sum())loss: float = err + 0.05 * float(np.abs(f).sum())
print(f.tolist())      # -> [0.6, 0.0, 0.4, 0.0, 0.0]print(x_hat.tolist())  # -> [0.6, 0.0, 0.4]print(round(err, 2))   # -> 0.14print(round(loss, 2))  # -> 0.19

The prints match the hand run: two survivors, rebuild [0.6,0,0.4][0.6, 0, 0.4], error 0.140.14, loss 0.190.19 at λ=0.05\lambda = 0.05.

Watch Out For

Most of the dictionary goes dead

You train, rebuild error looks fine, then you count firing rates and 8 in 10 features never fire. The cause is usually a high sparsity price or dead initialization: λ\lambda pushed small features to zero early, their decoder columns stopped getting gradients, and they never recovered.

You will see it as a flat histogram with a tall bar at zero firing plus a few overused features. Fix it by lowering λ\lambda or raising kk first, keeping decoder columns at unit length, and re seeding columns that fired on fewer than about one in a million inputs. Track firing rate per feature from the start, not just mean loss.

You trust the auto label

A feature fires on your ten dog sentences, the labeling model names it dogs, then you zero it and the model still completes the dog sentence. Or you clamp it high and get Eiffel Tower trivia. The label described your sample, not the feature. Common culprits are feature splitting, where dogs live across two lookalikes, and shrinkage, where L1 pulled strengths down so your clamp was too weak.

Treat every label as a guess. Test it on held out sentences, then confirm with activation patching or a small activation steering sweep before you claim meaning. If the effect does not move with the knob, rename or drop the feature.

The Quick Version

  • A sparse autoencoder spreads one dense hidden state across a wide dictionary so only a few features fire per input.
  • The loss is rebuild error plus a sparsity price, or a hard TopK rule that keeps exactly kk features alive.
  • In the tiny run here, [0.8,−0.3,0.5][0.8, -0.3, 0.5] keeps [0.6,0,0.4][0.6, 0, 0.4] alive and rebuilds with squared error 0.140.14.
  • Tighter sparsity buys cleaner names at the cost of worse rebuilds, wider dictionaries buy room at the cost of dead features and compute.
  • Labels are guesses until a patch or steer moves behavior, so test before you trust them.