Equivariance and Group Representations

Equivariant maps commute with symmetry transforms, so rotating the input rotates the output predictably instead of scrambling it.

Rotate then process equals process then rotate for an equivariant map

Why Does This Exist?

Images, molecules, and point clouds carry symmetries: a rotated cat is still a cat, and a rotated molecule has the same energy. Plain networks relearn each orientation from data, wasting capacity and samples. Equivariant architectures bake the symmetry in: transform the input and the features transform predictably.

Representation theory tells which linear maps qualify, turning architecture search into algebra. Group background: Lie groups.

The one idea you need first

A representation $\rho(g)$ assigns each group element an invertible matrix with $\rho(gh) = \rho(g)\rho(h)$ . Equivariance means $f(\rho_X(g)x) = \rho_Y(g)f(x)$ : act before or after, same result.

Think of It Like This

A compass rose printed on tracing paper

Sketch a compass rose on tracing paper over a map. Spin the paper 90 degrees and north points west on the table, but the rose still reads correctly relative to its own print.

An equivariant network is that tracing paper: input spins move features along with it in a predictable frame, never scrambling labels. Invariance is the special case where the output ignores the spin entirely.

Where the analogy stops: learned representations mix irreducible blocks in ways no paper sketch shows.

How It Actually Works

Linear equivariant maps between representation spaces form a structured subspace (solved by Schur's lemma and Clebsch-Gordan rules). Convolutions are translation-equivariant by weight sharing; steerable CNNs extend sharing to rotations via harmonic bases. Invariant pooling collapses orbits after equivariant features.

Worked example

2D rotation by $90^\circ$ , $R = [[0,-1],[1,0]]$ , map $f(x) = (x_1^2 + x_2^2, x_1x_2)$ at $x = (1, 2)$ :

$f(1,2) = (5, 2)$ .
Rotate input: $Rx = (-2, 1)$ . $f(-2,1) = (5, -2)$ .
An invariant head (first component) returns $5$ both ways: stable. The second component flips sign predictably under the induced representation (determinant-weighted). A non-equivariant MLP has no such guarantee and must learn both cases separately.

Watch Out For

Claiming equivariance after data augmentation

Augmentation teaches approximate invariance on the training distribution but breaks under unseen poses and offers no guarantee. Safety-critical geometry needs architecture, not augmentation hope.

Test on held-out transforms and prove layerwise commutation for real guarantees.

Breaking equivariance with position encodings

Absolute position codes and zero-padding anchor the frame and silently kill translation symmetry. Accuracy on centered data hides the damage until deployment shifts inputs.

Use relative encodings and circular or symmetric padding where symmetry matters.

The Quick Version

Equivariant maps commute with group actions on inputs and outputs.
Representations turn abstract symmetries into concrete matrices.
Weight sharing over orbits replaces repeated learning per pose.
Invariance is equivariance with a trivial output action.
Encodings and padding can silently break designed symmetry.