Change of Basis

The same arrow gets different coordinates under different rulers; the change-of-basis matrix translates between the rulers.

$The arrow to 3 1 stays put while two grids rotate beneath it: the standard grid reads it as 3 right 1 up, and the tilted basis of 1 1 and 1 minus 1 reads it as 2 of the first plus 1 of the second$

The arrow to 3 1 stays put while two grids rotate beneath it: the standard grid reads it as 3 right 1 up, and the tilted basis of 1 1 and 1 minus 1 reads it as 2 of the first plus 1 of the second

Why Does This Exist?

PCA, Fourier features, and every embedding projection change the rulers, not the data. A point cloud stays where it is while you re-express every point in a basis that compresses better or separates classes. If coordinates feel glued to the standard axes, those methods read as magic instead of bookkeeping.

Vectors, matrices, and tensors defines vectors and bases. Linear transformations covers maps that move arrows. This page covers the subtler sibling: the arrow stays fixed while the coordinate system changes.

The one idea you need first

A basis is a set of arrows you measure everything against, and coordinates are the recipe: how much of each basis arrow to add to reach your target. Standard coordinates $[3, 1]$ mean $3$ of $[1,0]$ plus $1$ of $[0,1]$ . Change the basis arrows and the same target needs a different recipe.

Think of It Like This

Describing a walk in two languages

A tourist asks how to reach the fountain. A local says "three blocks east, one north." A sailor says "two leagues along the river heading, one across it." Same fountain, same walk, different reference directions, different numbers. Neither is wrong, and a phrasebook converts between them.

The change-of-basis matrix is that phrasebook. It never moves the fountain; it converts one description into the other. Forgetting which direction you translated is how tourists end up in the harbour, and how ML code mixes up projection directions.

How It Actually Works

Given a new basis with vectors $b_1, b_2$ as columns of a matrix $P = [b_1 \mid b_2]$ , coordinates convert like this:

New-basis coordinates $c$ become standard coordinates $v$ via $v = Pc$ (mix the basis arrows by the recipe).
Standard coordinates $v$ become new-basis coordinates via $c = P^{-1} v$ (unmix).

In plain words: $P$ translates new descriptions into standard ones, and its inverse translates back. The arrow never moves; only its description changes.

Worked example

Tilted basis: $b_1 = [1, 1]$ , $b_2 = [1, -1]$ . Target arrow: $v = [3, 1]$ in standard coordinates.

Find the recipe $c_1 b_1 + c_2 b_2 = v$ :

\begin{cases} c_1 + c_2 = 3 \ c_1 - c_2 = 1 \end{cases}

Adding: $2c_1 = 4$ , so $c_1 = 2$ . Then $c_2 = 1$ . Check: $2[1,1] + 1[1,-1] = [2+1, 2-1] = [3, 1]$ . The same arrow reads $[3, 1]$ on standard rulers and $[2, 1]$ on tilted ones.

Matrix form: $P = \begin{bmatrix} 1 & 1 \ 1 & -1 \end{bmatrix}$ , and indeed $P \begin{bmatrix} 2 \ 1 \end{bmatrix} = \begin{bmatrix} 3 \ 1 \end{bmatrix}$ .

Watch Out For

Translating the phrasebook backwards

$P$ and $P^{-1}$ go opposite directions, and swapping them is the most common bug: coordinates that should shrink blow up instead. The symptom is a projection that lands nowhere near the data, often with suspiciously large values. The fix is the round-trip test: convert there and back and confirm you recover the input exactly, $P^{-1}(Pv) = v$ , on a hand-checkable vector before trusting the pipeline.

Reading new coordinates as standard ones

After computing $c = [2, 1]$ , plotting it on standard axes points at the wrong place entirely. The numbers only mean something with their basis attached. The symptom is a visualization where clusters smear or rotate for no reason. Label every coordinate with its basis, the way units label a measurement, and convert back to standard coordinates before plotting on standard axes.

The Quick Version

A basis is the set of rulers; coordinates are the recipe against those rulers.
The same arrow has different coordinates in different bases.
$P$ (basis vectors as columns) converts new coordinates to standard: $v = Pc$ .
$P^{-1}$ converts standard coordinates to new ones: $c = P^{-1}v$ .
$[3, 1]$ standard reads $[2, 1]$ in the tilted basis of $[1,1]$ , $[1,-1]$ .
Always round-trip a test vector, and never plot coordinates on foreign axes.