Dual Spaces

The dual space collects every continuous way to read a vector as a single number. Each readout is linear, each has a size, and together they see everything the primal space contains.

Vectors lie along the primal axis while one functional stands across it, pairing with each vector to produce a number

Why Does This Exist?

Optimization constraints, weak formulations of PDEs and attention scores all read vectors as scalars: a price times a quantity, a test function against a solution, a query against a key. The dual space organizes every such readout into a space of its own, with its own norm, so duality arguments (certificates, adjoints, biduals) run on rails.

This page defines the dual and computes one dual norm exactly. The finite-dimensional picture it generalizes is orthogonality and projections; rank language is in vector spaces and rank.

The one idea you need first

A linear functional $f$ maps vectors to scalars with $f(\alpha x + \beta y) = \alpha f(x) + \beta f(y)$ . Continuous means bounded: $|f(x)| \le C\|x\|$ . The dual $X^\ast$ keeps exactly the continuous ones.

Think of It Like This

Price tags on grocery shelves

Groceries are vectors (apples, bread, milk). A price tag is a functional: it reads any basket as one number, the total, linearly (twice the basket costs twice as much). The wall of all possible price tags is the dual space.

Some tags are wild (infinite on normal baskets): those are discontinuous and expelled. Where it stops: groceries live once, but tags can tag tags. That second wall, the bidual, sometimes matches the shop and sometimes dwarfs it (non-reflexive spaces).

How It Actually Works

The dual space $X^\ast$ holds all continuous linear functionals $f: X \to \mathbb{R}$ (or $\mathbb{C}$ ), with the dual norm:

\|f\|_\ast = \sup_{\|x\| \le 1} |f(x)|

In plain words: the biggest readout over the unit ball. $X^\ast$ is always Banach, even when $X$ is not. Hahn-Banach extends functionals from subspaces without growing their norm, so the dual always separates points: for $x \ne 0$ some unit functional reads $\|x\|$ off it. On Hilbert space, Riesz says every functional is inner product with one vector ( $f(x) = \langle w, x \rangle$ , $\|f\|_\ast = \|w\|$ ).

Worked example

On $\mathbb{R}^3$ with Euclidean norm, $f(x) = 2x_1 - x_2 + 0.5x_3$ . By Cauchy-Schwarz, $|f(x)| = |w \cdot x| \le \|w\|_2 \|x\|_2$ with $w = (2, -1, 0.5)$ , and $x = w/\|w\|$ attains it. So $\|f\|_\ast = \|w\|_2 = \sqrt{4 + 1 + 0.25} = \sqrt{5.25} \approx 2.291$ .

At $x = (1, 0, 2)$ : $f(x) = 2 + 0 + 1 = 3 \le 2.291 \times \sqrt{5} \approx 5.12$ . The cap holds with room to spare.

Code

The dual norm of a coordinate functional is the representing vector's length.

1import numpy as np2
3
4def dual_norm(w: np.ndarray) -> float:5    return float(np.linalg.norm(w))  # Riesz: functional norm equals vector norm6
7
8w = np.array([2.0, -1.0, 0.5])9print(round(dual_norm(w), 3))  # -> 2.29110print(float(w @ np.array([1.0, 0.0, 2.0])))  # -> 3.0

Norm $2.291$ , readout $3.0$ : pairing and cap exactly as computed by hand.

Watch Out For

Assuming the dual looks like the primal

Symptom: treating $(\ell^1)^\ast$ as $\ell^1$ (it is $\ell^\infty$ ) or $(L^1)^\ast$ as $L^1$ (it is $L^\infty$ ), then missing functionals. Fix: memorize the three shifts ( $1 \leftrightarrow \infty$ , $p \leftrightarrow q$ with $1/p + 1/q = 1$ ); only Hilbert and reflexive $L^p$ ( $1 < p < \infty$ ) mirror cleanly.

Using discontinuous functionals as if bounded

Symptom: pointwise evaluation on $L^2$ (Dirac delta) treated as a continuous readout; approximations blow up. Fix: deltas are not $L^2$ -continuous; move to the Sobolev dual or use mollified evaluation with explicit width.

The Quick Version

The dual $X^\ast$ collects continuous linear readouts with the sup-over-unit-ball norm.
$X^\ast$ is always Banach; Hahn-Banach guarantees it separates every point.
On Hilbert space each functional is inner product with one vector of equal norm.
$f(x) = 2x_1 - x_2 + 0.5x_3$ has dual norm $\sqrt{5.25} \approx 2.29$ .
$(\ell^1)^\ast = \ell^\infty$ : duals shift exponents, rarely mirroring the primal.