Hyperbolic Embeddings

Why Does This Exist?

Flat embeddings fight hierarchies. A visual taxonomy — animal to dog to husky — branches multiplicatively, but Euclidean volume grows only polynomially with radius. Cramming an exponentially growing tree into flat space forces distant cousins together and stretches siblings apart; the distortion shows up as confused fine-grained classes. Adding dimensions postpones the problem without fixing its shape.

Hyperbolic space matches the shape. Its volume grows exponentially with radius, the same rate trees grow, so a hierarchy embeds with little distortion in two dimensions where flat space needs dozens. Vision taxonomies (species, products, scenes) are the natural tenants. The flat-space neighbor is metric learning; the curved-surface sibling is manifold learning.

The one idea you need first. The Poincaré ball pictures hyperbolic space as the inside of a disk where distances blow up toward the edge: walking near the boundary covers enormous ground per step. Place the hierarchy's root at the center and leaves near the rim. Siblings under one parent sit near each other along the rim, while different branches separate by vast boundary distances — exactly the tree metric, drawn to scale.

Think of It Like This

How It Actually Works

Embed each image or label as a point in the Poincaré ball. Distance follows the hyperbolic formula, which behaves Euclidean near the center and explodes toward the boundary. Training uses contrastive or entailment losses rewritten with that distance: pull same-branch items together, push different branches apart along the rim. Classification reads off the nearest prototype; hierarchy depth reads off the radius (deeper means further out).

The Poincaré distance between points $u$ and $v$ is:

$d(u, v) = \operatorname{arcosh}\left(1 + \frac{2\lVert u - v\rVert^2}{(1 - \lVert u\rVert^2)(1 - \lVert v\rVert^2)}\right)$

The denominator shrinks as either point nears the boundary ($\lVert \cdot \rVert \to 1$), inflating the distance — that inflation is the exponential room.

Worked example

Take $u = (0.5, 0)$ and $v = (-0.5, 0)$ in the unit disk. Squared norms: $\lVert u\rVert^2 = \lVert v\rVert^2 = 0.25$. Difference: $\lVert u - v\rVert^2 = 1.0$. Plug in:

$1 + \frac{2 \times 1.0}{(1 - 0.25)(1 - 0.25)} = 1 + \frac{2}{0.5625} = 1 + 3.5556 = 4.5556$

Then $\operatorname{arcosh}(4.5556) = \ln(4.5556 + \sqrt{4.5556^2 - 1})$. Since $4.5556^2 = 20.753$ and $\sqrt{19.753} \approx 4.4446$, this is $\ln(9.0002) \approx 2.197$.

Euclidean distance says $1.0$; hyperbolic distance says $2.2$. Two points halfway to opposite rims are already far apart — the boundary pushes everything away from everything, which is the room hierarchies rent.

Code

The point of this snippet is the distance law: a few lines reproduce the worked example and show the boundary inflation directly.

1import math2
3
4def poincare_distance(u: list, v: list) -> float:5    norm_u = sum(a * a for a in u)  # squared norms toward the boundary6    norm_v = sum(b * b for b in v)7    diff = sum((a - b) ** 2 for a, b in zip(u, v))8    arg = 1.0 + 2.0 * diff / ((1.0 - norm_u) * (1.0 - norm_v))9    return math.log(arg + math.sqrt(arg * arg - 1.0))  # arcosh via log10
11
12print(round(poincare_distance([0.5, 0.0], [-0.5, 0.0]), 3))  # -> 2.19713print(round(poincare_distance([0.1, 0.0], [-0.1, 0.0]), 3))  # -> 0.403

Near the center ($0.2$ apart Euclidean) the distance is $0.403$, close to flat. Near the rim the same Euclidean gap inflates to $2.197$: position, not just separation, sets the scale.

Watch Out For

The Quick Version

• Hyperbolic space grows exponentially with radius, matching how hierarchies branch.
• The Poincaré ball puts coarse concepts at the center and fine leaves near the rim.
• The worked fixture gives hyperbolic distance $2.197$ for a Euclidean gap of $1.0$ near the rim.
• Train with hyperbolic contrastive or entailment losses; read depth off the radius.
• Optimize on the manifold, not with naive Euclidean steps, or boundary gradients explode.
• Tangled multi-parent ontologies fit poorly; mostly-tree taxonomies gain most.

What to Read Next

• Manifold Learning
• Metric Learning for Embeddings