3D Representations Compared for Vision

Every 3D method picks one shape for geometry, and that shape decides memory, compute, and what details survive.

Why Does This Exist?

A photo is a grid, but 3D has no single grid. Point cloud processing handles points, meshes serve graphics, voxels serve convolutions, and fields serve learning. Beginners pick the representation their library defaults to and then fight memory or missing detail. This umbrella page compares the family so each component page can teach its mechanism.

Children in this wave: voxels, meshes, point clouds, signed distance functions, occupancy fields, implicit surfaces, depth maps, and multi-plane images.

Think of It Like This

Maps of the same city in different drawers

A city can be a Lego brick model, a street-centerline map, a dot for every resident, or a function that answers distance to the nearest wall. Each answers some questions fast and others badly. You would not navigate alleys with Lego bricks.

Representations are map drawers. Memory, compute, and fidelity are the labels on each drawer. The analogy stops at continuity: only fields answer at any real coordinate.

How It Actually Works

Write the trade as three numbers per shape. Memory $M$ is bytes stored. Compute $C$ is cost per query or convolution. Fidelity $F$ is smallest detail kept.

Worked example

Take a $10$ m room at $0.05$ m detail. Voxels need $(10/0.05)^3 = 8{,}000{,}000$ cells; at 1 byte that is $8$ MB mostly empty. Points need $100{,}000 \times 3 \times 4 = 1.2$ MB and keep exact positions but no surface. A mesh of $50{,}000$ triangles needs $50{,}000 \times 3 \times 3 \times 4 = 1.8$ MB plus connectivity and renders fast. A field MLP of $256 \times 256$ weights needs about $0.26$ MB and answers any point but costs a network pass per query. Pick voxels for 3D CNNs, points for LiDAR, meshes for rendering, fields for reconstruction.

Watch Out For

Converting representations to dodge the trade

Voxelize points, mesh the voxels, then render the mesh, and each step quantizes $0.05$ m further. The symptom is detail that no single step explains losing. Stay in one representation end to end where possible.

Judging fidelity by parameter count

A $0.26$ MB field can beat an $8$ MB voxel grid on smooth shapes because continuity matters more than bytes. Compare rendered error at the target detail, not file size.

The Quick Version

Voxels suit convolutions but explode cubically with finer detail.
Points suit sensors but carry no surface or inside-outside answer.
Meshes suit rendering but hate topology change and learning.
Fields suit learning and any-resolution queries but cost compute per sample.
Depth maps and multi-plane images suit cameras but store one viewpoint only.