PointNet and PointNet++ on Raw Points

Points arrive shuffled with no grid. PointNet reads each point alone then max-pools so order never matters; PointNet++ adds local neighborhoods.

Per-point MLPs max-pool into one global vector while hierarchy adds local groups

Why Does This Exist?

Point clouds are sets: shuffling $1024$ points must not change the answer, which rules out plain sequence nets. PointNet exists as the order-free baseline that point cloud processing introduces: shared MLPs plus symmetric pooling. PointNet++ fixes its blindness to local curves by grouping neighborhoods hierarchically. The architecture-deep version lives in core ML PointNet; this page is the vision use and the merged Plus upgrade.

Graph and attention successors are DGCNN and Point Transformer.

Think of It Like This

Judging Lego builds from sorted piles

Dump two identical Lego builds on tables in different orders and judge by counting brick types, not sequence. Then zoom into wheels and windows locally before judging the whole castle.

Max-pooling is the counting and set abstraction is the zoom. The analogy stops at rotation: brick counts ignore pose while points need alignment, handled by a small spatial transformer.

How It Actually Works

PointNet maps each point $p_i$ through shared $h(p_i)$ into $1024$ dims, then $g = \max_i h(p_i)$ gives permutation invariance since max ignores order. A mini spatial transformer predicts a $3 \times 3$ alignment from the cloud itself. PointNet++ samples farthest points, balls $r$ -neighborhoods around each, runs mini-PointNets locally, and repeats: set abstraction layers shrink $1024$ to $256$ to $64$ centroids with growing radii $0.1$ , $0.2$ , $0.4$ m, capturing curves at multiple scales.

Worked numbers

Four points $(0,0,0)$ , $(1,0,0)$ , $(0,1,0)$ , $(0,0,1)$ through $h(p) = [x + y, z]$ give $[0,0]$ , $[1,0]$ , $[1,0]$ , $[0,1]$ ; max-pool gives $[1,1]$ regardless of input order. Shuffling the list returns the same $[1,1]$ . On ModelNet40 with $1024$ points, reference accuracy moves from about $89$ percent PointNet to $91$ percent PointNet++.

Code

1def pointnet_pool(feats: list[list[float]]) -> list[float]:2    dims = len(feats[0])3    return [max(f[d] for f in feats) for d in range(dims)]  # symmetric: order-free4
5
6print(pointnet_pool([[0, 0], [1, 0], [1, 0], [0, 1]]))  # -> [1, 1]

Watch Out For

Expecting fine local detail from vanilla PointNet

Global pooling keeps one max per channel, so small handles and thin legs vanish. The symptom is great classification with blobby segmentations. Use PointNet++ grouping or a local operator such as KPConv when parts matter.

The Quick Version

PointNet reads raw points with shared MLPs plus max-pooling for order freedom.
Shuffled inputs give identical global vectors by construction.
A learned $3 \times 3$ transform adds rough rotation robustness.
PointNet++ groups neighborhoods hierarchically for local geometry.
Reference ModelNet40 rises from $89$ to $91$ percent with hierarchy.
Use Plus or local operators when parts and curves matter.