Mask R-CNN

Mask R-CNN takes a two-stage detector and gives every detected box its own pixel silhouette, so overlapping objects of the same class stay separate.

Why Does This Exist?

Faster R-CNN draws boxes, and semantic segmentation paints class blobs that melt two touching people into one. Counting cells, placing grasp points, or measuring tumors needs both answers at once: which pixels, and which object each pixel belongs to. Mask R-CNN (He et al., 2017) took the detector the field already trusted and added one parallel branch: keep the class and box heads, and predict a small binary mask per region beside them. Same proposals, same features, one extra head, and instance masks fall out at nearly the speed of detection alone.

This page is the architecture: the three heads, RoIAlign, the three-part loss, and inference. The task overview lives in instance segmentation, and the overlap math it optimizes is in Dice loss. When 30-plus frames per second matter more than crisp edges, the one-stage family (YOLACT, SOLO) is the trade to make.

One idea first: a region of interest (RoI) is a single proposal box cropped from the shared feature map. Older pooling rounded its coordinates to integers, which shifted small masks by whole pixels. RoIAlign instead samples each crop with bilinear interpolation, so a 14 by 14 patch lands exactly on the object.

Think of It Like This

A portrait studio that frames, names, and cuts silhouettes

One sitting, three jobs. The sitter poses once, and that pose is the shared backbone features. The framer draws a tight rectangle around them (the box head). The receptionist writes the name on the folder (the class head). The cutter traces the outline on tracing paper and hands you the silhouette (the mask head). Nobody re-poses the sitter. All three work from the same sitting, which is why the whole thing trains in one backward pass.

Where it stops: the cutter works from a palm-sized tracing, a 14 by 14 crop, not the life-size sitter, so wispy edges come out chunky. And each silhouette is cut independently, so overlapping people still need non-maximum suppression to crown one winner per face first.

How It Actually Works

A backbone with a Feature Pyramid Network feeds a Region Proposal Network, which narrows thousands of anchors to about 300 proposal boxes. Every proposal is the mask head's whole world: anything the proposals miss gets no mask, however good the head is. Each survivor is cropped twice with RoIAlign, at two resolutions for two jobs: a 7 by 7 patch goes to the class head (a classifier over $C + 1$ classes, background included) and the box head (four coordinate offsets), while a 14 by 14 patch goes to the mask head.

The mask head is a small fully convolutional stack: four 3 by 3 convolutions with 256 channels, a 2 by 2 transposed convolution with stride 2 that doubles resolution, and a final 1 by 1 convolution. Its output is $K$ masks of size $m \times m$ with $m = 28$ , one channel per class. Each pixel passes through its own sigmoid $\sigma$ , so channels never compete. The mask loss is binary cross-entropy evaluated only on the ground-truth class channel $k^*$ :

\mathcal{L}_{\text{mask}} = -\frac{1}{m^2} \sum_{i=1}^{m} \sum_{j=1}^{m} \left[ y_{i,j} \log \sigma(s_{i,j}^{k^*}) + (1 - y_{i,j}) \log (1 - \sigma(s_{i,j}^{k^*})) \right]

Here $y_{i,j}$ is the true binary label at pixel $(i, j)$ and $s_{i,j}^{k^*}$ is the head's logit there. The full training loss simply adds the three heads, with $p$ the predicted class probability and smooth- $L_1$ on the box offsets:

\mathcal{L} = \mathcal{L}_{\text{cls}}(p) + \mathcal{L}_{\text{box}} + \mathcal{L}_{\text{mask}}

At inference the order flips: score every proposal, suppress duplicates, keep the survivors, then take each winner's $k^*$ -th mask, resize it to its box, and cut at probability 0.5.

Why the mask loss ignores every other class

The class head already decides dog versus cat, so the mask head only traces outlines. If all $K$ channels shared one softmax, a confident cat channel would strangle the dog outline at shared pixels. Independent sigmoids plus loss on $k^*$ alone decouple the jobs: tracing never argues about naming.

Worked loss example

Take one kept proposal that predicts person with $p = 0.7$ . Classification loss is $-\log(0.7) \approx 0.357$ . Say one box coordinate is off by $0.5$ : smooth- $L_1$ in its inner region is $0.5 \times 0.5^2 = 0.125$ . For the mask, use a 2 by 2 fixture with truth $[[1, 0], [1, 1]]$ and head logits $[[1.5, -0.5], [0.5, 2.0]]$ . Sigmoids give $[0.818, 0.378, 0.622, 0.881]$ , and binary cross-entropy per pixel gives $[0.201, 0.474, 0.474, 0.127]$ , averaging to about $0.319$ . The region total is $0.357 + 0.125 + 0.319 \approx 0.801$ , and one backward pass teaches all three heads from it.

Code

The mask term is ordinary binary cross-entropy after a sigmoid, which this snippet checks on the fixture above.

1import numpy as np2
3truth = np.array([[1, 0], [1, 1]])             # tiny mask fixture4logits = np.array([[1.5, -0.5], [0.5, 2.0]])   # one class channel of the head5
6prob = 1 / (1 + np.exp(-logits))               # sigmoid per pixel, no class fight7bce = -(truth * np.log(prob) + (1 - truth) * np.log(1 - prob))8print([round(float(v), 3) for v in prob.flat]) # -> [0.818, 0.378, 0.622, 0.881]9print(round(float(bce.mean()), 3))             # -> 0.319

Watch Out For

Training all K masks instead of the true one

Averaging the mask loss over every class channel punishes the head for 79 masks nobody asked for, and their gradients drown the one channel that matters. The symptom is masks that look alike across classes and training that crawls. Index the ground-truth channel $k^*$ before the loss, which is what the reference implementations gather.

Trusting 28 by 28 edges on thin structures

A 28 by 28 mask stretched over a large box turns wires, whiskers, and fingers into staircases, and cutting at 0.5 after the resize shifts the boundary a pixel either way. The symptom is strong box scores beside ragged masks on thin objects. When edges matter more than speed, step up to PointRend subdivision instead of a bigger mask head.

The Quick Version

Mask R-CNN is Faster R-CNN plus a parallel mask head, so class, box, and mask predict together.
RoIAlign crops land exactly: 7 by 7 feeds class and box, 14 by 14 feeds the mask head.
The head outputs one 28 by 28 mask per class with independent sigmoids, and the loss counts only the true class channel.
One proposal at person 0.70 with a small box slip and the worked 2 by 2 mask totals about 0.801.
Inference suppresses duplicate boxes first, then resizes the winning mask to its box and cuts at 0.5.
Thin structures come out staircased, so reach for PointRend when edges outrank speed.