Instance Segmentation with Mask R-CNN

Separate Object Identities

Mask R-CNN returns an object category, a bounding box and a separate pixel mask for each detected instance. Two touching objects can share a semantic class while retaining identities A and B. The solid and dashed outlines identify those separate outputs; color is unnecessary to tell them apart.

This page follows the region-based route from candidate boxes to masks. For a dense encoder–decoder alternative, see U-Net segmentation.

Propose Candidate Regions

A region of interest (RoI) is a candidate rectangle. The drawing places two such candidates over one shared feature field; the curves represent object structure, not measured neural activations. A region proposal network supplies candidates. Shared backbone computation avoids processing the whole image independently for every box. Proposals are candidates, so later classification can reject them.

Preserve Fractional Boundaries

A boundary at feature coordinate 1.25 lies between grid locations. Rounding it to 1 shifts it by 0.25 feature units: twenty drawing units at the pictured scale of eighty per feature unit. The solid mint edge preserves the original coordinate; the dashed edge shows rounding. This one-boundary example isolates quantization rather than simulating the complete RoIPool operation.

Sample Between Grid Locations

Zoom into one feature cell, with values 0,2 across the top and 4,6 across the bottom. At local coordinate (x,y)=(0.25,0.5), the four bilinear weights are 0.375,0.125,0.375,0.125. The result is 0×0.375 + 2×0.125 + 4×0.375 + 6×0.125 = 2.5. Coordinates increase right and down. The spokes identify contributors; their widths do not encode weights.

The Mask R-CNN method uses this interpolation to avoid quantizing RoI boundaries, bins and sample locations.

Aggregate Samples Within Bins

Return to feature coordinates. The RoI runs from (0.5,0.5) to (2.5,2.5) and is divided into two-by-two bins. Every bin is one feature unit wide and high. With sampling ratio two, the first bin samples (0.75,0.75), (1.25,0.75), (0.75,1.25) and (1.25,1.25). Each dot receives bilinear interpolation, followed by aggregation within its bin. Sixteen samples produce four output bins, not sixteen output pixels. The dashed grid is the underlying feature lattice.

Branch From Aligned Features

The aligned region grid feeds class/box recognition and a spatial mask predictor. The lower route preserves a grid because a mask needs local layout; the upper output represents a category and refined rectangle. These are separate predictions from shared evidence. The two-by-two grids are schematic, not a claim about a trained model's output resolution.

Select The Class Mask

In the class-specific formulation, each RoI predicts one binary mask plane per foreground class. The class prediction selects plane A here. Each pixel uses a sigmoid independently; the planes do not compete through a per-pixel class softmax. Hatches show thresholded foreground in illustrative two-by-two planes. Object identity comes from the RoI, so two objects classified A still receive separate masks.

Train The Matched Plane

For a positive RoI, use the ground-truth class plane for mask loss, not whichever class the model currently prefers. A two-pixel crop has targets [1,0] and probabilities [0.8,0.3]. Average binary cross-entropy is −(ln(0.8)+ln(1−0.3))/2 ≈ 0.289909. Bars encode probabilities and ticks encode targets; the gaps are probability errors, not loss magnitudes. Classification, box regression and mask losses contribute separate training terms.

Place The Local Mask

The selected mask is expressed in local RoI coordinates. Resize it to the detected box and place it at that box's image position. The toy diagonal two-by-two mask becomes four-by-four by duplicating each source cell to a two-by-two block, making the coordinate mapping visible. This drawing uses nearest-neighbor replication only for inspection; practical probability-mask resizing uses interpolation before thresholding. It cannot create arbitrary boundary detail absent from the local prediction.

Where It Breaks

A gets a retained region and a mask. B is visible but receives no retained RoI; there is no B feature crop for the mask head to process. Better mask sampling alone cannot repair this missed proposal. Small objects, proposal filtering and coarse mask resolution can each limit output quality at different stages.

Region-based masks buy per-object separation at the cost of proposal handling and per-region prediction. If the task only needs a dense class map, a semantic segmentation model may fit better. Measure mask overlap separately from box overlap and AP.

The Quick Version

• Instance masks distinguish objects sharing one semantic class.
• Region proposals select candidates from shared image features.
• Fractional boundaries avoid rounding-induced spatial shifts.
• Bilinear interpolation blends four neighboring feature values.
• RoIAlign aggregates samples inside fixed output bins.
• Recognition and spatial masks branch from aligned region features.
• The predicted class selects an independent binary mask plane.
• Mask loss supervises the ground-truth class plane for positive regions.
• Local masks are resized into detected image boxes.
• Missing regions prevent downstream mask predictions.