Rich Feature Hierarchies for Accurate Object Detection

The Problem

Before R-CNN, object detection on challenging datasets like PASCAL VOC was stagnating. The dominant approaches relied heavily on complex, hand-engineered feature ensembles (like SIFT and HOG) paired with sliding-window classifiers. While Convolutional Neural Networks (CNNs) had recently achieved breakthrough performance in image classification (via AlexNet), translating this success to object detection—where the model must locate and classify multiple objects in a single image—proved incredibly difficult.

The Idea

The paper proposed a straightforward, modular system: bridge the gap between image classification and object detection by running a CNN on pre-extracted "region proposals." Instead of sliding a classifier over every possible window in the image at every scale, the system first guesses where objects might be, and then uses a CNN to extract features from those specific regions.

How It Works

R-CNN (Regions with CNN features) operates in three distinct modules:

Region Proposals: The system uses an external algorithm, specifically Selective Search, to generate around 2,000 category-independent region proposals for the input image. These are rough bounding boxes that likely contain some object.

Feature Extraction (CNN): Because the region proposals are arbitrary shapes and sizes, the image data within each proposal is warped/resized to a fixed resolution (e.g., 227x227). A large Convolutional Neural Network (pre-trained on ImageNet and fine-tuned on the specific detection dataset) processes each warped region to extract a dense, 4096-dimensional feature vector.

Classification: The extracted feature vectors are fed into a set of class-specific linear Support Vector Machines (SVMs). The SVMs score the regions, and a greedy non-maximum suppression step is applied to reject overlapping boxes, leaving only the most confident detections.

Why It Mattered

R-CNN radically improved object detection metrics, increasing the mean average precision (mAP) on the PASCAL VOC benchmark by over 30% compared to previous best results. It proved conclusively that the hierarchical features learned by CNNs for image classification could be successfully transferred to object detection. It established deep learning as the default tool for computer vision localization tasks.

What Came After

While accurate, R-CNN was notoriously slow because it ran a full CNN forward pass independently for every single one of the 2,000 region proposals. This bottleneck immediately drove subsequent research to optimize the architecture, leading to Fast R-CNN (which shared computation across the image) and Faster R-CNN (which replaced the external Selective Search with a fully integrated Region Proposal Network), paving the way for modern real-time detectors.