VINS Visual Inertial Fusion System

Cameras see shape but not scale; an IMU feels motion but drifts fast. VINS fuses both so fast shakes stay smooth and scale stays metric.

Fast IMU steps carry the pose between slower camera frames in one fused window

Why Does This Exist?

Pure vision fails during motion blur and texture loss, and monocular vision never learns meters. An IMU measures acceleration and rotation at hundreds of hertz through blur, but integrates into drift within seconds. VINS exists to fuse the two tightly: vision corrects inertial drift while the IMU carries tracking through visual dropouts and fixes absolute scale.

This page covers the fused estimator. Vision-only motion is visual odometry; loop repair on top is loop closure. Calibration of each camera lives in camera calibration and parameters.

Think of It Like This

Walking with eyes plus an inner ear

Your eyes read the hallway while your inner ear feels each step and turn. Close your eyes for a second and the inner ear keeps the count; shake your head and the eyes re-anchor the drift. Neither sense walks well alone.

Frames are eyes and the IMU is the inner ear. The analogy stops at bias: inner ears do not slowly offset while MEMS gyros do, which is why bias estimation runs nonstop.

How It Actually Works

IMU preintegration compresses hundreds of inertial readings between two frames into one relative motion factor with covariance, avoiding reintegration when the linearization point moves. The sliding-window estimator minimizes visual reprojection plus inertial plus marginalization priors together over poses, velocities, biases, and inverse depths. Initialization aligns vision-only structure with IMU gravity and scale through a short excited motion, solving gyroscope bias, velocity, gravity direction, and scale before the filter trusts meters.

Worked numbers

With $200$ Hz IMU and $20$ Hz images, ten inertial steps sit between frames. A $0.02$ rad per s gyro bias integrates to $0.002$ rad over $0.1$ s, corrected by about $40$ tracked corners at $0.5$ px noise. After five seconds of shaking motion, scale error falls from $12$ percent uninitialized to under $1$ percent fused.

Code

1def preintegrate_dt(count: int, imu_hz: float) -> float:2    return count / imu_hz  # time compressed into one factor3
4
5print(preintegrate_dt(10, 200.0))  # -> 0.05

Watch Out For

Initializing without enough motion

Sitting still leaves scale, gravity, and biases unobservable, so the initializer returns confident fiction. The symptom is a fused path with correct shape at half size. Excite all axes briefly before trusting metric output, and gate initialization on acceleration variance.

The Quick Version

VINS fuses camera reprojection with IMU preintegration in one window.
Preintegration compresses fast inertial readings into one motion factor.
Joint optimization estimates poses, velocities, biases, and depths together.
Initialization from excited motion fixes scale and gravity direction.
The IMU bridges blur and texture loss; vision bounds inertial drift.
Static starts leave scale unobservable and must be rejected.