ARIMA
ARIMA forecasts tomorrow by repeating a slice of recent movement and correcting for recent mistakes, after flattening any trend first.
Why Does This Exist?
Yesterday beats the average. When you're forecasting daily signups, last week's mean ignores whether the line is climbing, while yesterday's count already carries that climb. So momentum wins. But a momentum-only rule breaks in two places. First, a steady climb keeps moving the target, so any formula fitted on raw levels chases it forever. Second, every forecast misses a little, and those misses carry signal: undershoot three days running and tomorrow likely undershoots too, unless you correct for it.
ARIMA bundles the three repairs into one small model. Flatten the climb first (the I step, differencing). Repeat a slice of recent movement (the AR step). Then add back a slice of recent misses (the MA step). Box and Jenkins published this combination in 1970, and it stayed the default baseline for decades: fast, open to inspection, and hard to beat on short series with no outside drivers.
This page teaches the mechanics on one ARIMA(, , ) you can check by hand. The flattening step gets its full treatment in stationarity. The plots that choose the two orders live in autocorrelation. The simpler rival that skips the error correction is exponential smoothing.
Basics first, in three sentences. A series is stationary when its mean and spread stay put over time. Most real series are not: they trend, so classical formulas misfire. Differencing fixes the common case by modelling day-to-day changes instead of raw levels, which is exactly the I in ARIMA.
Think of It Like This
Steering a ship into harbor
You steer a large ship toward a dock. The ship has momentum: three seconds of drifting left means the next second likely continues left unless you act. That's the AR half. But your hands are imperfect, and the last three corrections each pushed slightly too far. You have logged those misses, so you subtract a slice of them from the next turn instead of repeating them. That's the MA half. Before any of this, you read the water: with a steady current you hold one baseline rudder angle and steer only the changes. Flattening the current first is the I half.
Where it stops: a ship answers the wheel the same way every time, while a market can jump on news no logbook contains. And the harbor current stands still while you dock, but real series change regime mid-stream. When the water itself shifts, no slice of past motion steers right.
How It Actually Works
ARIMA takes three orders, written ARIMA(, , ). Here counts how many past values the AR half looks at, counts how many rounds of differencing flatten the series, and counts how many past misses the MA half corrects for. Small orders win in practice: this page runs ARIMA(, , ), one lag, one differencing round, one past miss.
Flatten first with differencing
Differencing replaces each level with its change since the previous step. Write the level on day as , read "y sub t", and the change as , read "w sub t". Take signup counts in hundreds: , , , . The raw levels climb, but the changes are , , , a flat series hovering near its mean of about . That flatness is the point. AR and MA math assumes the statistical weather stays put, so you model the changes and add the last level back at the end.
The AR pull and the MA fix
On the flattened series, the forecast for the next change, written , has three parts. A drift carries the baseline step. An AR pull , with read "phi", repeats a fraction of the latest change. An MA fix , with read "theta", adds back a fraction of the latest miss, where the miss is actual minus forecast, and a positive miss means the last forecast undershot:
With , and , the pull repeats three-fifths of the latest gain, and the fix returns three-tenths of the latest miss. The plots that set and from data are the ACF and PACF in autocorrelation: a sharp PACF cutoff suggests , a sharp ACF cutoff suggests .
Worked example
The latest gain is and the latest miss is , so yesterday's forecast undershot by . Plug in:
The AR pull contributes , the MA fix contributes , drift adds . Add the step back onto the last level of :
So the forecast is hundred signups, about 1,652 people. Without the MA fix the step would be and the forecast : the correction for yesterday's undershoot lifts the call by . That lift is the whole MA idea in one number.
A note on scope. Plain ARIMA handles repeating seasons poorly: a spike every December wants a model that reaches back twelve steps, which is seasonal ARIMA rather than a bigger . Compare against the simpler rival in exponential smoothing before adding machinery.
Code
The point is the check: difference the levels, apply the three-part rule, and confirm the hand math.
ys: list[float] = [10.0, 12.0, 13.0, 15.0]ws: list[float] = [b - a for a, b in zip(ys, ys[1:])]c: float = 0.2phi: float = 0.6theta: float = 0.3e_last: float = 0.4
w_next: float = c + phi * ws[-1] + theta * e_lasty_next: float = ys[-1] + w_next
print(f"gains {ws}") # -> gains [2.0, 1.0, 2.0]print(f"next step {w_next:.2f}") # -> next step 1.52print(f"forecast {y_next:.2f}") # -> forecast 16.52The printout matches the hand computation: gains , , , a next step of , a forecast level of .
Watch Out For
Fitting the trend instead of the changes
With on climbing data, the AR weights do the trend's job and the forecast lags every turn: each prediction sits below the next reading on the way up, and the misses lean one-sided instead of centering on zero.
Symptom: one-sided misses that grow with the slope, plus a fitted mean that belongs to no actual week. Fix: difference once, confirm the changes hover near a steady mean (ours sit at , , ), and only then fit and . The full flattening checklist is in stationarity.
Ordering up until the past is memorized
Raising and always sharpens the in-sample fit, because each new weight soaks up one more wiggle of history. Then next week's error gets worse: the model learned noise with names on it.
Symptom: tiny training residuals while holdout forecasts trail a plain baseline. Fix: keep orders tiny, stay at ARIMA(, , ) until evidence says otherwise, and grade every candidate on a held-out tail with a rolling origin, as backtesting describes.
The Quick Version
- ARIMA forecasts tomorrow from recent movement plus recent misses, after differencing flattens the trend.
- ARIMA(, , ) sets three small orders: past values used, differencing rounds applied, past misses corrected.
- One ARIMA(, , ) turns gains of , , into a next step of and a forecast level of .
- The MA fix exists because misses repeat: yesterday's undershoot of lifts the call by .
- A trend fitted without differencing lags every turn, and fat orders memorize noise, so difference first and grade on a held-out tail.