ARIMA

Why Does This Exist?

In the early days of statistics, there were two competing theories for how to forecast a time series.

The first camp believed in AutoRegression (AR): "To predict tomorrow's stock price, just look at the price from the last 3 days and draw a line." The second camp believed in Moving Averages (MA): "To predict tomorrow, look at how wrong our predictions were over the last 3 days (the errors), and correct for them."

In 1970, statisticians Box and Jenkins proved that combining both approaches created a vastly superior model. They added an "Integrated" (I) step to enforce Stationarity, and the ARIMA model was born. It remained the undisputed king of forecasting for 40 years until gradient boosting and deep learning arrived.

Think of It Like This

How It Actually Works

ARIMA is defined by three parameters: $(p, d, q)$.

1. The I-term ($d$): Integration (Differencing)

ARIMA requires strict stationarity. If your data is trending upwards, ARIMA will fail. The $d$ parameter tells the model how many times to apply differencing (subtracting $t-1$ from $t$) to force the data to become flat. Usually, $d=1$.

2. The AR-term ($p$): AutoRegressive

This tells the model how many past days (lags) to look at. If $p=2$, the formula to predict tomorrow includes a weighted sum of yesterday and the day before. We determine the value of $p$ by looking at the PACF plot. $Y_t = c + \phi_1 Y_{t-1} + \phi_2 Y_{t-2}$

3. The MA-term ($q$): Moving Average

This is the hardest part to understand. It does not mean the classical moving average indicator you see on stock charts. It refers to the past forecast errors ($\epsilon$). If $q=1$, the model calculates how wrong it was yesterday, and factors that error into today's prediction to auto-correct itself. We determine $q$ by looking at the ACF plot. $Y_t = c + \theta_1 \epsilon_{t-1}$

The full ARIMA model simply adds these equations together.

SARIMA (Seasonal ARIMA)

Standard ARIMA cannot handle seasonality (e.g., spikes that happen every 12 months). To fix this, statisticians created SARIMA, which adds three more parameters $(P, D, Q)_s$. It essentially runs a second, parallel ARIMA model that jumps back $s$ steps at a time (e.g., looking at exactly this month last year).

Show Me the Code

In Python, statsmodels is the standard library for ARIMA. However, determining the exact $(p,d,q)$ parameters manually via ACF/PACF plots is tedious. Modern practitioners use the pmdarima library, which provides an auto_arima function that tests hundreds of combinations and automatically selects the best one (similar to R's famous auto.arima).

1import pandas as pd2import pmdarima as pm3
4# A dataset of monthly airline passengers (highly seasonal)5# Since it is monthly, seasonal period m=126data = pd.read_csv('airline_passengers.csv', index_col='Month')7
8# Run Auto-ARIMA to automatically find the best (p,d,q) and (P,D,Q)9# It uses the AIC metric to score the models10model = pm.auto_arima(11    data['Passengers'], 12    seasonal=True, m=12,13    trace=True,          # Prints the models it tries14    error_action='ignore',  15    suppress_warnings=True16)17
18print(model.summary())19
20# Forecast the next 12 months21forecast, conf_int = model.predict(n_periods=12, return_conf_int=True)22print("\nForecast for next year:\n", forecast)

Watch Out For

The Quick Version

• ARIMA is a classical statistical model for forecasting time series.
• AR (p): Uses past values to predict the future (momentum).
• I (d): Differences the data to make it stationary (flat).
• MA (q): Uses past forecast errors to correct the future trajectory.
• Standard ARIMA cannot handle repeating cycles; you must use SARIMA to model seasonality.
• While it has been largely superseded by gradient boosting for complex business datasets, it remains an extremely fast and powerful baseline.

What to Read Next

• Exponential Smoothing
• Time Series Decomposition