Correlation vs Causation

Why Does This Exist?

Machine learning models are fundamentally correlation engines. If you train a model to predict ice cream sales, and give it access to data on shark attacks, it will confidently learn that shark attacks are a strong positive predictor of ice cream sales. In a purely predictive setting—where you just want to know how much ice cream to stock—this is fine. The model is right: days with more shark attacks are indeed days with higher ice cream sales.

The problem starts when you want to act on the model's findings. If a business decides to reduce shark attacks (by banning swimming, for instance) in order to increase ice cream sales, the strategy will fail. The relationship was spurious, driven entirely by a hidden third factor: summer weather. People buy more ice cream when it's hot, and they swim more (leading to more shark attacks) when it's hot.

Causal inference exists because most business, medical, and policy decisions are interventions. We don't just want to predict the future; we want to change it. To do that, we have to isolate which variables actually drive the outcome, rather than just which variables happen to move alongside it in observational data.

Think of It Like This

Imagine you are looking at the dashboard of a car. You notice that every time the speedometer needle moves to the right, the car goes faster.

This is the difference between seeing and doing. In probability terms, seeing is $P(Y | X)$—the probability of $Y$ given that we observe $X$. Doing is $P(Y | do(X))$—the probability of $Y$ given that we physically force $X$ to happen. If $X$ is the speedometer needle and $Y$ is the speed, $P(Y | X)$ is high, but $P(Y | do(X))$ is exactly zero.

How It Actually Works

The shift from correlation to causation is a shift from purely statistical models to structural models. In standard statistics, variables are just columns of numbers that co-vary. In causal inference, variables belong to a directed graph (a causal graph) where arrows represent the flow of physical or logical influence.

The Ladder of Causation

Judea Pearl formalized this distinction into a three-level hierarchy:

1. Association (Seeing): What does a symptom tell me about a disease? This is standard machine learning. $P(y|x)$.
2. Intervention (Doing): What if I take an aspirin? This requires us to understand how the system reacts to an external change. $P(y|do(x))$.
3. Counterfactuals (Imagining): Was it the aspirin that cured my headache, or would it have gone away anyway? This requires us to imagine alternate histories for a specific individual. $P(y_x|x', y')$.

Spurious Correlations and Confounding

The most common reason correlation fails to imply causation is confounding. A confounder is a variable $Z$ that causes both $X$ and $Y$.

If $Z \rightarrow X$ and $Z \rightarrow Y$, then $X$ and $Y$ will be correlated in the data even if there is no arrow between them. If you train a regression model to predict $Y$ using $X$, it will assign a non-zero weight to $X$. The model is biased because it cannot distinguish between the direct effect of $X$ on $Y$ and the spurious path that flows backwards from $X$ up to $Z$ and then down to $Y$.

To extract the true causal effect of $X$ on $Y$, we must block this spurious path. This is usually done by "controlling for" $Z$ in our model (e.g., adding it as a feature in a regression, or matching samples that have identical values of $Z$).

Show Me the Code

Here is how a spurious correlation appears in a regression model, and how adding the confounder reveals the truth.

1import numpy as np2import statsmodels.api as sm3
4# Set random seed for reproducibility5np.random.seed(42)6
7# Generate a confounding variable: Temperature8temperature = np.random.normal(25, 5, 1000)9
10# Shark attacks are caused by temperature, plus noise11shark_attacks = 0.5 * temperature + np.random.normal(0, 2, 1000)12
13# Ice cream sales are caused by temperature, plus noise14# Notice: Ice cream sales are NOT caused by shark attacks!15ice_cream_sales = 2.0 * temperature + np.random.normal(0, 3, 1000)16
17# 1. Naive Model: Predict ice cream sales using ONLY shark attacks18X_naive = sm.add_constant(shark_attacks)19model_naive = sm.OLS(ice_cream_sales, X_naive).fit()20# -> The coefficient for shark attacks is ~4.0, highly significant.21# -> The model strongly believes shark attacks drive sales.22
23# 2. Causal Model: Predict ice cream sales controlling for temperature24X_causal = sm.add_constant(np.column_stack((shark_attacks, temperature)))25model_causal = sm.OLS(ice_cream_sales, X_causal).fit()26# -> The coefficient for shark attacks is ~0.0.27# -> The coefficient for temperature is ~2.0.28# -> Controlling for the confounder eliminates the spurious correlation.

Watch Out For

The Quick Version

• Correlation means two variables move together. Causation means manipulating one variable mechanically changes the other.
• Machine learning naturally optimizes for correlation, which is perfect for prediction but disastrous for intervention and decision-making.
• Spurious correlations are often driven by a confounder—a hidden third variable that causes both observed variables.
• To estimate a causal effect from observational data, you must understand the data-generating process and control for the right variables (while deliberately ignoring others).

What to Read Next

• potential-outcomes — The mathematical framework for defining causal effects via counterfactuals.
• causal-graphs — How to draw and use directed acyclic graphs (DAGs) to identify confounders.
• confounding-and-colliders — The structural patterns that create spurious correlations, and when not to control for a variable.