Instrumental Variables (IV)

Why Does This Exist?

In observational data, you are plagued by unmeasured confounders (e.g., user motivation, hidden health conditions) that ruin your ability to calculate a true causal effect.

• You can't use Propensity Score Matching, because you don't have the data for the confounder.
• You can't use Difference-in-Differences, because the confounder changes over time.
• You can't run an A/B test, because it's unethical or impossible.

Instrumental Variables (IV) is the ultimate fallback technique. It allows you to calculate a causal effect even when massive, unmeasured confounders are present. It does this by finding a "cheat code" variable—an Instrument—that mimics a randomized experiment.

Think of It Like This

How It Actually Works

In causal graph terms, you have an unmeasured confounder $Z$ creating a backdoor path between Treatment $T$ and Outcome $Y$: $T \leftarrow Z \rightarrow Y$.

An Instrument $I$ is a variable that satisfies three strict conditions:

1. Relevance: The instrument actually causes (or predicts) the treatment ($I \rightarrow T$).
2. Exclusion Restriction: The instrument only affects the outcome by going through the treatment. There is no direct path $I \rightarrow Y$.
3. Unconfoundedness: The instrument shares no common causes with the outcome.

If you have a valid instrument, you can use Two-Stage Least Squares (2SLS) to estimate the causal effect.

Two-Stage Least Squares (2SLS)

• Stage 1: Regress the Treatment on the Instrument. This throws away all the variation in the Treatment caused by the unmeasured confounder, keeping only the "clean", randomized variation caused by the instrument.
• Stage 2: Regress the Outcome on the predicted Treatment from Stage 1. This gives you the causal effect.

IV in A/B Testing (Non-Compliance)

IV is heavily used in tech for A/B tests with Non-Compliance. If you randomly assign 1,000 users to receive a new ML feature (the Treatment Group), but only 400 of them actually click the button to turn it on, your A/B test is broken. The users who clicked it are systematically different from the ones who didn't. Here, the random assignment (the Coin Flip) is the Instrument. It doesn't directly improve the user's experience, but it nudges them to use the feature. 2SLS calculates the causal effect for the "Compliers"—the people who used the feature because they were in the treatment group.

Show Me the Code

Here is a simplified example of Two-Stage Least Squares to solve non-compliance in an A/B test.

1import numpy as np2import pandas as pd3import statsmodels.api as sm4
5np.random.seed(42)6N = 100007
8# Unmeasured Confounder (Motivation)9motivation = np.random.normal(0, 1, N)10
11# 1. The Instrument (Random Assignment to Treatment Group)12# 50/50 chance, completely independent of motivation13instrument_assigned = np.random.binomial(1, 0.5, N)14
15# 2. The Treatment (Did they actually use the new ML feature?)16# Highly motivated people use it anyway. Assigned people get a +30% nudge.17prob_use = 0.2 + 0.3 * instrument_assigned + 0.2 * motivation18prob_use = np.clip(prob_use, 0, 1)19treatment_used = np.random.binomial(1, prob_use)20
21# 3. The Outcome (Revenue)22# True causal effect of the feature is +$50. Motivation also drives revenue.23outcome = 100 + 50 * treatment_used + 20 * motivation + np.random.normal(0, 5, N)24df = pd.DataFrame({'Z_assigned': instrument_assigned, 'X_used': treatment_used, 'Y_revenue': outcome})25
26# --- Naive Regression (Wrong) ---27X_naive = sm.add_constant(df['X_used'])28model_naive = sm.OLS(df['Y_revenue'], X_naive).fit()29print(f"Naive Effect: +${model_naive.params['X_used']:.2f}") 30# -> ~$63 (Overestimates because of motivation)31
32# --- Two-Stage Least Squares (2SLS) ---33# Stage 1: Predict Treatment using the Instrument34X_stage1 = sm.add_constant(df['Z_assigned'])35model_stage1 = sm.OLS(df['X_used'], X_stage1).fit()36df['X_predicted'] = model_stage1.predict(X_stage1)37
38# Stage 2: Predict Outcome using the PREDICTED Treatment39X_stage2 = sm.add_constant(df['X_predicted'])40model_stage2 = sm.OLS(df['Y_revenue'], X_stage2).fit()41print(f"2SLS Causal Effect: +${model_stage2.params['X_predicted']:.2f}")42# -> ~$50.08 (Perfectly recovers the true causal effect!)

Watch Out For

The Quick Version

• Instrumental Variables solve the problem of unmeasured confounding when you cannot run a clean A/B test.
• An Instrument is a variable that is randomly distributed, affects the treatment, but does not affect the outcome directly.
• The standard algorithm for calculating the effect is Two-Stage Least Squares (2SLS).
• In tech, IV is primarily used to analyze A/B tests with Non-Compliance, where users don't actually do what their random assignment told them to do.

What to Read Next

• randomized-experiments — Review why clean random assignment is so powerful.
• double-machine-learning — How to use advanced ML models (like Random Forests) inside the two stages of causal estimation.
• difference-in-differences — Another quasi-experimental method that relies on time instead of an instrument.