Skip to content
AI360XpertBeta

Potential Outcomes

Every choice splits into two futures, treated and not, but you only ever see one side of the split.

Four shoppers hold effects 1, 0, 0 and 1 for a true average of 0.50, while one lucky split of who got treated reads 1.00
Four shoppers hold effects 1, 0, 0 and 1 for a true average of 0.50, while one lucky split of who got treated reads 1.00

Why Does This Exist?

A shopper gets a discount code and buys. Did the code cause the purchase, or would the same person have bought anyway? The receipt alone cannot answer it. You need the same shopper in two futures: one with the code, one without.

That comparison is the whole subject. Call TT the treatment, 1 for treated and 0 for control. Call Y(1)Y(1) the outcome if treated and Y(0)Y(0) the outcome if not. The individual effect is their gap. The trouble is that only one future ever shows.

This page covers that two future language and the average that rescues it. The arrow maps that name which variables to hold fixed live in causal graphs. The two junctions that fool raw comparisons live in confounding and colliders. The matching fix when you cannot randomize lives in propensity score methods. The randomization fix lives in ab testing for ml.

Think of It Like This

Two tomato plants from one seed packet

Plant two seeds from the same packet in two identical pots on one windowsill. Water only the left pot with a new feed. A month later the left plant stands taller. Did the feed help, or did that seed just start stronger?

You want the same seed in both pots at once: fed and unfed. Real life gives you two different seeds and one watering choice each. Averaging over many pairs of pots washes out the seed luck, which is why the framework moves from one plant to a whole sill of them.

Where it stops: plants in separate pots never share water, but shoppers share codes, reviews, and stock. When one unit changes another outcome, the two future picture needs extra care.

How It Actually Works

For unit ii, meaning one shopper or patient, define two potential outcomes. Yi(1)Y_i(1) means the outcome if unit ii is treated. Yi(0)Y_i(0) means the outcome if the same unit is not treated. The individual treatment effect, meaning the causal push for that unit, is:

ITEi=Yi(1)−Yi(0)\text{ITE}_i = Y_i(1) - Y_i(0)

Only one side shows. If TiT_i is the treatment actually received, the observed outcome YiY_i is:

Yi=TiYi(1)+(1−Ti)Yi(0)Y_i = T_i Y_i(1) + (1 - T_i) Y_i(0)

Treat and you see Yi(1)Y_i(1). Hold back and you see Yi(0)Y_i(0). The other side is the counterfactual, forever hidden for that unit. That is the fundamental problem: no row ever shows both inputs to its own difference.

Worked example: four shoppers, one discount

Score purchase as 1 and no purchase as 0. Four shoppers hold both futures:

Ana:Y(1)=1,Y(0)=0,effect 1Ben:Y(1)=1,Y(0)=1,effect 0Cara:Y(1)=0,Y(0)=0,effect 0Dan:Y(1)=1,Y(0)=0,effect 1\begin{aligned} \text{Ana} &: Y(1)=1, Y(0)=0, \text{effect } 1 \\ \text{Ben} &: Y(1)=1, Y(0)=1, \text{effect } 0 \\ \text{Cara} &: Y(1)=0, Y(0)=0, \text{effect } 0 \\ \text{Dan} &: Y(1)=1, Y(0)=0, \text{effect } 1 \end{aligned}

Add the four effects:

1+0+0+1=21 + 0 + 0 + 1 = 2

Divide by four for the average treatment effect, meaning the expected push across the population:

ATE=2/4=0.50\text{ATE} = 2 / 4 = 0.50

The true lift is 50 points. Now suppose Ana and Ben happen to get the code while Cara and Dan do not. Observed treated mean is (1+1)/2=1.00(1 + 1)/2 = 1.00. Observed control mean is (0+0)/2=0.00(0 + 0)/2 = 0.00. The naive gap is:

1.00−0.00=1.001.00 - 0.00 = 1.00

That split reads double the truth because the treated pair started stronger. That gap between 1.00 seen and 0.50 true is the diagram headline: four complete pairs on the left averaging 0.50, one partial view on the right reading 1.00.

When the naive gap equals the truth

Write E\mathbb{E} for the population average. The naive comparison is E[Y∣T=1]−E[Y∣T=0]\mathbb{E}[Y \mid T=1] - \mathbb{E}[Y \mid T=0], meaning average among those who got treated minus average among those who did not. It equals E[Y(1)]−E[Y(0)]\mathbb{E}[Y(1)] - \mathbb{E}[Y(0)] only when assignment TT is independent of both futures. Random assignment buys that independence. Choice by shoppers, doctors, or past behavior breaks it, and the 1.00 versus 0.50 split above is exactly that break.

Code

Rebuild the four shopper table and show the true average beside the lucky split.

y1: list[int] = [1, 1, 0, 1]y0: list[int] = [0, 1, 0, 0]
effects: list[int] = [a - b for a, b in zip(y1, y0)]true_ate: float = sum(effects) / len(effects)print(effects)  # -> [1, 0, 0, 1]print(f"{true_ate:.2f}")  # -> 0.50
treated_seen: list[int] = [y1[0], y1[1]]control_seen: list[int] = [y0[2], y0[3]]naive: float = sum(treated_seen) / 2 - sum(control_seen) / 2print(f"{naive:.2f}")  # -> 1.00

The printout matches the hand arithmetic: effects [1,0,0,1][1, 0, 0, 1], true average 0.500.50, naive split 1.001.00. Swap who gets treated and the naive number moves while the true average stays put.

Watch Out For

Reading the treated average as the treated future

Symptom: a dashboard shows treated buyers convert at 1.00 and the team reports a 100 point lift, while holdout buyers convert near 0.00 for reasons that predate the code. Fix: label Y(1)Y(1) as everyone if treated and Y∣T=1Y \mid T=1 as those who happened to get treated, and never subtract the second pair without an independence argument from ab testing for ml or an adjustment from propensity score methods.

One shopper changes the next row

Symptom: a vaccine or referral lift looks strong in isolation, then fades on rollout because treated units protect or recruit their controls, so each YiY_i depends on more than its own TiT_i. Fix: check the no interference assumption first. If spillover is real, randomize whole groups instead of individuals and estimate the group effect.

The Quick Version

  • Each unit holds two futures, Y(1)Y(1) treated and Y(0)Y(0) not, with individual effect Y(1)−Y(0)Y(1) - Y(0).
  • Only the future matching the actual assignment shows, so no individual effect is ever directly visible.
  • Four shoppers with effects 1, 0, 0 and 1 give a true average effect of 2 divided by 4, which is 0.50.
  • One lucky split treats the two strongest shoppers, so its naive gap reads 1.00 minus 0.00, which is 1.00.
  • Naive gaps equal causal averages only when assignment is independent of both futures, which randomization buys.
  • When units interfere, the two future table breaks and group assignment is the repair.