AI Fairness

Why Does This Exist?

Loan, hiring and court tools decide for thousands of people in one run. A single accuracy number can hide a split: one group gets the benefit far more often, or pays for the errors far more often. You need named definitions before you can argue about which split is acceptable, and before an audit means anything.

This page gives you the definitions, the clash between them, and a ten plus ten example you can recheck by hand. Where the skew comes from lives in algorithmic bias. The person level split lives in individual vs group fairness. The legal reading lives in disparate impact. The audit routine lives in model auditing.

Think of It Like This

How It Actually Works

Let $A$ be the group label, with values 0 and 1. Let $Y$ be the true outcome, where 1 means the person qualifies. Let $\hat{Y}$ be the model decision, where 1 means the model selects the person. Every definition below compares a rate across the two values of $A$.

• Selection rate for group $a$ is $P(\hat{Y} = 1 \mid A = a)$: the share selected.
• True positive rate for group $a$ is $P(\hat{Y} = 1 \mid Y = 1, A = a)$: of the qualified, the share selected.
• False positive rate for group $a$ is $P(\hat{Y} = 1 \mid Y = 0, A = a)$: of the unqualified, the share wrongly selected.
• Base rate for group $a$ is $P(Y = 1 \mid A = a)$: the share who truly qualify.

The main definitions

• Demographic parity asks for equal selection rates: $P(\hat{Y} = 1 \mid A = 0) = P(\hat{Y} = 1 \mid A = 1)$. It says nothing about who was qualified.
• Equal opportunity asks for equal true positive rates: qualified people get selected at the same rate in both groups.
• Equalized odds asks for both: equal true positive rates and equal false positive rates.
• Calibration asks that a score means what it says in each group: $P(Y = 1 \mid \text{score} = s, A = a) = s$ for every score $s$ and group $a$.

Worked example with ten and ten

Group A has ten people and two true qualifiers. Group B has ten people and six true qualifiers. So the base rates are 0.20 and 0.60. Now force demographic parity at five selections each.

Group A selections: 5 of 10, so the selection rate is $5 / 10 = 0.50$. Both qualifiers are inside the five, so the true positive rate is $2 / 2 = 1.00$. Group B selections: 5 of 10, selection rate $5 / 10 = 0.50$. Five of the six qualifiers are inside, so the true positive rate is $5 / 6 \approx 0.833$.

Equal selection, unequal hit rates, with a gap of $1.00 - 0.833 = 0.167$. The diagram draws exactly this: ten dots per group, a box round five each, and the mismatch called out beside the true positive row.

Why you cannot have them all

Flip the demand. Force equal opportunity at 100 percent instead: select both qualifiers in A and all six in B. True positive rates match at 1.00, but selection rates split to $2 / 10 = 0.20$ against $6 / 10 = 0.60$. When base rates differ, one equality breaks the other. The general result says you cannot hold equal false positive rates, equal false negative rates and calibration all at once outside trivial cases. So you pick the definition that fits the stakes, write it down, and report all three rates anyway.

Code

Two tiny rates, computed on the worked example. No libraries, so you can paste this anywhere.

1def selection_rate(pred: list[int]) -> float:2    return sum(pred) / len(pred)3
4
5def true_positive_rate(y: list[int], pred: list[int]) -> float:6    hits = sum(1 for actual, got in zip(y, pred) if actual == 1 and got == 1)7    return hits / sum(y)8
9
10y_a: list[int] = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]11p_a: list[int] = [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]12y_b: list[int] = [1, 1, 1, 1, 1, 1, 0, 0, 0, 0]13p_b: list[int] = [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]14
15print(selection_rate(p_a))  # -> 0.516print(true_positive_rate(y_a, p_a))  # -> 1.017print(selection_rate(p_b))  # -> 0.518print(round(true_positive_rate(y_b, p_b), 3))  # -> 0.83319print(round(abs(true_positive_rate(y_a, p_a) - true_positive_rate(y_b, p_b)), 3))  # -> 0.167

Selection matches at 0.50 while the true positive gap prints 0.167. That printout is the clash in one number.

Watch Out For

The Quick Version

• Demographic parity equalizes selection rates. Equal opportunity equalizes true positive rates. Equalized odds asks for both error rates.
• Base rates differ in real data, here 0.20 against 0.60, and that difference forces the clash.
• Our ten plus ten fixture selects five each and lands true positive rates at 100 and 83 percent.
• Forcing equal 100 percent hit rates instead splits selection to 20 against 60 percent.
• Proxy features rebuild a deleted group column, so removal never suffices alone.
• Report all three rates per group and name the governing definition before release.

What to Read Next

• Algorithmic Bias
• Individual Vs Group Fairness
• Disparate Impact in AI Models
• Model Auditing