Core ML

Fairness Metrics

Math cannot define what is fair, but it can measure exactly how unequal your model's mistakes are across different demographic groups.

Why Does This Exist?

A machine learning model optimizes for aggregate accuracy. If ninety percent of your training data belongs to one demographic group, a model can achieve ninety percent accuracy by simply memorizing that group and guessing randomly on the rest. Overall accuracy hides local failures.

Fairness metrics exist because "fairness" is a social construct, not a mathematical one. We need quantitative ways to test whether a model's benefits and harms (like false positives and false negatives) are distributed equally across protected attributes like race, gender, or age. Because different definitions of fairness mathematically contradict each other, these metrics force you to explicitly choose which type of equality you are optimizing for.

Think of It Like This

Imagine a bank giving out loans to two different neighborhoods.

Demographic Parity says: "Both neighborhoods should have the exact same percentage of applicants approved, regardless of anything else."

Equal Opportunity says: "If someone is actually capable of repaying the loan (the ground truth is positive), they should have the exact same chance of being approved, regardless of which neighborhood they live in."

You often cannot satisfy both at once. If one neighborhood historically has a lower repayment rate, enforcing Demographic Parity will lower the approval rate for qualified people in that neighborhood (violating Equal Opportunity). You have to choose which metric matches your definition of fairness.

How It Actually Works

Fairness metrics generally fall into three categories, each measuring a different property of the model's predictions $\hat{Y}$ , the ground truth $Y$ , and a protected attribute $A$ .

1. Independence (Demographic Parity)

Also known as statistical parity. A model satisfies demographic parity if the prediction is statistically independent of the protected attribute: $P(\hat{Y} = 1 | A = 0) = P(\hat{Y} = 1 | A = 1)$

What it means: The model predicts the positive outcome at the same rate for all groups. When to use it: When historical bias has contaminated the ground truth labels $Y$ (e.g., historical hiring data is biased), so you cannot trust the historical baseline and want to enforce equal representation.

2. Separation (Equalized Odds)

A model satisfies equalized odds if its prediction is conditionally independent of the protected attribute, given the ground truth: $P(\hat{Y} = 1 | Y = y, A = 0) = P(\hat{Y} = 1 | Y = y, A = 1)$ for $y \in \{0, 1\}$

What it means: The model must have both the same True Positive Rate (TPR) and the same False Positive Rate (FPR) across all groups. When to use it: When the ground truth $Y$ is reliable, and you want to ensure the model's mistakes (both false positives and false negatives) do not disproportionately affect one group.

A relaxed version of this is Equal Opportunity, which only requires the True Positive Rates to be equal: $P(\hat{Y} = 1 | Y = 1, A = 0) = P(\hat{Y} = 1 | Y = 1, A = 1)$ .

3. Sufficiency (Predictive Rate Parity)

A model satisfies sufficiency if the ground truth is conditionally independent of the protected attribute, given the prediction: $P(Y = 1 | \hat{Y} = 1, A = 0) = P(Y = 1 | \hat{Y} = 1, A = 1)$

What it means: If the model predicts a positive outcome, the likelihood of that prediction being correct is the same across all groups. This is equivalent to having equal Precision (Positive Predictive Value) across groups.

The Impossibility Theorem

A fundamental mathematical proof in fairness literature states that unless the base rates of the groups are identical (which is almost never true in practice), a model cannot simultaneously satisfy Equalized Odds and Predictive Rate Parity. You must choose one metric over the other.

Show Me the Code

You can measure fairness metrics by calculating standard classification metrics for each subgroup and comparing them.

1import numpy as np2
3def equal_opportunity_difference(y_true, y_pred, sensitive_attr, priv_group, unpriv_group):4    """Calculates the difference in True Positive Rates between two groups."""5    6    def calculate_tpr(y_t, y_p):7        true_positives = np.sum((y_t == 1) & (y_p == 1))8        actual_positives = np.sum(y_t == 1)9        return true_positives / actual_positives if actual_positives > 0 else 010        11    # Mask for privileged group12    priv_mask = (sensitive_attr == priv_group)13    tpr_priv = calculate_tpr(y_true[priv_mask], y_pred[priv_mask])14    15    # Mask for unprivileged group16    unpriv_mask = (sensitive_attr == unpriv_group)17    tpr_unpriv = calculate_tpr(y_true[unpriv_mask], y_pred[unpriv_mask])18    19    # Difference (ideal is 0.0)20    return tpr_unpriv - tpr_priv21
22# Example usage:23# y_true = np.array([1, 1, 0, 1, 1, 0])24# y_pred = np.array([1, 0, 0, 1, 1, 1])25# sensitive_attr = np.array(['M', 'M', 'M', 'F', 'F', 'F'])26# diff = equal_opportunity_difference(y_true, y_pred, sensitive_attr, 'M', 'F')27# -> 0.5 (Females have higher TPR in this dummy data)

Watch Out For

Blindly forcing Demographic Parity

Enforcing demographic parity when base rates differ can force the model to reject qualified candidates in one group and accept unqualified candidates in another, simply to hit a quota. This can cause harm if the ground truth labels are actually unbiased.

Ignoring intersectionality

Measuring fairness for "Race" and "Gender" separately might show the model is fair. But if you measure "Race AND Gender" (e.g., Black Women), the model might have severe bias that was hidden when looking at the single-axis aggregates.

The Quick Version

Standard metrics like accuracy hide bias because they aggregate performance across all groups.
Demographic Parity ensures equal selection rates across groups, regardless of ground truth.
Equal Opportunity ensures equal True Positive Rates, meaning qualified individuals have the same chance regardless of group.
The Impossibility Theorem proves you cannot satisfy all fairness metrics simultaneously if the groups have different base rates. You must choose the metric that aligns with the specific harm you are trying to prevent.