AI Fairness

AI fairness asks whether a model treats different groups equitably — and reveals that multiple intuitive definitions of fairness are mathematically incompatible, forcing explicit value choices that cannot be avoided by technical means alone.

Why Does This Exist?

ML systems are now used for loan approvals, hiring, medical diagnoses, criminal justice risk scores, and content moderation. These are decisions that affect people's lives. When a model systematically favors or disadvantages groups based on protected attributes — race, gender, age, disability — it can perpetuate or amplify existing social inequities at machine speed and scale.

The COMPAS recidivism prediction tool (used in U.S. courts to predict re-offense likelihood) was found to have a false positive rate for Black defendants that was twice as high as for White defendants. Understanding what fairness means — and why it's hard — is the first step toward doing better.

Think of It Like This

A court that acquits at different rates by race

Imagine a court that acquits 80% of innocent White defendants but only 60% of innocent Black defendants, even though both groups commit the same proportion of crimes. No one would call this fair. ML models can do the equivalent — systematically failing for one group while succeeding for another — while reporting a high aggregate accuracy that hides the disparity.

How It Actually Works

Formal fairness definitions

There is no single definition of fairness. The major definitions make different value judgments:

Group fairness: Demographic parity (statistical parity) $P(\hat{Y} = 1 \mid A = 0) = P(\hat{Y} = 1 \mid A = 1)$ The positive prediction rate must be equal across groups. Does not require equal accuracy — only equal selection rates.

Equalized odds $P(\hat{Y} = 1 \mid Y = y, A = 0) = P(\hat{Y} = 1 \mid Y = y, A = 1) \quad \forall y \in \{0, 1\}$ Equal true positive rates AND equal false positive rates across groups. Requires the model to make errors at the same rate for everyone.

Equal opportunity (a relaxation) $P(\hat{Y} = 1 \mid Y = 1, A = 0) = P(\hat{Y} = 1 \mid Y = 1, A = 1)$ Only equal true positive rates — qualified individuals get equal chances.

Individual fairness Similar individuals should be treated similarly: $d_Y(\hat{f}(x_1), \hat{f}(x_2)) \leq L \cdot d_X(x_1, x_2)$ . Requires defining a meaningful similarity metric on individuals, which is often as contested as fairness itself.

Calibration $P(Y = 1 \mid \hat{p} = p, A = a) = p$ for all groups $a$ . The predicted probability is an accurate estimate of the true probability for each group.

The impossibility theorem (Chouldechova, 2017)

When base rates differ between groups (i.e., $P(Y=1|A=0) \neq P(Y=1|A=1)$ ), it is mathematically impossible to simultaneously satisfy:

Equal false positive rates across groups
Equal false negative rates across groups
Overall calibration

You must choose which notion to prioritize. This is a values choice, not a technical one.

Code

1import numpy as np2import pandas as pd3from fairlearn.metrics import (4    MetricFrame, demographic_parity_difference, equalized_odds_difference5)6from sklearn.linear_model import LogisticRegression7from sklearn.model_selection import train_test_split8
9# ── Simulate a dataset with a protected attribute ─────────────────────────────10np.random.seed(42)11n = 200012group = np.random.binomial(1, 0.4, n)  # protected attribute A ∈ {0, 1}13
14# Features correlated with group (simulating historical bias)15X = np.column_stack([16    np.random.randn(n),                           # feature 117    np.random.randn(n),                           # feature 218    group + np.random.randn(n) * 0.5,             # feature correlated with group19])20
21# Labels: base rates differ by group (group 1 has higher base rate)22base_rate = np.where(group == 0, 0.2, 0.35)23y = np.random.binomial(1, base_rate)24
25X_train, X_test, y_train, y_test, group_train, group_test = train_test_split(26    X, y, group, test_size=0.3, random_state=4227)28
29# ── Train an unconstrained classifier ────────────────────────────────────────30clf = LogisticRegression(max_iter=1000, random_state=42)31clf.fit(X_train, y_train)32y_pred = clf.predict(X_test)33
34# ── Evaluate fairness metrics ─────────────────────────────────────────────────35from sklearn.metrics import accuracy_score, precision_score, recall_score36
37mf = MetricFrame(38    metrics={39        "accuracy":  accuracy_score,40        "precision": precision_score,41        "recall":    recall_score,42    },43    y_true=y_test,44    y_pred=y_pred,45    sensitive_features=group_test,46)47
48print("Per-group metrics:")49print(mf.by_group.round(3))50
51print(f"\nOverall accuracy:       {accuracy_score(y_test, y_pred):.3f}")52print(f"Demographic parity diff: {demographic_parity_difference(y_test, y_pred, sensitive_features=group_test):.3f}")53print(f"Equalized odds diff:     {equalized_odds_difference(y_test, y_pred, sensitive_features=group_test):.3f}")

Watch Out For

Proxy discrimination through correlated features

Removing the protected attribute (race, gender) from the feature set does not remove discrimination if other features are correlated with it. Zip codes, names, and many behavioral features are strong proxies for race and ethnicity. "Fairness through unawareness" — simply dropping the protected attribute — is one of the least effective fairness interventions.

Fairness theater: measuring without acting

Publishing a fairness audit without changing the model or the decision process is fairness theater. Measuring disparities is the beginning of the process, not the end. After measuring, choose which fairness criterion your use case requires (this is a value judgment, not a technical one), apply a mitigation strategy (pre-processing, in-processing, or post-processing), and re-evaluate.

The Quick Version

Multiple mathematical definitions of fairness exist (demographic parity, equalized odds, calibration) — each reflects different values.
The impossibility theorem: when base rates differ across groups, you cannot simultaneously satisfy all fairness criteria — you must choose.
Removing protected attributes doesn't eliminate bias; proxy features carry the same signal.
Fairlearn, AIF360, and similar tools make fairness measurement practical.
Fairness is a socio-technical problem: measurement and mitigation are necessary but not sufficient without process and accountability changes.