Algorithmic Bias

Models copy the past at scale, so skewed data, skewed labels and skewed reuse all show up as skewed decisions unless you measure each stage.

Dropping the protected column does not remove bias because a correlated proxy carries the same signal into the model.

Why Does This Exist?

Learning from data sounds neutral until you ask where the data came from. Past hiring favored some groups, past policing watched some streets, past notes described some patients in less detail. A model trained on those traces repeats them, politely and at scale, unless someone checks each stage for the skew.

This page names where bias enters and follows one hiring fixture from data to deploy. How to score the gap lives in AI fairness. The legal reading lives in disparate impact. The person level split lives in individual vs group fairness. The audit routine lives in model auditing.

Think of It Like This

An apprentice taught from old files

A firm trains a new hiring assistant on ten years of past decisions from a time when its engineers were mostly men. The apprentice studies hard and learns the pattern cold: resumes like the old hires get shortlisted. No one told the apprentice to prefer men. The files did.

Now the apprentice screens thousands of applications a day without tiring. The old preference, once a few managers habit, becomes the house rule. That is a model on biased data: a faithful student of an unfair past, promoted to judge the future.

Where it stops: you can sit the apprentice down and show new cases. A model keeps scoring the old weights until someone retrains the pipeline and fixes the labels, the features and the reuse loop around it.

How It Actually Works

Let $G$ be the group label, here 1 for men and 0 for women. Let $Q$ be true qualification, which you cannot see directly. Let $H$ be the recorded hire, 1 for hired, which is what the data actually stores. Let $S$ be the selection rate per group. Let $DI$ be the disparate impact ratio, the smaller $S$ divided by the larger one.

The five sources

Historical bias. $H$ reflects old preferences, so training on $H$ teaches the preference, not just $Q$ .
Representation bias. One group appears rarely, so the model learns its patterns poorly and errs there more.
Measurement bias. Labels record a proxy, such as rearrest standing in for reoffense, and the proxy itself leans by group.
Aggregation bias. One model serves groups whose feature patterns differ, so the single fit suits the majority.
Feedback bias. Deployments feed themselves: more patrols in flagged streets produce more arrests there, which confirm the flag.

How it travels the pipeline

Bias can enter at every step, and a fix at one step leaves the others intact.

Problem framing picks the target, and a leaning target leans everything after it.
Data collection samples and labels unevenly.
Feature work keeps proxies, such as postal code, that guess $G$ well.
Training weights errors unevenly across groups.
Evaluation reports one average that hides the split.
Deployment loops the outputs back in as new inputs.

Worked example with ten and ten

Ten men and ten women apply with equal qualifications. The firm hires 7 men and 3 women. Selection rates are $S = 7 / 10 = 0.70$ for men and $S = 3 / 10 = 0.30$ for women. The disparate impact ratio is $DI = 0.30 / 0.70 \approx 0.43$ , far under the 0.80 floor.

Now drop the gender column and retrain. A postal code proxy still guesses gender right on 9 of 10 applications, so $C = 9 / 10 = 0.90$ of the signal survives. The new model scores almost the same gap, because the information never left. The diagram draws this in three panels: real world data with both columns, feature selection with gender crossed out, and the deployed model with the proxy arrow leaking the signal through.

Code

The gap and the leak, in plain Python with no libraries.

1def selection_rate(hired: list[int]) -> float:2    return sum(hired) / len(hired)3
4
5male: list[int] = [1, 1, 1, 1, 1, 1, 1, 0, 0, 0]6female: list[int] = [1, 1, 1, 0, 0, 0, 0, 0, 0, 0]7di: float = selection_rate(female) / selection_rate(male)8print(round(di, 3))  # -> 0.4299
10correct: int = 911total: int = 1012print(correct / total)  # -> 0.9

Disparate impact prints 0.429. Proxy accuracy prints 0.9. Read together they say the column is gone and the signal stayed.

Watch Out For

Trusting removal of the sensitive column

Symptom: gender leaves the features, the audit gap barely moves, and the team concludes the data must be fair. The proxy kept 90 percent of the signal. Fix: measure how well remaining features predict the removed column, then constrain training or set per group thresholds, and re audit the gap.

Fixing only the model weights

Symptom: a fairness penalty during training looks good in the notebook while labels, framing or the deploy loop stay skewed, so live gaps return within weeks. Fix: audit the full chain with model auditing: target choice, labels, features, weights, slices and the feedback path that reingests outputs.

The Quick Version

Bias enters as history, thin samples, leaning labels, one fit for all groups, and loops that confirm themselves.
Our fixture hires 7 of 10 men and 3 of 10 women, scoring disparate impact 0.43.
A postal code proxy guessing gender at 0.90 keeps the signal after the column is dropped.
Pipeline fixes must cover framing, data, features, training, slices and deploy loops together.
Report selection rates per group, never one average alone.
Name the governing fairness definition from AI fairness before tuning anything.