Data Concepts

Data Governance, Anonymisation and Privacy-Preserving ML

Removing names is not anonymising. Four ordinary columns can identify almost anyone, and the only real defences trade accuracy for a measurable privacy budget.

Three ordinary columns narrow a population of 67 million down to fewer than one person per combination, which is why dropping the name column anonymises nothing

Why Does This Exist?

A team drops name, email and phone from a dataset, calls it anonymised, and shares it with a partner. Every remaining column is innocuous: postcode district, date of birth, sex, a few behavioural counts.

Take the United Kingdom's 67 million people. There are about 3,000 postcode districts, roughly 36,500 plausible dates of birth, and two values for sex. That's 219 million combinations for 67 million people — an average of 0.31 people per combination.

Which means most combinations contain one person or nobody. Run the arithmetic and about 85% of the occupied combinations hold exactly one individual. Those three columns are an identifier. Cross-reference them with an electoral roll, a data breach, or a social media profile, and you have a name back.

Removing direct identifiers doesn't anonymise data. That's the fact this whole subject is built on, and it's why the serious techniques all cost you something measurable.

Think of It Like This

Twenty questions, and the room is smaller than you think

Twenty yes-or-no questions can distinguish about a million things, because each one halves the field. That's why the game works.

Now the reverse. To pick one person out of 67 million you need about 26 halvings — and a column with 3,000 values does about eleven of them on its own. Date of birth does fifteen. Two columns and you're done.

So the intuition to correct is this: identification doesn't require an identifier. It requires enough columns, and "enough" is a much smaller number than anyone expects, because information compounds multiplicatively rather than additively.

Redacting the name is like refusing to answer the first question.

How It Actually Works

The vocabulary that makes the problem statable

Direct identifiers name someone: name, email, national insurance number. Quasi-identifiers don't, but combine into one: postcode, date of birth, sex, job title, employer. Sensitive attributes are what you're protecting: diagnosis, salary, religion, sexuality.

The whole failure mode is treating quasi-identifiers as safe because none of them identifies anyone alone.

The techniques, weakest first

Redaction and masking. Drop or hash the direct identifiers. Necessary, and nowhere near sufficient, as the arithmetic above shows. A hashed email is also not anonymous: hash a candidate email and compare, and the space of real email addresses is small enough to enumerate.

Generalisation and k-anonymity. Coarsen the quasi-identifiers until every combination covers at least $k$ people — a postcode district becomes a region, a date of birth becomes a five-year band. It's the classic approach and it has two known holes. Homogeneity: if all $k$ people in a group share the same diagnosis, you've learned the diagnosis without identifying anyone. And composition: two separately k-anonymous releases can be intersected to break both. l-diversity and t-closeness patch the first hole and not the second.

Aggregation with thresholds. Publish counts, suppress cells below a minimum. Simple, effective, and defeated by differencing: two queries that differ by one person reveal that person.

Differential privacy. The one with a real guarantee. Add calibrated noise so that the output is statistically almost the same whether or not any single individual is in the data. The strength is a parameter, epsilon, and it's a budget: every query spends some, and the guarantee degrades as they accumulate, which is what makes the differencing attack impossible rather than merely inconvenient.

It's the only technique here that survives an attacker with outside information, and it costs accuracy. That's not a flaw — it's the trade made explicit and measurable, which every other technique also makes and doesn't tell you about.

Federated learning. Don't move the data; move the model. Each device or hospital trains locally and sends updates. Useful, and not private on its own: gradients leak information about the rows that produced them, and reconstruction attacks on gradient updates work. Federated learning plus secure aggregation plus differential privacy is a privacy story. Federated learning alone is a data-residency story.

Synthetic data is not a privacy technique by default

A generator can memorise and reproduce a training record, and it's most likely to do so for rare records — which are precisely the identifiable ones. So synthetic data inherits its privacy from how it was generated: train the generator with differential privacy and you have a claim, otherwise you have a hope. Test it with membership inference: can an attacker tell whether a specific record was in the training set?

Governance is the part that isn't a technique

Purpose limitation, so data collected for billing isn't used for a risk model without a fresh basis. Retention deadlines, and the ability to actually delete on request — which means knowing every derived dataset a row reached, so this is a lineage problem before it's a legal one. Access control by role. And a record of the lawful basis for training, which belongs in the data card.

The engineering version of the deletion right is worth stating plainly: if you can't enumerate which datasets and which models a record influenced, you can't comply, no matter what the policy says.

Worked example

3,000 postcode districts, 36,500 dates of birth, 2 values for sex. That's 219 million combinations.

Divide 67 million people across them and you get 0.306 people per combination. Modelling occupancy as Poisson with that mean, the share of occupied combinations holding exactly one person is

\frac{\lambda e^{-\lambda}}{1 - e^{-\lambda}} = \frac{0.306 \times 0.736}{1 - 0.736} = 0.855

So 85.5% of the groups that contain anybody contain exactly one person. Three columns, no names, and the majority of the population is uniquely identified.

Coarsening helps in a way you can compute: replace the district with one of 12 regions and a birth date with a 5-year band, and the combination count drops to roughly 12 × 20 × 2 = 480, so the average group holds about 140,000 people. That's genuine k-anonymity, and it cost you most of the geographic and age signal your model wanted.

Show Me the Code

1import numpy as np2
3population: int = 67_000_0004districts: int = 3_000      # postcode districts5birthdays: int = 36_500     # a century of dates6sexes: int = 27
8def mean_per_group(buckets: int) -> float:9    return population / buckets10
11def fraction_unique(lam: float) -> float:12    """Of the groups holding anyone, the share holding exactly one person."""13    return float(lam * np.exp(-lam) / (1 - np.exp(-lam)))14
15fine: int = districts * birthdays * sexes        # raw quasi-identifiers16coarse: int = 12 * 20 * sexes                    # regions and 5-year age bands17
18print(fine / 1e6, round(mean_per_group(fine), 3))       # -> 219.0 0.30619print(round(fraction_unique(mean_per_group(fine)), 3))  # -> 0.85520print(coarse, round(mean_per_group(coarse)))             # -> 480 139583

The first two lines are why "we removed the names" isn't a claim. The third is what fixing it costs.

Watch Out For

Calling a dataset anonymised because the identifiers are gone

df.drop(columns=["name", "email", "phone"]), then sharing it.

The remaining columns identify people, and the arithmetic isn't close: three ordinary columns make 85% of individuals unique. Adding behavioural columns makes it worse, because a purchase history or a set of visited pages is close to a fingerprint on its own.

Hashing doesn't fix it either. A hashed email is a stable pseudonym, so it still links records across datasets, and the space of real email addresses is small enough to enumerate and match. Pseudonymisation is a useful control and it is not anonymisation, and the two get conflated in exactly the meetings where the distinction matters.

Before sharing, count the distinct combinations of every quasi-identifier and check the smallest group size. If any combination has fewer than $k$ people, generalise until it doesn't, or don't share those columns. Then write what you did in the data card, because the next person will ask.

Treating federated or synthetic data as private by construction

Two versions of the same mistake, both common in project proposals.

"It's federated, the data never leaves the hospital." True, and gradient updates carry information about the rows that produced them — reconstruction attacks recover recognisable training examples from updates. Federated learning gives you data residency. Privacy needs secure aggregation and differential privacy on top.

"It's synthetic, so there's no personal data." A generator trained without a privacy guarantee can and does reproduce training records, most readily the rare ones, which are the identifiable ones. A synthetic dataset can therefore contain a real person's record verbatim.

Both are testable rather than arguable. Run a membership-inference check: train an attacker to decide whether a given record was in the training set, and report its advantage over chance. If it's meaningfully above chance, the privacy claim is false and you have the number to say so. And if you need a guarantee rather than an assessment, differential privacy during training is the only route that provides one.

The Quick Version

Removing direct identifiers isn't anonymising. Quasi-identifiers combine into an identifier.
3,000 districts × 36,500 birth dates × 2 sexes is 219 million combinations for 67 million people, so 85% of occupied groups hold exactly one person.
Hashing an email makes a stable pseudonym, not an anonymous value, and the address space is enumerable.
k-anonymity generalises until every group holds $k$ people. It fails on homogeneous groups and on composition across two releases.
Aggregation with thresholds is defeated by differencing two queries.
Differential privacy is the one with a guarantee, and epsilon is a budget every query spends. It costs accuracy, and that trade is the point.
Federated learning gives data residency. Gradients leak, so privacy needs secure aggregation and differential privacy on top.
Synthetic data inherits privacy from how the generator was trained, and memorises rare records by default.
Deletion rights are a lineage problem: you can't delete what you can't trace.
Coarsening to regions and 5-year bands gives real k-anonymity and costs most of your geographic and age signal.