Counting and Basic Probability

Every first choice branches into every second choice, so total outcomes multiply; probability is just the outcomes you want divided by all of them.

Three shirts times two pants make six outfits: every first choice branches into every second choice, so total outcomes multiply and probability divides wanted outcomes by all of them

Why Does This Exist?

Every password rule, dataset split, and grid search is a counting question in disguise. How many passwords does an 8 character rule allow? How many ways can you split 100 rows into 80 and 20? How many hyperparameter combinations does your search actually cover? Without counting, those questions get hand waves instead of numbers, and a search space quoted as "large" might be 360 or 360 million.

Counting also supplies both halves of the simplest probability in the field. Probability over equally likely outcomes is one division: the outcomes you want divided by all outcomes. Counting produces both numbers. The full treatment of what probability means beyond that division lives in probability foundations. This page covers the counting machinery underneath it.

The one idea you need first

A probability like $1/6$ is a fraction with a story: 1 wanted outcome sitting on top of 6 total outcomes. If the 6 is wrong, the probability is wrong, and in ML the 6 is where the work hides. Counting gets the denominator right.

Think of It Like This

Getting dressed from two drawers

Your shirt drawer holds 3 shirts and your pants drawer holds 2 pairs of pants. Pick a shirt first: 3 options. For each shirt you picked, you still have the full choice of 2 pants, because choosing a shirt never removes a pair of pants. So each of the 3 shirt branches sprouts 2 pants branches, giving $3 \times 2 = 6$ outfits.

Probability enters the moment someone asks for odds. What are the odds of the striped shirt with black pants? One branch out of six, so $1/6$ . That is the whole game: count the branches you want, count all branches, divide.

Where the analogy stops: drawers are independent, but real choices often are not. Picking players for a team removes them from later picks, which is where permutations and combinations take over.

How It Actually Works

Two principles do nearly all the work.

The multiplication principle. If one choice has $m$ options and a second independent choice has $n$ options, the pair has $m \times n$ outcomes. Chains keep multiplying: $m \times n \times k$ for three choices. A 4 digit PIN has $10^4 = 10{,}000$ outcomes because each of the 4 positions offers 10 digits.

The addition principle. If you want one outcome from group A ( $m$ options) or one from a disjoint group B ( $n$ options), the total is $m + n$ . The groups must not overlap; shared members get counted once, not twice.

Worked example

Wardrobe: 3 shirts (red, striped, plain) and 2 pants (black, khaki).

Total outfits: $3 \times 2 = 6$ . Written out: (red, black), (red, khaki), (striped, black), (striped, khaki), (plain, black), (plain, khaki).

Probability of (striped, black) with all outfits equally likely: $1/6 \approx 0.167$ .

Addition check: choosing today's shirt or borrowing 1 of 2 hats gives $3 + 2 = 5$ options, because shirts and hats are disjoint groups.

Scale it to ML: a grid search over 4 learning rates, 3 batch sizes, and 5 depths covers $4 \times 3 \times 5 = 60$ runs. Quote that number before booking the GPUs.

Watch Out For

Adding choices that should multiply

The symptom is a search space that looks small: "4 learning rates and 3 batch sizes, so 7 runs." But each learning rate pairs with every batch size, giving $4 \times 3 = 12$ runs, and the missing 5 never execute. The fix is grammatical: "and" between independent choices almost always means multiply, while "or" between disjoint groups means add. If in doubt, list a tiny case fully, the way the worked example lists all six outfits.

Assuming every outcome is equally likely

Counting gives the denominator; equal likelihood is a separate claim that is often false. A weighted sampler, an imbalanced dataset, or a biased coin all break the "divide by total" step even when the count is perfect. The symptom is a baseline probability that reality keeps missing, like expecting each class $1/3$ of the time on 90/5/5 data. Fix it by checking the sampling process before trusting the fraction, and reach for probability distributions when weights differ.

The Quick Version

Independent choices multiply: $m$ options then $n$ options give $m \times n$ outcomes.
Disjoint alternatives add: $m$ options or $n$ options give $m + n$ outcomes, with shared members counted once.
Probability over equally likely outcomes is wanted outcomes divided by total outcomes.
A grid of 4 rates, 3 batch sizes, and 5 depths is $4 \times 3 \times 5 = 60$ runs, not 12.
Equal likelihood is a claim about the process, not a reward for counting correctly.