Math
Linear algebra, calculus, probability for ML
Probability
- Math
Probability Foundations
The rules for reasoning when you do not know the answer — how to combine chances, and what changes the moment you learn something new.
- Math
Bayes' Theorem
The rule for flipping a conditional around — and the reason a highly accurate test for a rare thing still returns mostly false alarms.
- Math
Bayesian Inference
Keep the whole distribution over parameters instead of one best guess, so the model can report how much it does not know — and pay for it with an integral that has to be dodged rather than solved.
- Math
Expectation, Variance and Covariance
Three summary numbers: where the values centre, how far they scatter from that centre, and whether two of them scatter together.
- Math
MAP Estimation
Maximum likelihood with an opinion added: multiply the likelihood by a prior before maximising, and the penalty terms you already use for regularisation fall out of the algebra.
- Math
Maximum Likelihood Estimation
Pick the settings that make what you actually observed the least surprising thing that could have happened — which is, precisely, what training a model does.
- Math
Monte Carlo Methods
Replace an integral you cannot solve with an average over random samples: the error falls as one over the square root of the sample count, and — uniquely — that rate does not care how many dimensions you are in.
- Math
Probability Distributions
A fixed budget of belief spread across every outcome that could happen — and the shape you assume it takes is what silently decides your loss function.
- Math
Combinatorics and Counting
Mathematical techniques for counting the number of possible outcomes in finite sets.
- Math
Importance & Rejection Sampling
Tricks for generating random numbers from a hard shape by drawing from an easy shape instead, and then correcting the mistakes.
- Math
Markov Chain Monte Carlo (MCMC)
When you can't measure the whole space, you take a clever random walk that naturally spends more time in the most important areas.
- Math
Markov Chains
A system that leaps from state to state where the next destination only depends on where you are right now, not how you got there.
- Math
Multivariate Gaussian Distribution
The classic bell curve extended into multiple dimensions, shaping out an elliptical cloud where things are most likely in the center.
- Math
Softmax Function and its Derivative
Converts a vector of raw scores into a probability distribution where all values sum to 1.
Calculus
- Math
Convexity and Loss Landscapes
Some surfaces have exactly one bottom and walking downhill always finds it; neural network surfaces do not, and the reason training still works is not what most people assume.
- Math
Gradients
A list of slopes, one per knob, that together point in the single direction where the error climbs fastest — so the opposite direction is where training goes.
- Math
Automatic Differentiation
Frameworks do not solve derivatives symbolically or estimate them numerically — they apply the chain rule mechanically to the operations you actually ran, which is exact and cheap.
- Math
The Chain Rule
When one thing feeds into another, you find the total effect by multiplying the effects along the path — which is exactly how a network learns which of its early weights to blame.
- Math
Computational Complexity for ML
Counting operations tells you how a cost grows; counting bytes moved tells you how long it actually takes — and in machine learning the second one usually wins.
- Math
Derivatives and Partial Derivatives
A single number that answers one question: if I nudge this input a little, how much does the output move, and in which direction?
- Math
Floating Point Formats
Every number format splits its bits between how large a value it can hold and how finely it can distinguish nearby ones — and training cares far more about the first than the second.
- Math
Jacobian and Hessian
One matrix holds every first derivative of a function with many inputs and outputs; another holds every second derivative and tells you which way the surface curves.
- Math
Constrained Optimisation and Duality
Optimising with a rule you cannot break — and the discovery that the multiplier enforcing the rule is also the price of relaxing it.
- Math
Numerical Stability
Floating point is not arithmetic — it is arithmetic with a budget of about sixteen digits, and the failures come from spending that budget carelessly rather than from any bug in your logic.
- Math
Taylor Series and Local Approximation
Replace a complicated function near one point with a simple polynomial that matches it there — the trick behind every optimisation step you have ever taken.
Statistics
- Math
Bootstrap and Resampling
Instead of deriving a formula for how uncertain your estimate is, resample your own data thousands of times and watch how much the answer moves.
- Math
Central Limit Theorem
Averages become bell-shaped no matter what they are averages of — which is the one fact that makes error bars, p-values and confidence intervals possible at all.
- Math
Confidence Intervals
A range around an estimate, built by a recipe that lands on the truth most of the time — which is a promise about the recipe, not about the range you happen to be holding.
- Math
Descriptive Statistics
The handful of numbers that summarise a column of data — and the reason one of them alone will mislead you about almost any real dataset.
- Math
Hypothesis Testing
A procedure for asking whether chance alone could plausibly have produced what you observed — and a p-value is the answer to that question, not to the one you probably wanted.
- Math
Multiple Comparisons
Test enough things at the usual threshold and a false positive stops being unlikely and becomes near certain — which is what every hyperparameter sweep is quietly doing.
- Math
Statistical Power and Sample Size
The probability your test finds a real effect. Work it out before you run the experiment, because a test with low power that returns nothing has not told you the effect is absent — it has told you nothing at all.
- Math
t-SNE & UMAP Math Basics
These algorithms compress complex high-dimensional data into 2D or 3D maps by ensuring that points close together in high dimensions stay close together in low dimensions.
Integration
- Math
Change of Variables
Transform a random variable and its density does not simply move with it — it gets rescaled by how much the transform stretched or squeezed space, and that scaling factor is a Jacobian determinant.
- Math
Integrals in Probability
For anything measured on a continuous scale, probability is area rather than height — which is why the chance of hitting an exact value is zero and why so many useful quantities have no formula.
- Math
Integration and Area Under a Curve
Adding up infinitely many infinitely thin slices to get a total — and the surprising fact that doing so is just differentiation run backwards.
Graphs
- Math
Computational Graphs
Every operation your model performs is recorded as a node in a one-way network, and training works by walking that network backwards — which is also why training eats so much memory.
- Math
Graph Fundamentals
Things and the connections between them — and the useful surprise that writing those connections in a grid turns the whole picture into a matrix you can do algebra on.
- Math
Graph Laplacian and Spectral Views
Degree minus adjacency turns a graph into a matrix whose eigenvalues describe how connected it is — which is how a discrete structure becomes something linear algebra can answer questions about.
Information Theory
- Math
Cross-Entropy
The price you pay for being confidently wrong: it reads off the probability your model gave the correct answer and charges you the negative logarithm of it.
- Math
Entropy
How surprised you should expect to be by the next outcome, measured as a number — highest when everything is equally likely, zero when you already know the answer.
- Math
Kullback–Leibler Divergence
How many extra bits you pay for encoding reality with the wrong model — and because the penalty is charged against reality's probabilities, not the model's, swapping the two gives a different number.
- Math
Mutual Information
How much knowing one variable shrinks your uncertainty about another — which catches every kind of dependence, including the curved and the categorical relationships correlation reports as zero.
Linear Algebra
- Math
Distance and Similarity Metrics
Several defensible ways to measure how alike two things are, which disagree with each other — so picking one is a modelling decision, not a formatting choice.
- Math
Dot Product
Multiply two lists of numbers position by position and add up the result — a single number saying how much the two point the same way.
- Math
Eigenvalues and Eigenvectors
The rare directions a matrix leaves pointing the same way, merely stretched — and the stretch factors that tell you what the matrix really does.
- Math
Einsum and Tensor Shapes
Give every axis a letter, write down which letters survive, and the operation follows — replacing a stack of reshapes and transposes with one readable line.
- Math
Linear Systems and Least Squares
When there is no exact answer, settle for the closest one — and the closest one has a property so specific it hands you a formula.
- Math
Linear Transformations
A matrix is not a grid of numbers, it is an instruction for reshaping space — and reading its columns tells you exactly where the axes end up.
- Math
Matrix Multiplication
Every entry of the answer is one dot product: a row from the left meeting a column from the right, which is why the inner dimensions have to agree.
- Math
Norms
A single number for how big a vector is — and there is more than one reasonable way to measure that, which is why the choice changes what your model learns.
- Math
Positive Definite Matrices
Matrices that curve upward in every direction — the property that makes a surface a bowl with one bottom, and the reason some optimisation problems are easy.
- Math
Singular Value Decomposition
Every matrix, no matter its shape, splits into a rotation, a set of independent stretches, and another rotation — and the stretches come ranked, so you can keep the ones that matter and drop the rest.
- Math
Vector Spaces and Rank
How many genuinely different directions a set of numbers actually contains — usually far fewer than its size suggests, which is the redundancy every compression trick exploits.
- Math
Vectors, Matrices and Tensors
Three names for the same idea at different sizes: a container of numbers, described entirely by how many axes it has and how long each one is.
- Math
Camera Models: Pinhole, Intrinsic/Extrinsic
Describes how a 3D world is mapped onto a 2D image plane via camera parameters.
- Math
Determinants, Matrix Inverse, and Trace
Fundamental properties of square matrices that describe volume scaling, reversibility, and sum of eigenvalues.
- Math
Epipolar Geometry
Epipolar geometry connects two views of the same 3D scene, showing how a point in one image restricts where it can appear in the other.
- Math
Homogeneous Coordinates and Projective Geometry
Adds an extra dimension to points, allowing translations and projective transformations to be represented as matrices.
- Math
Image Transformations: Affine, Rotation, Homography
Maps points from one image to another, representing perspectives and transformations.
- Math
Matrix Decompositions: LU, QR, Cholesky
Techniques for factoring matrices to solve linear systems more efficiently.
- Math
Orthogonality and Projections
Understanding perpendicular vectors and how to project one vector onto another or a subspace.
- Math
PCA Derivation
Principal Component Analysis rotates our coordinate system so that the first axis points in the direction where the data varies the most.
Optimization
- Math
Common Loss Functions
Loss functions translate how 'wrong' a model's prediction is into a single number that the algorithm can mathematically minimize.
- Math
Convex Sets & Convex Function Properties
A shape where you can see every point from every other point without leaving it, leading to optimization problems with only one true bottom.
- Math
Gradient Descent Variants
Advanced optimization algorithms like SGD, Momentum, RMSprop, and Adam that improve upon basic gradient descent.
- Math
KKT Conditions
The ultimate test for whether you have found the best possible solution when your choices are limited by hard rules.
- Math
Learning Rate Schedules
Starting slow to avoid breaking the model, speeding up to make progress, and then slowing down again to carefully settle into the perfect solution.
- Math
Newton's Method and BFGS
Second-order optimization methods that use curvature information for faster convergence.
Foundations
- Math
Complex Numbers
Numbers with a real and imaginary part, essential for understanding frequency domains and certain transformations.
- Math
Fourier Transform / Frequency Domain, Convolution Theorem
Translates signals from the time domain into the frequency domain.
- Math
Image Pyramids & Edge Detection
Image pyramids represent an image at multiple resolutions, while edge detection finds the boundaries where pixel intensities change sharply.
- Math
Interpolation Methods
Interpolation fills in the missing gaps between known data points by estimating intermediate values using adjacent neighbours.