Covariance
A measure of whether two variables tend to be above or below their means at the same time.
Think of It Like This
Like watching whether two kite flyers both see their kites rise and dip at the same moments as the wind shifts across an open field.
Covariance is . When both variables are above their means together the product is positive; when they move oppositely it is negative; averaging those products says which tendency wins. Note that a variable's covariance with itself is exactly its variance.
The catch is that it has no fixed scale. Its magnitude depends entirely on the units involved, so measuring a length in metres rather than millimetres changes the covariance by a factor of a thousand while the relationship is unchanged. A covariance of 4.2 therefore means nothing on its own — divide by both standard deviations to get a correlation coefficient, which is bounded and interpretable.
Collecting the pairwise covariances of many features gives a covariance matrix, which is symmetric, has the variances down its diagonal, and is the object PCA decomposes.