Correlation Coefficient
Covariance rescaled by both standard deviations, giving a unit-free strength of linear relationship between minus one and one.
Think of It Like This
Like measuring how closely two dancers mirror each other's footwork on a scale from synchronized lockstep to opposite directions, ignoring how tall they are.
The Pearson correlation is , always in . One is a perfect straight-line increase, minus one a perfect decrease, zero no linear relationship.
The word linear is the part that gets skipped and matters most. Correlation only detects straight-line association. over a symmetric range around zero has a correlation of exactly zero while being perfectly determined by . So a zero correlation is not evidence of independence, and dropping a feature because it correlates poorly with the target is a good way to discard something genuinely predictive.
Pearson is also sensitive to outliers, since it inherits variance's squaring. When the relationship is monotonic but not straight, or the data has extreme values, Spearman's rank correlation is the more honest choice.