Lagrange Multiplier
A mathematical strategy used to find the local maxima and minima of a function subject to equality constraints by introducing a new scalar variable.
Think of It Like This
Like trying to find the highest point on a mountain, but you are strictly forbidden from leaving the hiking trail.
This method transforms a constrained optimization problem into an unconstrained one by combining the objective function and the constraint. The multiplier dictates how the objective function's value changes as the constraint is relaxed. It is fundamental in deriving the dual formulation of Support Vector Machines and handling complex loss functions.