Convex Function
A function shaped like a bowl, where any straight line between two points on it stays above the surface — so its only stationary point is the global minimum.
Think of It Like This
A single valley with no side pockets: walk downhill and you cannot end up anywhere but the bottom.
Convexity is the property that makes optimisation easy, because it guarantees that any local minimum is the global minimum. Gradient descent on a convex function converges to the best answer from any starting point, and the result does not depend on initialisation or random seed.
Linear regression, logistic regression, and support vector machines are all convex, which is why they train reproducibly and have well-understood convergence guarantees. Neural networks are emphatically not — every hidden layer with a nonlinearity destroys convexity, and permuting a layer's neurons yields an identical loss, so equivalent minima exist in enormous numbers.
Formally the test is curvature: a twice-differentiable function is convex exactly when its Hessian has no negative curvature in any direction. The practical consequence of losing convexity is that deep learning results depend on initialisation, ordering, and seed, which is why reporting a single run is weak evidence.