Higher-Order Derivatives

Differentiate the derivative and you get curvature: the second derivative tells whether the slope itself is rising or falling, which sorts valleys from hills.

The W shaped curve x to the fourth minus 3 x squared near x equals 1: the tangent slopes downward at minus 2 while the curve bends upward with curvature 6, a falling slope inside a valley

Why Does This Exist?

First derivatives find flat points; they cannot tell valleys from hills from passes. That sorting needs curvature, the second derivative, and it powers real decisions: Newton's method steps use curvature to converge quadratically, the second-order terms of Taylor series carry the approximation's accuracy, and in deep learning the Hessian's eigenvalues reveal sharp versus flat minima that generalize differently.

Derivatives of common functions supplies the first-derivative table this page differentiates again. This page answers what the repeats mean.

The one idea you need first

The first derivative $f'(x)$ is the slope at $x$ . The second derivative $f''(x)$ is the slope of the slope: it measures how the slope itself changes as you move. Positive means slopes are increasing as you go right, which bends the curve upward like a bowl.

Think of It Like This

Reading the road's bend, not just its grade

A grade sign says the road currently falls 2 percent: that is the first derivative. But it says nothing about what comes next. If the grade signs ahead read steeper and steeper downhill, you are cresting into a dive; if they read flatter, you are bottoming into a valley. The rate at which the grade itself changes is the second derivative.

Drivers brake for the bend, not the grade. Optimizers should too: a steep slope into a sharp valley floor needs a short step, while the same slope on a gentle straightaway takes a long one. Curvature sizes the step, which is Newton's whole insight.

How It Actually Works

Differentiate repeatedly: $f'$ is first order (slope), $f''$ is second order (curvature), $f'''$ third, and so on. The sign chart for $f''$ :

$f''(x) > 0$ : concave up, bowl shaped, slopes increasing left to right.
$f''(x) < 0$ : concave down, dome shaped, slopes decreasing.
$f''(x) = 0$ : possibly an inflection where curvature flips, test further.

The second-derivative test: at a flat point ( $f' = 0$ ), positive $f''$ confirms a local minimum and negative confirms a local maximum.

Worked example

$f(x) = x^4 - 3x^2$ . Differentiate term by term:

$f'(x) = 4x^3 - 6x$ . At $x = 1$ : $4 - 6 = -2$ . The curve falls through $x = 1$ .
$f''(x) = 12x^2 - 6$ . At $x = 1$ : $12 - 6 = 6 > 0$ . Curvature bends upward.

So at $x = 1$ the function value is $1 - 3 = -2$ , the slope is $-2$ (still descending), but the upward curvature 6 says a valley floor is near. Indeed the minimum sits at $x = \sqrt{1.5} \approx 1.22$ , just right of our point: falling slope plus upward bend means the bottom is ahead, not behind.

Watch Out For

Reading the second derivative as the slope direction

The symptom is "positive second derivative, so the function rises," which confuses bend with grade. In the worked example $f''(1) = 6$ sits beside a falling slope of $-2$ : the valley bends up while the path still descends. Fix it with two separate questions: $f'$ says which way you walk, $f''$ says how the ground bends underfoot. Never let one answer both.

Paying full Hessian price in high dimensions

Second derivatives generalize to the Hessian matrix with $n^2$ entries for $n$ parameters: a million-parameter model means a trillion-entry matrix nobody stores. The symptom is an out-of-memory crash from calling a full second-order routine on a real network. Use curvature cheaply instead: Hessian-vector products, diagonal approximations, or quasi-Newton updates like BFGS that build curvature estimates from gradients. Full Hessians are for analysis and small problems.

The Quick Version

$f''$ is the slope of the slope: positive bends upward, negative bends downward.
At a flat point, $f'' > 0$ confirms a valley and $f'' < 0$ confirms a hill.
For $x^4 - 3x^2$ at $x = 1$ : value $-2$ , slope $-2$ , curvature 6, valley ahead.
Curvature sizes Newton steps, which is why they converge fast near minima.
Bend and grade are separate questions: upward bend can accompany a falling slope.
In $n$ dimensions curvature is an $n \times n$ Hessian, so approximate it.