Derivative
The rate at which a function's output changes per unit change in its input, measured at one point.
Think of It Like This
A speedometer, not an odometer — it reads the rate right now, not the total so far.
The derivative is the slope of the line that just touches the curve at . Formally it is the limit of rise over run as the run shrinks to nothing. For the derivative is , so at the output climbs four times as fast as the input.
Two pieces of information come out and they get used differently. The sign says which way to move, which is where training's minus sign comes from. The size says how sensitive the output is, which is why one global learning rate struggles when different parameters have derivatives orders of magnitude apart.
The word to hold on to is local. A derivative describes the slope right here and promises nothing further away, which is precisely why training takes many small steps rather than jumping to where the slope suggests the minimum is. A derivative of 4 does not predict the output will rise by 4; it predicts a rise of about 4 per unit, for a small enough step.