Gradient

Written $\nabla \mathcal{L}$ and read "nabla" or "del", the gradient collects one partial derivative per parameter into a single vector — one number saying how much the loss would rise if you increased that parameter alone.

It points at steepest ascent, so training subtracts it. That minus sign is not decoration, and getting it wrong produces a loss that climbs smoothly while looking like a learning rate problem.

Two properties are worth keeping to hand. Its shape always matches the thing you differentiated with respect to, so a $200 \times 50$ weight matrix has a $200 \times 50$ gradient — mismatched shapes mean a bug. And its magnitude is the steepness, which is the number practitioners actually watch: collapsing toward zero means learning has stalled, exploding means the next step is enormous and clipping is needed.