Learning Rate
A setting that controls how big a step the model takes each time it updates its weights during training. Too big and it overshoots the target; too small and training crawls.
Think of It Like This
Your step size walking downhill in fog — big strides are fast but risk flying past the bottom.
After each batch, the optimizer moves every weight a little way down its gradient, and the learning rate scales exactly how far. Set it too high and the loss bounces around or blows up; set it too low and training takes forever to converge. It's usually the single most important hyperparameter to tune, and most training runs schedule it to shrink over time.
def sgd_step(weight: float, grad: float, lr: float) -> float: """One gradient-descent update; lr sets how far downhill we move.""" return weight - lr * grad