Skip Connection
An architectural feature that routes the output of one neural network layer past several subsequent layers to be directly added to a deeper layer.
Think of It Like This
Like an express train track that allows commuters to skip the congested local stops and merge directly into the downtown terminal.
Skip connections are the defining feature of ResNets and Transformers. By providing an alternate, identity-mapping path for gradients to flow backward during training, they completely neutralize the vanishing gradient problem. This innovation allowed researchers to successfully train networks with hundreds, or even thousands, of layers.