Skip to content
AI360Xpert
Playgrounds

Supervised Learning

Ridge & Lasso Regularization

Regularization adds a penalty for complexity to a model's loss function. Instead of just minimizing the error on the training data, the model must minimize the error plus the penalty, which prevents overfitting.

How a penalty term shrinks model coefficients and differentially selects features (Lasso) vs shrinking them together (Ridge).

Stage 1 of 3: 1. The Coefficients

Penalty type: ridge. Lambda: 0.00. MSE: 0.068.

  • True Signal
  • Noise
  • Data
  • Fitted Curve
Mean Squared Error0.068Mean Squared Error: 0.068
Active Features8Active Features: 8
Bias (Intercept)0.029Bias (Intercept): 0.029

A standard linear regression assigns weights to all available features, including pure noise.

  1. Penalty Math
  2. Coefficient Values
  3. Curve Shape

Check your understanding

1 questions in the bank. Each attempt draws a fresh set in a fresh order, so a second go is a real second go.

Regularization is the mathematical technique of adding a penalty for complexity to a model's loss function. Instead of just minimizing the error on the training data, the model must minimize the error plus the penalty. This prevents overfitting.

Ridge (L2 Penalty)

Ridge regression adds a penalty proportional to the sum of the squared weights (λ∑wi2\lambda \sum w_i^2). This smoothly pulls all weights towards zero, distributing the penalty across correlated features rather than discarding them.

Lasso (L1 Penalty)

Lasso regression adds a penalty proportional to the sum of the absolute weights (λ∑∣wi∣\lambda \sum |w_i|). The geometry of the absolute value function creates sharp corners that force less important weights exactly to zero, effectively acting as an automatic feature selector.

Reference

Ridge (L2)
Penalty = \lambda \sum w_i^2
Lasso (L1)
Penalty = \lambda \sum |w_i|

Break it on purpose

Setting the penalty too high shrinks all weights toward zero, causing the model to predict a constant and underfit the data.