Foundations
Linear Regression (OLS)
How Ordinary Least Squares (OLS) balances the pull from every data point to mathematically guarantee the single best-fitting line.
Stage 1 of 4: The Data
Slope is 0.00, intercept is 0.00, MSE is 33.34
- Data
The Mean Squared Error is very high! The line is far away from the data trend.
Check your understanding
1 questions in the bank. Each attempt draws a fresh set in a fresh order, so a second go is a real second go.
Fit a straight line through data points by minimizing the squared errors, using the exact closed-form math.
Linear regression is the foundation of predictive modeling. Given a set of points (like square footage and house price), we want to find the single straight line that best captures the trend.
But how do we define "best"? We define it by looking at the errors: the vertical distance from each data point to our line. If we square these errors (which heavily penalizes points that are far away from the line) and average them, we get the Mean Squared Error (MSE).
The goal of Linear Regression is to find the slope and intercept that make this MSE as small as possible. Because squaring the errors creates a smooth, bowl-shaped mathematical surface, we can actually use calculus to jump directly to the exact answer without guessing. This direct mathematical solution is called Ordinary Least Squares (OLS).
Reference
- Residual
- The vertical distance from a data point to the prediction line.
- MSE
- Mean Squared Error. The average of all squared residuals.
- OLS
- Ordinary Least Squares. The closed-form math that perfectly minimizes MSE.
Break it on purpose
If you pull the line manually to fit just a few points perfectly, the errors on the other points explode. OLS compromises to minimize the total squared error.