Generalized Linear Models

Why Does This Exist?

Helpdesk tickets arrive as counts. Quiet days see one or two, busy days see thirty, one Tuesday saw forty one. Marketing wants the cost of one extra promo email in tickets.

Fit least squares and it answers, then predicts minus 4.8 tickets on a zero promo day. Not zero, not flagged. A number that cannot exist. Spread breaks too: quiet days vary by two tickets while busy days swing by fifteen, but least squares assumes constant spread.

Linear regression predicts a mean directly with bell noise. Logistic regression runs the same linear score through a sigmoid with coin flip noise. Two looks, one machine with two dials. Turn the dials for counts and the negative vanishes structurally. The shared fitting principle is maximum likelihood, and its plain square loss form lives in ordinary least squares.

Think of It Like This

How It Actually Works

Call one day features $x$, weights $w$, intercept $b$. The linear predictor is unchanged from least squares:

$\eta$ is one real number per row. Call $\mu$ the expected tickets that day. The link $g$ ties them:

$g^{-1}$ runs at prediction time and rules out impossible values. With $g = \log$, the inverse is $\mu = e^{\eta}$, positive for every $\eta$ ever. No clipping step, no post fix.

The family sets noise around $\mu$. Gaussian, Bernoulli, Poisson, Gamma all belong to one exponential family, which forces a mean variance rule for free. Fix the family and you fix how spread grows with level.

Three settings you will actually use

Identity link with Gaussian noise gives $\mu = \eta$ and constant variance. Maximising likelihood equals minimising squared error. That is ordinary least squares.

Logit link with Bernoulli noise gives $\mu$ as a probability and variance $\mu(1-\mu)$, peaking at 0.5 and vanishing at the ends. That is logistic regression.

Log link with Poisson noise gives $\mu = e^{\eta}$ and variance equal to mean. That is count regression: tickets per day, clicks per hour, defects per batch. It answers the minus 4.8 problem with one keyword change in most libraries.

Worked example: what one extra email costs

Suppose the fitted Poisson rate is $\mu = e^{0.4 + 1.1 e}$, where $e$ counts promo emails. At zero emails the rate is $e^{0.4} \approx 1.49$, about 1.5 tickets. Each extra email multiplies the rate by $e^{1.1} \approx 3.00$, about triple. At two emails the mean is $1.49 \times 3.00^2 \approx 13.4$ tickets with variance 13.4, while at zero the variance is only 1.5. Spread grows with level by construction. Least squares on the same 500 days extrapolates its straight line to minus 4.8 at zero, because nothing stops its identity link below zero. The diagram shows the score passing through the log inverse link to a positive mean with Poisson spread around it.

When Poisson is not enough

Poisson promises variance equals mean. Real counts break it through clustering: one outage spawns thirty tickets and unmodelled shifts lift the whole rate. That is overdispersion. Coefficients stay roughly right while standard errors come out too small, so effects look significant when they are not.

Check on residuals, not raw counts, since raw variance mostly reflects features working. Conditioned on the fit, variance over mean near 1.0 supports Poisson. Near 2.0 or higher, move to negative binomial, which is Poisson with a second parameter for exactly this slack.

Code

The point is the check: rebuild the rate math with plain arithmetic, no fitting library.

1import math2
3intercept: float = 0.44slope: float = 1.15
6at_zero: float = math.exp(intercept)7per_email: float = math.exp(slope)8at_two: float = math.exp(intercept + slope * 2.0)9print(f"{at_zero:.2f} {per_email:.2f} {at_two:.1f}")10# -> 1.49 3.00 13.411
12coef: float = 0.313print(f"{math.exp(coef):.2f}")14# -> 1.35

Quiet days sit at 1.49 tickets, each email multiplies by 3.00, two emails give 13.4. A log coefficient of 0.30 means $e^{0.30} = 1.35$, so 35 percent more, not 0.30 more tickets.

Watch Out For

The Quick Version

• A GLM has three parts: linear predictor $\eta = w^\top x + b$, link $g$, and noise family.
• The inverse link $g^{-1}$ rules out impossible values structurally, so log link never predicts minus 4.8 tickets.
• Identity plus Gaussian is least squares, logit plus Bernoulli is logistic regression, log plus Poisson is count regression.
• Our ticket model gives 1.49 at zero emails and multiplies by 3.00 per email, reaching 13.4 at two emails.
• Log coefficients multiply: 0.30 means $e^{0.30} = 1.35$, a 35 percent lift.
• Overdispersion leaves coefficients intact but shrinks standard errors, and negative binomial fixes it.

What to Read Next

• Linear Regression
• Logistic Regression
• Ordinary Least Squares