Cook's Distance for Generalized Linear Models

Why Measure Influence in a GLM?

A generalized linear model (GLM) can fit most observations well while still depending heavily on one case. A case may stand out because its response is unusual given its fitted mean, because its predictor values give it high leverage, or because both features occur together. Cook’s distance combines these considerations to screen for observations that may substantially affect the fitted results if removed.

Cook’s distance is an influence diagnostic, not a test that labels an observation as erroneous. A large value calls for investigation: check the data, the model specification, and the observation’s context. It does not by itself establish that the case should be excluded. The treatment here is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.

Definition: GLM Cook’s distance is a scaled measure of how much a fitted model changes when one observation is omitted. A commonly used first-order approximation is $$ D_i \approx \frac{r_{P,i}^{\,2}h_{ii}}{p(1-h_{ii})^2}, $$ where $r_{P,i}$ is the scaled Pearson residual, $h_{ii}$ is the diagonal element of the GLM weighted hat matrix, and $p$ is the number of fitted regression coefficients (the design-matrix rank). This is an approximation, not an exact identity for every GLM.

The Quantities in the Approximation

Suppose the conditional distribution of response $Y_i$, given predictors, has mean $\mu_i$ and variance $\phi V(\mu_i)/w_i$. Here $\phi$ is the dispersion parameter, $V(\mu_i)$ is the variance function, and $w_i$ is a prior weight. For an unweighted observation, $w_i=1$. The scaled Pearson residual is

r_{P,i}=\frac{y_i-\hat{\mu}_i}{\sqrt{\hat{\phi}V(\hat{\mu}_i)/w_i}}.

The residual’s sign indicates whether the observed response is above or below its fitted mean. Cook’s distance uses its square, so the sign does not matter to the influence measure. The squared residual reflects response discrepancy relative to the model’s variance, not merely the raw difference $y_i-\hat{\mu}_i$.

The GLM weighted hat matrix is formed from the design matrix and the final iteratively reweighted least-squares weights. If $X$ is the design matrix and $W$ is the diagonal matrix of those fitting weights, one common expression is

H=W^{1/2}X(X^\mathsf{T}WX)^{-1}X^\mathsf{T}W^{1/2}, \qquad h_{ii}=H_{ii}.

This expression assumes the coefficient information matrix is nonsingular; when the design is rank deficient, it must be interpreted using the estimable rank and an appropriate generalized inverse. The value $h_{ii}$ measures the observation’s leverage in the weighted fit. The quantity $p$ counts fitted coefficients, including an intercept when one is present, not the number of observations.

Why Residual Size and Leverage Appear Together

The approximation comes from viewing the converged GLM fit locally as a weighted least-squares problem. In that local problem, deleting a case changes the fitted coefficient vector more when the case has a large residual. The change is also amplified when the case has high leverage: the factor $1/(1-h_{ii})$ grows as $h_{ii}$ approaches one. Cook’s distance measures the resulting coefficient displacement in the model’s weighted information scale and divides by the number of fitted coefficients.

Approximate the fitted GLM locally.
At convergence, IRLS represents the fit by a weighted least-squares problem for a working response, with design matrix $X$ and weights $W$.

Consider the one-case deletion effect.
In weighted least squares, the approximate coefficient change from deleting case $i$ is proportional to $(X^\mathsf{T}WX)^{-1}x_i$ times that case’s weighted residual, divided by $1-h_{ii}$. Thus a larger residual or leverage can increase the change.

Measure the change on the fitted-model scale.
Taking the squared coefficient change in the information-matrix metric introduces a factor proportional to $h_{ii}$, the squared residual, and $1/(1-h_{ii})^2$. Scaling by $p$ gives the displayed Cook’s distance approximation.

For canonical-link GLMs, the local weighted working residual has a direct relationship to the Pearson residual at the fitted values. More generally, the displayed Pearson-residual formula is a diagnostic approximation; it should not be treated as an exact case-deletion calculation for every link, variance function, or software convention. The underlying reason for the formula is the local weighted-least-squares approximation, not a claim that all GLMs are globally linear.

Calculating and Interpreting Cook’s Distance

For a case with known $r_{P,i}$, $h_{ii}$, and $p$, substitute directly into the formula. A large residual with low leverage may have modest influence; high leverage with a small residual may also have modest influence. The combination can produce a large Cook’s distance. Since the denominator shrinks as leverage approaches one, even a moderate residual can matter when $h_{ii}$ is very high.

There is no universal cutoff that decides whether a case is influential in every GLM application. Values around or above one are sometimes used as a rough screening signal, but interpretation depends on sample size, model dimension, the distribution of the diagnostic values, and the purpose of the model. Compare cases with one another, inspect the residual and leverage separately, and determine whether conclusions or predictions are sensitive to the observation.

Worked Example: A Large Poisson Residual

A hypothetical Poisson GLM has fixed dispersion $\phi=1$. For one observation, $y_i=5$, $\hat{\mu}_i=1$, prior weight $w_i=1$, leverage $h_{ii}=0.25$, and $p=2$. Calculate its scaled Pearson residual and approximate Cook’s distance, then interpret the result.

Method and residual. For a Poisson model, $V(\mu)=\mu$. Therefore, $$ r_{P,i}=\frac{5-1}{\sqrt{1\cdot 1/1}} =\frac{4}{1}=4.0000. $$ The observation is four scaled residual units above its fitted mean.

Cook’s distance. Substitute $r_{P,i}=4$, $h_{ii}=0.25$, and $p=2$: $$ D_i\approx\frac{4^2(0.25)}{2(1-0.25)^2} =\frac{16(0.25)}{2(0.75)^2} =\frac{4}{1.125} =3.5556. $$

Interpretation and check. This is a substantial screening signal: the case combines a large residual with nontrivial leverage. The numerator is $4$, and the denominator is $1.125$, so the result must exceed $3$. The common mistake is to lose a factor of four when squaring or substituting the residual; using $0.8889$ would not match the stated formula.

Worked Example: A Binary Response With Moderate Leverage

In a hypothetical logistic GLM, one independent Bernoulli response is $y_i=1$, its fitted success probability is $\hat{\mu}_i=0.25$, its prior weight is $1$, its leverage is $0.20$, and the model has $p=2$ coefficients. Calculate the scaled Pearson residual and approximate Cook’s distance.

Method and residual. For a Bernoulli response, $V(\mu)=\mu(1-\mu)$, and the fixed dispersion is $\phi=1$. Thus $$ r_{P,i}=\frac{1-0.25}{\sqrt{0.25(1-0.25)}} =\frac{0.75}{\sqrt{0.1875}} =\frac{0.75}{0.4330127} =1.7321. $$ Its square is $1.7321^2\approx3.0000$.

Cook’s distance. $$ D_i\approx\frac{3(0.20)}{2(1-0.20)^2} =\frac{0.60}{2(0.80)^2} =\frac{0.60}{1.28} =0.46875 \approx0.4688. $$

Interpretation and check. The response is above its fitted probability, but the residual and leverage combine to give a more moderate Cook’s distance than in the Poisson example. The common mistake is to use the raw difference $1-0.25=0.75$ as the Pearson residual; it must be divided by the square root of the Bernoulli variance. Rounding only at the final step, the distance is $0.4688$ to four decimal places.

Worked Example: A Gamma Response With High Leverage

A hypothetical Gamma GLM uses variance function $V(\mu)=\mu^2$, dispersion $\phi=0.5$, and prior weight $w_i=2$. For one case, $y_i=12$, $\hat{\mu}_i=8$, $h_{ii}=0.60$, and $p=3$. Find its scaled Pearson residual and approximate Cook’s distance.

Method and residual. The variance used in the Pearson residual is $$ \frac{\phi V(\hat{\mu}_i)}{w_i} =\frac{0.5(8^2)}{2} =\frac{32}{2} =16. $$ Its square root is $4$, so $$ r_{P,i}=\frac{12-8}{4}=1.0000. $$

Cook’s distance. $$ D_i\approx\frac{1^2(0.60)}{3(1-0.60)^2} =\frac{0.60}{3(0.40)^2} =\frac{0.60}{0.48} =1.2500. $$

Interpretation and check. The residual is only one scaled unit, but leverage is high, so the case still merits investigation. The denominator is $0.48$, smaller than the numerator $0.60$, which verifies that the distance exceeds one. The common mistake is to treat $w_i=2$ as multiplying the variance; under the stated variance convention, the prior weight divides it.

Common Mistakes and Practical Use

Confusing outlying response with influence. A large residual signals an unusual response relative to the fitted mean; it does not automatically imply a large Cook’s distance. Leverage and the model dimension also matter.
Using raw rather than scaled residuals. The formula uses $r_{P,i}$, which accounts for the GLM variance, dispersion, and prior weight. A raw difference is not interchangeable with it.
Omitting the squared leverage adjustment. The denominator is $p(1-h_{ii})^2$, not $p(1-h_{ii})$. Leaving out the square can materially change the result.
Using the number of observations for $p$. Use the number of fitted coefficients, or the design-matrix rank when coefficients are not all independently estimable.
Treating a screening value as a decision rule. Cook’s distance does not establish that a case is wrong or should be removed. Review the data and model, and consider whether substantive conclusions change.

A very high leverage value also deserves caution because $1-h_{ii}$ becomes small and the approximation can become unstable. If the GLM has serious overdispersion, a poorly specified mean structure, or convergence problems, the diagnostic should be interpreted in that context rather than in isolation. Cook’s distance helps prioritize review; it does not repair a misspecified model or replace subject-matter judgment.

Key takeaway: GLM Cook’s distance is approximately the squared scaled Pearson residual multiplied by leverage, divided by the model dimension and $(1-h_{ii})^2$. Read it alongside the residual and leverage, and treat a large value as a prompt for investigation rather than an automatic exclusion.

Check Your Understanding

For each case, use the stated GLM variance and the approximation $D_i\approx r_{P,i}^{\,2}h_{ii}/[p(1-h_{ii})^2]$. Give Cook’s distance to four decimal places where needed.

A Poisson GLM has $\phi=1$, $w_i=1$, $y_i=3$, $\hat{\mu}_i=1$, $h_{ii}=0.20$, and $p=2$. Find $r_{P,i}$ and $D_i$. (Answer: $r_{P,i}=2.0000,\ D_i=0.6250$)
A Bernoulli GLM has $\phi=1$, $w_i=1$, $y_i=0$, $\hat{\mu}_i=0.20$, $h_{ii}=0.10$, and $p=2$. Find $r_{P,i}$ and $D_i$. (Answer: $r_{P,i}=-0.5000,\ D_i=0.0154$)
A normal identity-link GLM has $V(\mu)=1$, $\phi=4$, $w_i=1$, $y_i=14$, $\hat{\mu}_i=10$, $h_{ii}=0.40$, and $p=2$. Find $r_{P,i}$ and $D_i$. (Answer: $r_{P,i}=2.0000,\ D_i=2.2222$)
A Gamma GLM has $V(\mu)=\mu^2$, $\phi=0.25$, $w_i=1$, $y_i=15$, $\hat{\mu}_i=10$, $h_{ii}=0.50$, and $p=4$. Find $r_{P,i}$ and $D_i$. (Answer: $r_{P,i}=1.0000,\ D_i=0.5000$)
Two observations in the same model have $p=3$. Case A has $r_{P}=1.5$ and $h=0.20$; case B has $r_{P}=3$ and $h=0.05$. Calculate both Cook’s distances and identify which is larger. (Answer: $D_A=0.2344,\ D_B=0.1662$; case A has the larger Cook’s distance despite its smaller residual.)

What You'll Learn