Standardized Residuals and GLM Leverage

Why Standardize Pearson Residuals?

A fitted generalized linear model (GLM) does not estimate every observation independently. Each fitted mean is obtained using the model’s coefficients, and an observation can help determine those coefficients. An observation with unusual predictor values, or with a predictor combination represented by few observations, may have substantial influence on its own fitted value. Its residual can therefore have less variability than a residual from an otherwise comparable observation.

Leverage provides a way to account for this difference when examining Pearson residuals. A high leverage value does not by itself mean that an observation is poorly fitted. Instead, it signals that the observation’s fitted value is more affected by the model-fitting process, so its residual should be interpreted with that adjustment in mind.

Definition: For observation $i$, let $y_i$ be the response, $\hat{\mu}_i$ its fitted mean, $V(\hat{\mu}_i)$ the variance function, $w_i$ its prior weight, and $\hat{\phi}$ the dispersion parameter used for the residual calculation. The Pearson residual including dispersion is $$ r_{P,i}=\frac{y_i-\hat{\mu}_i}{\sqrt{\hat{\phi}V(\hat{\mu}_i)/w_i}}. $$ If $h_i$ is the GLM hat value for observation $i$, the leverage-standardized Pearson residual is $$ r_{P,i}^{\,\mathrm{std}}=\frac{r_{P,i}}{\sqrt{1-h_i}}. $$

The hat value $h_i$ is the $i$th diagonal element of the GLM hat matrix formed from the weighted design matrix at the fitted model. It is usually between 0 and 1. For a common GLM with linear predictor $\eta_i=x_i^\mathsf{T}\beta$, link function $g(\mu_i)=\eta_i$, and prior weights $w_i$, the iteratively reweighted least-squares weight is proportional to $$ W_i=\frac{w_i}{V(\hat{\mu}_i)\{g'(\hat{\mu}_i)\}^2}. $$ The corresponding hat matrix is $$ H=W^{1/2}X(X^\mathsf{T}WX)^{-1}X^\mathsf{T}W^{1/2}, $$ where $X$ is the model matrix and $W$ is the diagonal matrix of the working weights. A constant dispersion factor does not change the hat values. Any offset is fixed in the linear predictor; it is not an additional estimated column of $X$.

Why the Adjustment Uses the Square Root

The factor $\sqrt{1-h_i}$ comes from the approximate variance of a residual after fitting the model. In weighted linear least squares, the fitted values are projections onto the columns of the design matrix. The residual-making matrix is $I-H$, where $I$ is the identity matrix and $H$ is the hat matrix. Since $H$ is symmetric and idempotent, the covariance matrix of the standardized residual vector is proportional to $$ (I-H)(I-H)^\mathsf{T}=I-H. $$ Its $i$th diagonal element is $1-h_i$. Thus, after accounting for model fitting, the variance of the $i$th standardized residual is approximately $1-h_i$. Dividing by its standard deviation, $\sqrt{1-h_i}$, puts it on a more comparable scale with residuals from observations having different leverage.

A GLM is not generally a linear model on the response scale. The same reasoning applies as a local approximation through the weighted least-squares step used to fit the GLM. Consequently, the standardized Pearson residual is a diagnostic approximation, not a guarantee that every resulting residual has an exact standard normal distribution. The approximation may be especially limited for small samples, highly discrete responses, or fitted means near a boundary.

Important convention: Some sources define an unscaled Pearson residual $r^0_{P,i}=(y_i-\hat{\mu}_i)\sqrt{w_i/V(\hat{\mu}_i)}$, without $\hat{\phi}$. Under that convention, the corresponding standardized residual is $r^0_{P,i}/\sqrt{\hat{\phi}(1-h_i)}$. Do not include dispersion in the raw residual and then include it again in the adjustment.

Calculating Standardized Pearson Residuals

For each observation, first calculate the fitted variance on the response scale, including dispersion and any prior weight. Next calculate the Pearson residual, then divide it by $\sqrt{1-h_i}$. Keep full precision in intermediate calculations and round only the final residual. When $\hat{\phi}=1$, as is standard for Poisson and Bernoulli GLMs, the dispersion term drops out.

Worked Example: A Poisson Claim Count With Moderate Leverage

A hypothetical Poisson frequency model has unit prior weights and dispersion fixed at $\hat{\phi}=1$. For one policy, the observed claim count is $y_i=9$, the fitted mean is $\hat{\mu}_i=4$, and the hat value is $h_i=0.36$. Calculate the Pearson and leverage-standardized Pearson residuals. State whether the observation remains notable after adjustment.

Method. For a Poisson response, $V(\mu)=\mu$. The Pearson residual is therefore the response difference divided by the square root of the fitted mean: $$ r_{P,i}=\frac{9-4}{\sqrt{1(4)/1}}=\frac{5}{2}=2.5. $$

The residual adjustment is $$ \sqrt{1-h_i}=\sqrt{1-0.36}=\sqrt{0.64}=0.8. $$ Thus $$ r_{P,i}^{\,\mathrm{std}}=\frac{2.5}{0.8}=3.1250. $$ The observation has a positive residual, and the adjustment makes its magnitude larger than the unadjusted value of $2.5$. A standardized value of $3.1250$ merits investigation, but it does not by itself establish the cause of the discrepancy.

Check. The fitted variance is $4$, so the unadjusted residual is $5/\sqrt{4}=2.5$. Since $h_i=0.36$, the residual standard deviation factor is $0.8$, and $2.5/0.8=3.1250$. The arithmetic is consistent.

Common mistake avoided: Multiplying the residual by $1-h_i$ would give the wrong adjustment. The variance factor is $1-h_i$, so the standard deviation factor is its square root.

Worked Example: Standardizing a Bernoulli Residual

A hypothetical Bernoulli GLM estimates the probability of a roadside-assistance add-on purchase. For one applicant, the observed response is $y_i=0$, the fitted purchase probability is $\hat{\mu}_i=0.64$, the dispersion is $\hat{\phi}=1$, the prior weight is $w_i=1$, and $h_i=0.51$. Calculate the standardized Pearson residual and interpret its sign.

Method. For a Bernoulli response, $V(\mu)=\mu(1-\mu)$. The fitted variance function is $$ V(0.64)=0.64(1-0.64)=0.64(0.36)=0.2304. $$ Its square root is $\sqrt{0.2304}=0.48$. The Pearson residual is $$ r_{P,i}=\frac{0-0.64}{\sqrt{1(0.2304)/1}}=\frac{-0.64}{0.48}=-1.333333\ldots. $$

The leverage adjustment is $$ \sqrt{1-h_i}=\sqrt{1-0.51}=\sqrt{0.49}=0.7. $$ Using the unrounded Pearson residual, $$ r_{P,i}^{\,\mathrm{std}}=\frac{-0.64/\sqrt{0.64(0.36)}}{0.7} =-1.9047619\ldots $$ Therefore, to four decimal places, the standardized Pearson residual is $-1.9048$. Its negative sign means the observed response, zero, is below the fitted mean of $0.64$; the leverage adjustment increases the magnitude.

Check. The variance-function square root is $0.48$, so the raw residual is $-0.64/0.48=-1.333333\ldots$. The adjustment divisor is $0.7$, giving $-1.9047619\ldots$, which rounds to $-1.9048$ to four decimal places.

Common mistake avoided: The Bernoulli residual is not just $y_i-\hat{\mu}_i$. It must be divided by $\sqrt{\hat{\mu}_i(1-\hat{\mu}_i)}$ before the leverage adjustment. In particular, $-0.64/0.48/0.7$ is about $-1.9048$, not $-1.3197$.

Worked Example: Including Dispersion and a Prior Weight

A hypothetical continuous loss-cost response is fitted with a normal GLM and identity link. For one exposure group, the observed response is $y_i=12$, the fitted mean is $\hat{\mu}_i=10$, the dispersion is $\hat{\phi}=4$, the prior weight is $w_i=2$, and the hat value is $h_i=0.75$. Calculate the standardized Pearson residual.

Method. For the normal identity-link model, $V(\mu)=1$. The response variance used for this weighted observation is $$ \frac{\hat{\phi}V(\hat{\mu}_i)}{w_i}=\frac{4(1)}{2}=2. $$ The raw Pearson residual is $$ r_{P,i}=\frac{12-10}{\sqrt{2}}=\frac{2}{\sqrt{2}}=\sqrt{2}=1.4142136\ldots. $$

The leverage factor is $$ \sqrt{1-h_i}=\sqrt{1-0.75}=\sqrt{0.25}=0.5. $$ Therefore, $$ r_{P,i}^{\,\mathrm{std}}=\frac{\sqrt{2}}{0.5}=2\sqrt{2}=2.8284271\ldots. $$ To four decimal places, the standardized residual is $2.8284$. It is positive because the observed response exceeds its fitted mean.

Check. The observation-level variance is $2$, so the raw residual is $2/\sqrt{2}$. Dividing by $0.5$ doubles that value, giving $2.8284271\ldots$, which rounds to $2.8284$.

Common mistake avoided: The prior weight is in the denominator of the observation-level variance. Using $\hat{\phi}w_i$ would incorrectly make this variance $8$ instead of $2$.

Interpreting Leverage and Residual Size Together

The standardized residual answers a limited but useful question: relative to its approximate residual variability after fitting, how far is the observation from its fitted mean? It does not answer why the residual is large. The response may be unusual, the mean model may be inadequate, or the observation may reflect data quality or an unmodeled feature. A high hat value is a property of the design and fitting weights, not direct evidence that the response is incorrect.

For the same raw Pearson residual, an observation with greater leverage has a smaller $\sqrt{1-h_i}$ and therefore a larger standardized residual. This adjustment can reveal an observation that looks less unusual before accounting for its reduced residual variance. However, when $h_i$ is extremely close to 1, the divisor approaches zero. The resulting standardized residual may be unstable, and the model may have little information independent of that observation for estimating its fitted mean.

A useful diagnostic comparison considers both the unadjusted Pearson residual and $h_i$, rather than relying on a single number. A large standardized residual identifies a case for follow-up; it is not a universal pass/fail test, nor does it prove that a model is misspecified. Any informal threshold should be considered in light of the response distribution, sample size, and the number of observations being reviewed.

Calculate the response-scale variance.
Use $\hat{\phi}V(\hat{\mu}_i)/w_i$, ensuring that the distribution, dispersion, and prior-weight convention match the model.

Find the Pearson residual.
Subtract the fitted mean from the response and divide by the square root of the response-scale variance.

Apply the leverage adjustment.
Divide the Pearson residual by $\sqrt{1-h_i}$, not by $1-h_i$.

Interpret the result in context.
Check its sign, magnitude, and hat value; treat a large value as a diagnostic signal to investigate rather than a conclusive verdict.

Common Exam Traps

Omitting the square root. The approximate residual variance factor is $1-h_i$, but standardization divides by the standard deviation $\sqrt{1-h_i}$.
Using the wrong response variance. For Bernoulli data, use $\hat{\mu}_i(1-\hat{\mu}_i)$; for Poisson counts, use $\hat{\mu}_i$. Include dispersion and prior weights where specified.
Mixing scaled and unscaled residual conventions. If the Pearson residual already includes $\hat{\phi}$, do not divide by $\sqrt{\hat{\phi}}$ a second time.
Rounding too early. Retain the unrounded Pearson residual and square roots until the final reported value. Premature rounding can change the last reported decimal.
Calling every high-leverage point an outlier. Leverage describes the design’s potential to affect its own fitted value; residual size describes discrepancy from that fitted value. They are related diagnostically but are not interchangeable.

Key takeaway: Under the usual GLM approximation, fitting reduces the variance of a Pearson residual by a factor of $1-h_i$. The standardized Pearson residual is therefore the Pearson residual divided by $\sqrt{1-h_i}$, with dispersion and prior weights handled consistently in the initial residual.

Check Your Understanding

For each problem, use the specified GLM variance and report the final standardized Pearson residual to four decimal places unless the answer is exact.

A hypothetical Poisson claim-count observation has $y_i=3$, $\hat{\mu}_i=7$, $h_i=0.19$, unit prior weight, and $\hat{\phi}=1$. Calculate its standardized Pearson residual. (Answer: $-1.6798$)
A hypothetical normal identity-link GLM has $y_i=15$, $\hat{\mu}_i=12$, $\hat{\phi}=9$, $w_i=1$, and $h_i=0.36$. Calculate its standardized Pearson residual. (Answer: $1.2500$)
A hypothetical Bernoulli observation has $y_i=1$, $\hat{\mu}_i=0.25$, $h_i=0.20$, unit prior weight, and $\hat{\phi}=1$. Calculate its standardized Pearson residual. (Answer: $1.9365$)
A hypothetical Poisson count has $y_i=0$, $\hat{\mu}_i=2.25$, $h_i=0.64$, unit prior weight, and $\hat{\phi}=1$. Calculate the Pearson residual and the standardized Pearson residual. (Answer: $-1.5000$ and $-2.5000$)
Two hypothetical Poisson observations have unit prior weights and $\hat{\phi}=1$. Observation A has $y_A=16$, $\hat{\mu}_A=9$, and $h_A=0.19$; observation B has $y_B=7$, $\hat{\mu}_B=4$, and $h_B=0.64$. Calculate both standardized Pearson residuals and identify which has the larger absolute value. (Answer: A $=2.5926$, B $=2.5000$; A has the larger absolute value.)

What You'll Learn