Why Standardize a GLM Residual?
A GLM residual measures the difference between an observed response and the value predicted by the fitted model. But raw residuals do not all have the same variability: that depends on the response distribution and the fitted mean. In addition, an observation can help determine its own fitted value. An observation with high leverage may therefore have a smaller raw residual than its underlying discrepancy would suggest.
Standardized GLM residuals address these two issues in sequence. Pearson or deviance residuals account for the response distribution’s variance structure; a leverage adjustment then scales the residual by an estimate of its remaining variability after fitting. This makes unusually large conditional discrepancies easier to screen for. The resulting values are diagnostic measures, not exact standard normal test statistics.
Raw Residuals and the Leverage Adjustment
For a GLM with variance function \(V(\mu_i)\), dispersion parameter \(\phi\), and prior weight \(w_i\), the conditional variance of \(Y_i\) is \(\operatorname{Var}(Y_i\mid x_i)=\phi V(\mu_i)/w_i\). The Pearson residual divides the observed-minus-fitted difference by the estimated conditional standard deviation:
For an ordinary Poisson GLM, \(V(\mu)=\mu\) and \(\phi=1\). For a binomial observation that records \(y_i\) successes in \(m_i\) trials, \(V(\mu_i)=m_i p_i(1-p_i)\), where \(\mu_i=m_ip_i\). In both cases the Pearson residual reflects the response distribution’s variance, rather than using one common denominator for every observation.
The deviance residual uses the observation’s signed contribution to the model deviance. If \(d_i\) is that contribution, then \(r_{Di}=\operatorname{sign}(y_i-\hat{\mu}_i)\sqrt{d_i}\). For a Poisson observation, \(d_i=2[y_i\log(y_i/\hat{\mu}_i)-(y_i-\hat{\mu}_i)]\), with the convention \(0\log(0/\hat{\mu}_i)=0\). For a binomial observation with \(m_i\) trials, its contribution is \(d_i=2[y_i\log(y_i/\hat{\mu}_i)+(m_i-y_i)\log\{(m_i-y_i)/(m_i-\hat{\mu}_i)\}]\), using the same zero-term convention.
The leverage \(h_{ii}\) summarizes how strongly observation \(i\) participates in determining its fitted linear predictor. In the iteratively reweighted least-squares calculation, it is the \(i\)th diagonal element of the weighted hat matrix. For design matrix \(X\) and converged working-weight matrix \(W\), this matrix is \(H=W^{1/2}X(X^\mathsf{T}WX)^{-1}X^\mathsf{T}W^{1/2}\), when the inverse exists. An offset is included in the linear predictor but does not add an estimated coefficient. The standardized residuals divide by \(\sqrt{1-h_{ii}}\), not by \(1-h_{ii}\).
Why the Square-Root Adjustment Appears
The adjustment has a direct justification in the weighted least-squares approximation used to fit a GLM. Near the fitted model, the iteratively reweighted least-squares working response behaves like a linear-model response. In weighted coordinates, the fitted component is its projection through \(H\), so the residual component is projected through \(I-H\). Under the local approximation, the working errors have covariance proportional to the identity matrix. The covariance of the projected residuals is therefore proportional to \((I-H)(I-H)^\mathsf{T}=I-H\), since \(H\) is a symmetric projection matrix in weighted coordinates.
Thus the residual’s approximate standard deviation is multiplied by \(\sqrt{1-h_{ii}}\). Dividing by that quantity puts residuals with different leverage on a more comparable scale. Pearson residuals are connected to this working-residual approximation through the GLM variance function. Deviance residuals are also commonly divided by the same factor as an approximate diagnostic adjustment; their exact finite-sample distribution is not generally standard normal.
Worked Example: Standardizing a Poisson Count Residual
A hypothetical maintenance portfolio is modeled with a Poisson GLM. For one site, the observed number of equipment failures is \(y_i=8\), the fitted mean is \(\hat{\mu}_i=5\), the dispersion is fixed at \(\hat{\phi}=1\), and the fitted leverage is \(h_{ii}=0.20\). Calculate the standardized Pearson and deviance residuals. Use unrounded values during the calculations and round final residuals to four decimal places.
Method. Calculate each raw residual from its distribution-specific formula, then divide by \(\sqrt{1-h_{ii}}\). This avoids comparing the raw discrepancy of 3 directly with discrepancies from observations having different fitted means or leverage.
Pearson residual. The Poisson variance function is \(V(\mu)=\mu\), so \(r_{Pi}=(8-5)/\sqrt{5}=3/\sqrt{5}=1.341640786\ldots\). The leverage adjustment is \(\sqrt{1-0.20}=\sqrt{0.80}=0.894427191\ldots\). Therefore \(r_{Pi}^{\,*}=1.341640786\ldots/0.894427191\ldots=1.5000\).
Deviance residual. The contribution is \(d_i=2[8\log(8/5)-(8-5)]\). Here \(\log(8/5)=\log(1.6)=0.4700036292457356\ldots\), so \(8\log(1.6)=3.760029033965885\ldots\), \(8\log(1.6)-3=0.760029033965885\ldots\), and \(d_i=1.520058067931770\ldots\). The observed count exceeds the fitted mean, so the sign is positive. Thus \(r_{Di}=\sqrt{1.520058067931770\ldots}=1.232906\ldots\), and \(r_{Di}^{\,*}=1.232906\ldots/0.894427191\ldots=1.3784\).
Check and common mistake. The standardized Pearson residual is greater than its raw value because \(h_{ii}>0\) makes \(\sqrt{1-h_{ii}}<1\). Using the unrounded logarithm gives \(d_i\approx1.520058\), not \(1.520064\). Rounding \(\log(1.6)\) too early and then using that rounded value in the deviance calculation produces the discrepancy.
Worked Examples Across Response Types
Worked Example: A Binary Outcome With a High Pearson Residual
A hypothetical insurer models whether a policyholder files a small-claims notification using a Bernoulli logistic GLM. For one policyholder, \(y_i=1\), the fitted probability is \(\hat{p}_i=0.20\), the dispersion is 1, and the leverage is \(h_{ii}=0.36\). Calculate both standardized residuals to four decimal places.
Method. For one Bernoulli trial, \(m_i=1\) and \(\hat{\mu}_i=\hat{p}_i=0.20\). Calculate the Pearson residual using the binomial variance, then calculate the signed square root of the binomial deviance contribution.
Pearson calculation. The fitted variance is \(1(0.20)(0.80)=0.16\), with square root \(0.4\). Hence \(r_{Pi}=(1-0.20)/0.4=0.8/0.4=2.0000\). The leverage factor is \(\sqrt{1-0.36}=\sqrt{0.64}=0.8\), so \(r_{Pi}^{\,*}=2.0000/0.8=2.5000\).
Deviance calculation. Since the observed value is 1, the zero-failure term is zero: \(d_i=2[1\log(1/0.20)+0]=2\log(5)=3.218875824868201\ldots\). The sign is positive, giving \(r_{Di}=\sqrt{3.218875824868201\ldots}=1.794122\ldots\). Therefore \(r_{Di}^{\,*}=1.794122\ldots/0.8=2.2427\).
Check and common mistake. Both adjusted residuals exceed 2 in magnitude, which is a reason to inspect the case in context. Do not treat 2 as a universal pass/fail threshold or call these values exact normal-test statistics. Also do not use a normal variance of \(p(1-p)\) without checking that the response is one Bernoulli trial; grouped binomial data require the number of trials in the variance.
Worked Example: A Zero Count With a Negative Residual
A hypothetical service center’s monthly incident count is modeled by a Poisson GLM with a log-exposure offset. For one center-month, the observed count is \(y_i=0\), the fitted mean count is \(\hat{\mu}_i=2.5\), the dispersion is fixed at 1, and the leverage is \(h_{ii}=0.20\). Find the standardized Pearson and deviance residuals, rounded to four decimal places.
Method. Use the fitted mean count, which already reflects the exposure offset. The offset does not change the residual formulas; it affects the fitted mean.
Pearson calculation. The raw residual is \(r_{Pi}=(0-2.5)/\sqrt{2.5}=-\sqrt{2.5}=-1.581138830\ldots\). Since \(\sqrt{1-0.20}=\sqrt{0.8}=0.894427191\ldots\), \(r_{Pi}^{\,*}=-1.581138830\ldots/0.894427191\ldots=-\sqrt{3.125}=-1.7678\).
Deviance calculation. With \(y_i=0\), the log term is defined as zero, and \(d_i=2[0-(0-2.5)]=5\). The sign is negative because \(0<2.5\), so \(r_{Di}=-\sqrt{5}=-2.236067978\ldots\). The standardized value is \(-2.236067978\ldots/0.894427191\ldots=-\sqrt{6.25}=-2.5000\).
Check and common mistake. Both signs are negative because the observed count is below its fitted mean. The exact standardized deviance value is \(-2.5\), since \(5/0.8=6.25\). A common error is to assign the sign from the fitted value rather than from \(y_i-\hat{\mu}_i\), which would incorrectly make the residual positive.
Interpreting Standardized Values
A standardized residual is useful for ranking observations by the size of their conditional discrepancies. A value near zero indicates that the observed response is close to its fitted mean relative to the estimated variation and leverage. A large positive value indicates a response above the fitted mean; a large negative value indicates one below it. As a rough screening convention, values with absolute magnitude around 2 or more are often examined, with still larger values receiving attention. This is a screening convention, not a significance test.
The adjustment only addresses an approximate leverage effect. It does not correct a wrong link function, an omitted predictor, dependence among observations, an incorrect variance assumption, or an unusual observation. Nor does a large standardized residual alone establish that the observation is influential: residual size and leverage are related but distinct diagnostic properties. Interpret the residual alongside the model specification and the data context.
Check Your Understanding
For each problem, use the stated fitted leverage and round requested residuals to four decimal places. Treat dispersion as 1 unless another value is given.
- A Poisson GLM has \(y_i=6\), \(\hat{\mu}_i=4\), and \(h_{ii}=0.20\). Calculate the standardized Pearson residual. (Answer: \(1.1180\))
- A Poisson GLM has \(y_i=0\), \(\hat{\mu}_i=3\), and \(h_{ii}=0.25\). Calculate the standardized deviance residual, including its sign. (Answer: \(-2.8284\))
- A binomial GLM has \(m_i=10\), \(y_i=7\), fitted probability \(\hat{p}_i=0.50\), and \(h_{ii}=0.16\). Calculate the standardized Pearson residual. (Answer: \(1.3801\))
- A grouped binomial model has \(m_i=20\), \(y_i=5\), fitted probability \(\hat{p}_i=0.10\), and \(h_{ii}=0.25\). Calculate the standardized Pearson residual and state whether its magnitude exceeds 2. (Answer: \(2.5820\); yes)
- A Poisson model gives \(y_i=8\), \(\hat{\mu}_i=5\), and \(h_{ii}=0.20\). Calculate the standardized Pearson and deviance residuals and say whether either exceeds 2 in magnitude. (Answer: \(1.5000\) and \(1.3784\); neither)