Outlier Screening With GLM Residuals

Why Screen GLM Residuals?

A generalized linear model (GLM) does not generally have constant response variance. A raw difference between an observed response and its fitted mean can therefore be difficult to interpret: a difference of two may be substantial when the expected variance is small, but ordinary when the expected variance is large. Residual screening addresses this by measuring discrepancies relative to the model’s variance and, when appropriate, accounting approximately for the fact that the model parameters were estimated from the same observations.

A large residual is evidence that an observed response is unusual conditional on the fitted model. It is not, by itself, evidence that the observation strongly changes the fitted coefficients. These are separate questions: residuals measure response discrepancy, whereas coefficient influence concerns how the fitted model changes when an observation’s contribution is altered or removed.

Signed Pearson and Deviance Residuals

Suppose observations are conditionally independent under a correctly specified GLM, with mean \(\mu_i\) and variance \(\operatorname{Var}(Y_i)=\phi V(\mu_i)/w_i\). Here \(V(\mu_i)\) is the variance function, \(\phi\) is the dispersion parameter, and \(w_i\) is a prior weight. The fitted values replace \(\mu_i\) by \(\hat{\mu}_i\). The Pearson residual is

Its sign is the sign of \(y_i-\hat{\mu}_i\): positive means the observed response exceeds its fitted mean, and negative means it falls below that mean. The denominator reflects the response variance expected under the model. In a Poisson model, \(V(\mu)=\mu\) and typically \(\phi=1\); in a binomial model for a count of \(m_i\) successes, the variance is \(m_i p_i(1-p_i)\).

The deviance residual uses the observation’s contribution \(D_i\) to the model deviance:

For example, for a Poisson response with fitted mean \(\hat{\mu}_i\), the contribution is \(D_i=2[y_i\log(y_i/\hat{\mu}_i)-(y_i-\hat{\mu}_i)]\) when \(y_i>0\). When \(y_i=0\), the term \(y_i\log(y_i/\hat{\mu}_i)\) is defined as zero. For a binomial count \(y_i\) out of \(m_i\), fitted probability \(\hat{p}_i\), and fitted mean \(m_i\hat{p}_i\), the contribution is

with zero-count terms interpreted by continuity. Both residual types retain direction and provide a scale for screening, but they need not have identical magnitudes. Pearson residuals use the variance function directly; deviance residuals use the observation’s likelihood-based contribution to deviance.

Why Standardize?

Residuals computed after fitting are not generally independent, unit-variance observations. Fitting the model uses the responses to estimate coefficients, which tends to reduce the residual variance for observations that have more influence on the fitted values. A commonly used approximate standardized residual divides a Pearson residual by \(\sqrt{1-h_{ii}}\), where \(h_{ii}\) is the observation’s diagonal leverage value from the fitted model. With dispersion included in the residual’s scale as above, the same adjustment is applied to a deviance residual as an approximation:

This is an approximate adjustment, not a claim that every standardized GLM residual has an exact standard normal distribution. Discrete responses, small samples, estimated dispersion, and model misspecification can all affect the reference distribution. The adjustment is meaningful when the fitted model and its variance assumptions are suitable; it does not repair a misspecified model.

The basic reason for the adjustment is that the variance of a fitted residual is reduced by fitting. In the familiar linear-model case, the residual vector is obtained by projecting the response away from the fitted subspace. Its \(i\)th residual variance is proportional to \(1-h_{ii}\). Dividing by the square root of that factor restores a more comparable variance scale. GLM standardization uses this idea as an approximation; calculating and interpreting the leverage values is a separate diagnostic task.

Reading Residuals as Screening Evidence

A residual is a diagnostic signal, not a decision rule. Values with large absolute magnitude warrant attention, but no single cutoff is universally valid for every GLM, sample size, response distribution, and diagnostic purpose. As a rough screening convention, analysts often investigate standardized residuals with absolute values around 2 or larger, and give particular attention to values around 3 or larger. These are prompts for review, not formal tests that prove an observation is erroneous.

The sign matters as well as the magnitude. A large positive residual identifies a response larger than expected under the fitted model; a large negative residual identifies one smaller than expected. Comparing Pearson and deviance residuals can add context. A substantial difference between them may reflect the response distribution and the way deviance measures discrepancy, especially for discrete or highly asymmetric outcomes. It is a reason to inspect the observation and model assumptions, not a reason to choose whichever residual gives the preferred conclusion.

Screening should also account for the setting of the analysis. Check that the response was recorded correctly, that exposure and units are represented correctly, and that the observation is being compared with an appropriate fitted mean. An unusual residual can arise from data error, a rare but valid response, an omitted feature of the mean model, or a variance assumption that does not suit the data. The residual alone cannot determine which explanation is correct.

Response Outliers Are Not Automatically Influential

A response outlier is unusual relative to the fitted conditional response distribution. A coefficient-influential observation is one whose presence or contribution materially affects coefficient estimates or fitted conclusions. Large residual magnitude can be one ingredient in influence, but it is not sufficient: the observation’s predictor configuration and its relationship to the rest of the data also matter.

Consequently, residual screening answers a limited question: does this response look unusual given the fitted mean and variance? It does not answer whether the model’s coefficients would change substantially under a case-deletion comparison. That requires a separate influence assessment. Conversely, a moderate residual does not guarantee negligible influence. Keep these conclusions distinct when reporting diagnostics.

Common Mistakes in Residual Screening

• Comparing raw errors across different means. The same response difference can represent very different evidence when the model-based variances differ. Use a residual scaled to the GLM variance.
• Ignoring direction. The sign distinguishes observations above and below their fitted means. Do not report only a magnitude when direction matters.
• Treating an approximate standardized residual as an exact normal score. Discrete responses and model estimation affect its distribution. Screening thresholds are heuristics unless a suitable formal reference distribution has been established.
• Declaring influence from a large residual. A large residual signals response discrepancy. It does not by itself establish a substantial change in fitted coefficients.
• Deleting a flagged observation automatically. First check data quality, model specification, and whether the response is a plausible rare event. Removing a valid observation merely because it is inconvenient can distort the analysis.