Deviance Residuals in Sparse Data

Why Deviance Residuals Need Careful Reading

A residual for a discrete response cannot vary smoothly around zero in the way a residual for a continuous response might. For a binary observation, for example, the response is either zero or one, so a given fitted probability permits only two possible residual values. With rare events, those values can be very different in magnitude: an observed event may be surprising under a small fitted probability, while a non-event may be quite ordinary. Deviance residuals summarize this discrepancy through a likelihood comparison, but they do not remove the discreteness or make the resulting residuals normally distributed.

In a generalized linear mixed model (GLMM), the distribution used for a residual must also be identified. One common diagnostic uses the response distribution conditional on the random effects, evaluated at fitted random effects and fixed-effect estimates. This is a useful plug-in diagnostic, not a residual from a fully known distribution: the random effects and model parameters have been estimated. A marginal distribution instead averages over the random effects and is generally a mixture. The two choices need not give the same fitted probabilities or residuals.

The saturated model is a reference for measuring lack of fit, not a scientifically plausible model with many parameters to interpret. Under a correctly specified likelihood, \(d_i\) is nonnegative because the saturated likelihood cannot be lower than the fitted likelihood. The square root returns the discrepancy to the scale of a residual, and the sign records whether the observed response is above or below its fitted mean. A residual is therefore not merely a signed difference: its magnitude depends on how plausible the observation is under the fitted distribution.

Common Discrete-Response Formulas

For a Bernoulli response \(Y_i\in\{0,1\}\) with fitted event probability \(\hat p_i\), the deviance contribution is

Terms with a zero multiplier are defined as zero, using the limit \(0\log(0/a)=0\) for \(a>0\). Thus \(y_i=1\) gives \(d_i=2\log(1/\hat p_i)\), while \(y_i=0\) gives \(d_i=2\log\{1/(1-\hat p_i)\}\). The residual sign is positive for an observed event and negative for a non-event.

For a binomial observation \(Y_i\sim\operatorname{Binomial}(m_i,p_i)\), where \(m_i\) is the number of trials and \(\hat\mu_i=m_i\hat p_i\) is the fitted number of successes, the corresponding contribution is

Again, zero-count terms contribute zero. This formula applies when \(0<\hat p_i<1\); boundary fitted probabilities require care because likelihood ratios can be infinite or undefined. A Bernoulli observation is the special case \(m_i=1\).

For a Poisson response \(Y_i\sim\operatorname{Poisson}(\mu_i)\) with fitted mean \(\hat\mu_i>0\),

When \(y_i=0\), the log term is zero and \(d_i=2\hat\mu_i\). For other discrete families, the general likelihood definition still applies: compute twice the difference between the saturated and fitted log likelihoods under that family, then take the signed square root. A formula should not be transferred from the Poisson or binomial distribution to a different response family without checking its likelihood.

Why the Residuals Are Not Normally Distributed

A short likelihood argument explains both the usefulness and the limits of the deviance scale. For a fixed observation, the likelihood ratio comparing the saturated fit with the fitted model is \(\exp\{\ell_{i,\mathrm{sat}}-\ell_i\}\). Taking twice its log gives \(d_i\). Thus a large deviance contribution means that the observation is less well supported by the fitted model than by a model tailored to that observation. The signed square root preserves this ordering in magnitude while retaining the direction of the discrepancy. This construction does not imply a normal sampling distribution for \(r_{D,i}\).

For example, a Bernoulli observation with a fixed fitted probability has only two possible residuals, as the first example showed. A sparse Poisson count also puts much of its probability on zero, with occasional positive counts; its residual distribution can consequently be strongly asymmetric. In a sample with few events, there may be too few distinct observed values to make a histogram or a normal reference line informative. Repeated zeros can produce repeated residual values, while a small number of events can produce a few large positive residuals.

The sign convention is tied to \(y_i-\hat\mu_i\), but the deviance magnitude is tied to the likelihood. Consequently, it is possible for a deviance residual and a Pearson residual to have the same sign but different magnitudes. Pearson residuals scale the response difference by a model-based standard deviation; deviance residuals use a likelihood discrepancy. Neither transformation makes sparse discrete residuals independent standard-normal observations.

Using Deviance Residuals in a GLMM

For an observation-level conditional diagnostic, the analyst must state which fitted distribution is being compared with the observation. In a GLMM, this often means using the response distribution conditional on fitted random effects, with the fixed effects and random effects replaced by their estimates. The resulting \(\hat\mu_i\) or \(\hat p_i\) is a plug-in value. The residual therefore does not incorporate all uncertainty in the estimated random effects, fixed effects, or variance parameters.

A marginal diagnostic instead compares observations with a distribution averaged over random effects. Because that distribution is generally a mixture, its likelihood and residual calculations are not obtained simply by substituting a marginal mean into a conditional deviance formula. A mean alone does not specify the mixture distribution. The conditional and marginal targets answer different questions, so residuals should be labelled and interpreted accordingly.

When using conditional deviance residuals, check that the response family, trial counts or exposure offsets, and fitted means are appropriate for the observations being assessed. A wrong denominator for grouped-binomial data or an omitted exposure adjustment for counts can create apparent lack of fit that is actually a data specification error. Also check for boundary fitted probabilities or means: numerical values near zero or one can yield very large residuals and require stable likelihood calculations.

Useful diagnostic questions include whether a small number of observations dominate the residual magnitudes, whether observed zeros or extreme counts occur more often than the fitted model anticipates, and whether residual signs show a systematic pattern across fitted means or covariate values. Such patterns are prompts to investigate the model and data, not automatic proof of a particular failure. In sparse settings, simulated reference distributions or other diagnostics can be more informative than expecting residuals to look like a normal sample.

Comparison, Limitations, and Common Pitfall

A common mistake is to treat a deviance residual of \(+2\) as having the same universal interpretation as a normal \(z\)-score of \(+2\), or to count residuals outside a fixed normal range as if a normal reference distribution were guaranteed. The correction is to remember how the residual was constructed: it is a signed likelihood discrepancy under a fitted discrete distribution. Its reference behavior depends on that distribution, the fitted values, the sampling structure, and parameter estimation. In particular, a large residual identifies an observation worth investigating; it is not by itself a formal significance test or evidence that a specific model component is missing.

Deviance residuals are also not a substitute for checking the random-effects structure or the dependence assumptions. Conditional residual calculations may hide uncertainty from estimating random effects, while marginal calculations require the appropriate mixture distribution. Sparse data make these issues more consequential because a few observations can carry a large share of the apparent discrepancy.