Comparing Response, Pearson, Deviance, and Working Residuals

Four Residuals, Four Scales

For a single observation, a residual summarizes how its observed response differs from its fitted value. In a generalized linear model, however, there is more than one useful summary: the difference can be kept in response units, standardized by the model variance, expressed as a likelihood discrepancy, or translated to the linear-predictor scale. These choices answer related but distinct diagnostic questions.

We compare the residuals for a grouped-binomial observation with \(m\) trials and \(y\) observed successes, and for a Poisson observation with count \(y\). Write \(\mu\) for the fitted mean of the response being recorded. For grouped binomial data, let \(p\) be the fitted success probability, so that \(\mu=mp\). For a Poisson observation, \(\mu\) is the fitted count mean. All formulas below are unscaled: no additional dispersion multiplier is applied.

The formulas for these two response types are:

For the binomial deviance, a term with zero numerator is defined to be zero, as given by the limit \(x\log x\to 0\) as \(x\) decreases to zero. The displayed working-residual formulas use the logit link for binomial data and the log link for Poisson data. The working residual is not a single link-free quantity: it depends on the link and on whether the response is represented as a count or a proportion.

What the Scales Mean

The response residual stays in the recorded response's units. For grouped binomial counts, \(y-mp\) is a difference in numbers of successes; if the response is instead the observed proportion \(y/m\), its response residual is \(y/m-p\). For a Poisson count, \(y-\mu\) is a difference in counts. This directness is useful, but raw differences do not account for the fact that variability changes with the mean.

The Pearson residual accounts for that variance change. A grouped-binomial count has variance \(mp(1-p)\), and a Poisson count has variance \(\mu\). Dividing by the model standard deviation therefore puts the discrepancy on a variance-standardized scale. The sign remains the sign of observed minus fitted: positive means more successes or events than fitted, and negative means fewer.

The deviance residual also keeps that sign, but its magnitude comes from the likelihood comparison between the fitted model and a saturated model that fits the observation exactly. It is therefore a likelihood-based measure of discrepancy, not simply a rescaled response difference. Its value is dimensionless and can react differently from the Pearson residual when the observation is far from the fitted mean.

The working residual is on the linear-predictor scale: it measures the adjustment to the fitted linear predictor in the local weighted-least-squares representation used by IRLS. For a log link, it is a relative rather than absolute response discrepancy. For a logit link, the corresponding adjustment is on the log-odds scale. Thus, its size is meaningful in relation to the chosen link, not as a universal measure of response discrepancy.

Exact Links Between the Residuals

For these canonical link choices, the working residual and Pearson residual are related exactly. This relation makes clear why their magnitudes differ: the Pearson residual divides by a standard deviation, whereas the working residual divides by the fitted variance for a count response.

Explicitly, for Poisson data, \(r_W=(y-\mu)/\mu=[(y-\mu)/\sqrt{\mu}]/\sqrt{\mu}=r_P/\sqrt{\mu}\). For grouped binomial data, \(r_W=(y-mp)/[mp(1-p)]=[(y-mp)/\sqrt{mp(1-p)}]/\sqrt{mp(1-p)}\). These equalities also show that a working residual can be numerically larger or smaller than a Pearson residual depending on the fitted variance. \(\square\)

The representation of a grouped-binomial response matters here. If the response is the proportion \(y/m\), the working residual is \((y/m-p)/[p(1-p)]\), as in the theorem's derivation. If the response is the count \(y\), the variance includes the trial count \(m\), and its Pearson residual uses that count variance. Keeping these representations straight prevents an erroneous factor of \(m\).

Why Deviance and Pearson Often Agree Near the Mean

Although their definitions differ, deviance and Pearson residuals have approximately the same magnitude when the observation is close to its fitted mean. This is a local approximation, not an identity for all observations. The derivation below states the conditions and shows where the approximation comes from.

For Poisson data, substitute \(y=\mu+\delta\) into the unit deviance: \(D=2[(\mu+\delta)\log(1+\delta/\mu)-\delta]\). Taylor's theorem gives \(\log(1+x)=x-x^2/2+O(|x|^3)\) as \(x\to0\). Multiplying by \(\mu+\delta\) and subtracting \(\delta\) leaves \(D=\delta^2/\mu+O(|\delta|^3/\mu^2)\). The Pearson residual squared is exactly \(\delta^2/\mu\), proving the stated approximation.

For the grouped binomial, the unit deviance in terms of the observed proportion \(q\) is \(D=2m[q\log(q/p)+(1-q)\log((1-q)/(1-p))]\). With \(p\) fixed in \((0,1)\), let the bracketed function be \(f(q)\). Direct differentiation gives \(f(p)=0\), \(f'(p)=0\), and \(f''(p)=1/[p(1-p)]\). Taylor's theorem around \(p\), with \(q=p+h\), therefore gives \(f(p+h)=h^2/[2p(1-p)]+O(|h|^3)\). Multiplying by \(2m\) yields \(D=mh^2/[p(1-p)]+O(m|h|^3)\). The binomial Pearson residual squared is \(m h^2/[p(1-p)]\). Thus, for either family, the deviance residual's magnitude is locally close to the absolute Pearson residual. The expansion does not claim that their signs or magnitudes are identical for observations far from the fitted mean. \(\square\)

Choosing a Residual for a Diagnostic Question

Each residual has a useful role, but none should be interpreted without its scale. Response residuals preserve the original response units and make the direction and raw size of a discrepancy immediately clear. Pearson residuals provide a variance-standardized comparison across observations with different fitted means. Deviance residuals express each observation's signed contribution through likelihood discrepancy. Working residuals are useful for understanding the local IRLS adjustment on the linear-predictor scale.

For a single observation, these are descriptive quantities, not a complete assessment of model adequacy. A residual's value alone does not account for the rest of the fitted model, nor does it establish whether an observation is unusual under the model. Comparing residuals across observations, or assessing their reference behavior, requires additional context. The purpose here is to recognize what each number measures before using it in a diagnostic summary or plot.

Common Mistakes

• Treating all four residuals as if they shared units. A response residual may be in counts, while a working residual is on the linear-predictor scale. Their numerical magnitudes are not directly interchangeable.
• Using the wrong binomial variance. For a count of successes in \(m\) trials, the variance is \(mp(1-p)\). For the observed proportion, it is \(p(1-p)/m\). Omitting the trial count changes the Pearson residual.
• Assuming the working residual is link-independent. It uses the derivative of the link, so changing the link changes the working-scale adjustment. Changing between count and proportion representations also changes its formula.
• Assuming deviance and Pearson residuals are identical. Their squared values have the same leading term near the fitted mean, but the exact formulas differ, especially for larger discrepancies.
• Dropping the sign of the deviance residual. The deviance itself is nonnegative; the residual receives the sign of observed minus fitted so it retains direction.