Why the Residual Scale Matters
A raw response residual is the observed response minus its fitted mean. It is simple to interpret in the response variable’s units, but its magnitude does not have a universal meaning across observations. In many generalized linear models (GLMs), the response variance depends on the mean. A raw difference of five may be unusual for one fitted mean and entirely ordinary for another.
This matters when assessing model fit. A plot or comparison based only on raw residuals can give the largest visual weight to observations with large expected responses, even when their errors are modest relative to their variability. Discrete outcomes create another difficulty: binary responses, for example, can produce only two raw residual values for a given fitted probability. Pearson and deviance residuals help put observations on scales that account for the model and fitted mean, though neither is a complete diagnostic by itself.
What the Three Residuals Measure
A GLM specifies a response distribution and a mean structure. For independent responses, its variance is commonly written \(\operatorname{Var}(Y_i\mid x_i)=\phi V(\mu_i)\), where \(x_i\) denotes the predictors, \(\mu_i=E[Y_i\mid x_i]\), \(V\) is the variance function, and \(\phi\) is the dispersion parameter. Residual calculations use the fitted mean \(\hat{\mu}_i\), and usually an estimated or specified dispersion.
The raw residual retains the original response units. For a count, it is a count difference; for a severity, it is a monetary difference. But it does not divide by the expected scale of random variation. The Pearson residual does: its denominator is the fitted standard deviation under the model. When the assumed variance function is appropriate, this makes residual magnitudes more comparable across different fitted means.
The deviance residual instead measures a signed contribution to the likelihood-based deviance. It compares the fitted model with a saturated model, which fits each observation exactly. It is not generally identical to a Pearson residual: it uses the likelihood’s shape, so a large discrepancy can be treated differently depending on the response distribution and the direction of the discrepancy.
Worked Example: Poisson Counts at Different Fitted Means
Worked Example: Poisson Counts at Different Fitted Means
A hypothetical model predicts event counts for two units. Assume independent Poisson responses with means \(\mu_1=1\) and \(\mu_2=20\), and known fitted means \(\hat{\mu}_1=1\) and \(\hat{\mu}_2=20\). The observed counts are \(y_1=3\) and \(y_2=24\). Calculate all three residuals and compare their interpretation. Use \(\phi=1\), as in the standard Poisson model.
Method. For a Poisson response, \(V(\mu)=\mu\). Thus the Pearson residual divides the raw residual by \(\sqrt{\hat{\mu}}\). The Poisson deviance contribution is \(d_i=2[y_i\log(y_i/\hat{\mu}_i)-(y_i-\hat{\mu}_i)]\), with \(0\log 0\) interpreted as zero.
For the first unit, the raw residual is \(e_1=3-1=2\). Its Pearson residual is \(r_{P1}=2/\sqrt{1}=2.0000\). For its deviance contribution, \(\log(3/1)=\log 3=1.098612289\), so
For the second unit, the raw residual is \(e_2=24-20=4\). Its Pearson residual is \(r_{P2}=4/\sqrt{20}=0.8944\). Since \(\log(24/20)=\log(1.2)=0.182321557\),
Check. Both observations exceed their fitted means, so all three residuals are positive. The second raw residual, 4, is larger than the first, 2. Yet its Pearson and deviance residuals are smaller: its discrepancy is modest relative to the larger count scale. Values above are rounded to four decimals only after calculating from unrounded logarithms.
Common mistake avoided: Ranking the observations’ lack of fit by the absolute raw difference alone would label the second unit as more unusual. The model-based residuals show why that conclusion is not warranted: Poisson variability increases with the mean.
Why the Poisson Deviance Formula Holds
For a Poisson observation \(y\), the log-likelihood at mean \(\mu\), omitting the term \(-\log(y!)\) that is identical for both models, is \(y\log\mu-\mu\). The saturated model uses mean \(y\) when \(y>0\), giving log-likelihood \(y\log y-y\). Twice the fitted model’s loss in log-likelihood relative to that saturated fit is therefore
That is the Poisson deviance contribution. Taking its square root gives a residual magnitude, and assigning the sign of \(y-\hat{\mu}\) preserves whether the observation is above or below its fitted mean. When \(y=0\), the limiting value of \(y\log(y/\mu)\) is zero, so \(d=2\hat{\mu}\).
Worked Example: Binary Responses Have Few Raw Values
Worked Example: Binary Responses Have Few Raw Values
A hypothetical screening model assigns probability \(p=0.1\) to a positive outcome for one person and probability \(p=0.9\) to a positive outcome for another. Suppose the first outcome is \(y=1\), while the second is \(y=0\). Treat each response as Bernoulli with its stated probability and use \(\phi=1\). Calculate the raw, Pearson, and deviance residuals.
For a Bernoulli response, \(\hat{\mu}=p\) and \(V(p)=p(1-p)\). For the first person, \(e=1-0.1=0.9\), and the Pearson residual is \(0.9/\sqrt{0.1(0.9)}=0.9/0.3=3.0000\). Its deviance contribution is
For the second person, \(e=0-0.9=-0.9\), and the Pearson residual is \(-0.9/\sqrt{0.9(0.1)}=-3.0000\). The observed outcome is zero, so the deviance contribution is \(2\log[1/(1-0.9)]=2\log(10)=4.605170186\). The deviance residual is \(-2.1460\).
Check. The raw residuals have equal absolute value, 0.9, but both outcomes conflict strongly with their respective fitted probabilities. The Pearson and deviance residuals reflect that surprise and retain the direction of the discrepancy. The calculations also show that binary raw residuals are constrained: for fitted probability \(p\), the only possibilities are \(1-p\) and \(-p\).
Common mistake avoided: Treating equal raw magnitudes as evidence of equal fit without considering the fitted probabilities. A binary outcome of 1 is much less expected when \(p=0.1\) than when \(p=0.5\), even though raw residuals alone do not express that distinction consistently.
Mean-Dependent Variance Beyond Counts
The issue is not limited to discrete responses. Consider a Gamma GLM for a positive response with \(\operatorname{Var}(Y_i\mid x_i)=\phi\mu_i^2\). This variance function makes the standard deviation proportional to the mean. Consequently, raw residuals should be interpreted in relation to the response scale.
Worked Example: Equal Relative Errors in a Gamma Model
Suppose hypothetical positive severities follow Gamma distributions with shape \(4\) and scale \(\phi\mu_i\), where \(\phi=0.25\). A Gamma distribution with shape \(4\) and scale \(0.25\mu_i\) has mean \(\mu_i\) and variance \(4(0.25\mu_i)^2=0.25\mu_i^2\). Two fitted means are \(\hat{\mu}_1=10\) and \(\hat{\mu}_2=100\), and the observed values are \(y_1=12\) and \(y_2=120\). Compare raw and Pearson residuals.
For the first severity, \(e_1=12-10=2\). Its fitted variance is \(0.25(10)^2=25\), so its fitted standard deviation is \(5\), and \(r_{P1}=2/5=0.4000\). For the second, \(e_2=120-100=20\). Its fitted variance is \(0.25(100)^2=2{,}500\), its fitted standard deviation is \(50\), and \(r_{P2}=20/50=0.4000\).
Check. The second raw residual is ten times larger, but each observation is 20% above its fitted mean and has the same Pearson residual. The raw values are in severity units; the Pearson values express discrepancy relative to the assumed standard deviation. This comparison depends on the stated Gamma variance model and should not be transferred to a model with a different variance function.
Common mistake avoided: Interpreting a larger absolute error as automatically more concerning. Here the mean and standard deviation both increase by a factor of ten, leaving the standardized discrepancy unchanged.
Choosing and Interpreting a Residual Scale
Each residual scale answers a different question. Raw residuals are useful when the size and direction of an error in original units matter. Pearson residuals ask how large the discrepancy is relative to the fitted standard deviation. Deviance residuals express a signed, likelihood-based contribution to model deviance. A good diagnostic interpretation keeps the response distribution and variance assumption in view.
| Residual | Scale or construction | Useful interpretation | Limitation |
|---|---|---|---|
| Raw response | \(y_i-\hat{\mu}_i\) | Error in the response’s original units | Magnitude is not adjusted for mean-dependent variance |
| Pearson | Raw residual divided by fitted standard deviation | Discrepancy relative to the model’s variance | Depends on the assumed variance function and dispersion |
| Deviance | Signed square root of deviance contribution | Signed likelihood-based discrepancy from a saturated fit | May not equal Pearson scaling; its behavior depends on the response distribution |
A residual is not a direct probability that an observation is wrong, nor is a small value proof that the model is correct. Residual calculations use fitted means, and the fitting process itself affects their behavior. In particular, a familiar-looking raw residual plot can be misleading if the model predicts different response scales for different observations or if the response can take only a few values.
The practical rule is to start with the response distribution and its variance function. Then decide whether original-unit error, variance-scaled discrepancy, or likelihood-based discrepancy addresses the question. Comparing different observations using raw residual magnitudes alone is especially risky when their fitted means differ substantially.
Check Your Understanding
For each item, calculate the requested residuals from the stated model. Round numerical residuals to four decimals.
- A Poisson response has fitted mean \(\hat{\mu}=2\) and observed count \(y=5\). Find its raw, Pearson, and deviance residuals. (Answer: \(e=3\), \(r_P=2.1213\), \(r_D=1.7785\))
- A Bernoulli response has fitted probability \(p=0.2\) and observed outcome \(y=0\). Find its raw, Pearson, and deviance residuals. (Answer: \(e=-0.2\), \(r_P=-0.5000\), \(r_D=-0.6680\))
- Two independent Poisson observations have fitted means \(4\) and \(25\), with observed counts \(6\) and \(30\), respectively. Compare their raw residuals and calculate their Pearson residuals. (Answer: raw residuals \(2\) and \(5\); Pearson residuals \(1.0000\) and \(1.0000\))
- In a Gamma model with \(\operatorname{Var}(Y_i\mid x_i)=0.16\mu_i^2\), fitted means are \(20\) and \(80\), and observed values are \(24\) and \(96\). Find the raw and Pearson residuals. (Answer: raw residuals \(4\) and \(16\); Pearson residuals \(0.5000\) and \(0.5000\))
- For a Bernoulli observation with fitted probability \(p=0.5\) and observed outcome \(y=1\), find the raw, Pearson, and deviance residuals. (Answer: \(e=0.5\), \(r_P=1.0000\), \(r_D=1.1774\))