Why Residual Spread Must Be Judged Against the Model
A residual is the difference between an observed response and the value predicted by a model. But the size of that difference does not have a universal interpretation. A difference of five may be surprising when the model predicts a count of one, yet quite ordinary when it predicts a count of one hundred. In a generalized linear model (GLM), the appropriate comparison depends on the response family’s variance assumption as well as the fitted mean.
This distinction matters because a residual plot can look wider at larger fitted values even when the model is behaving as expected. Conversely, residuals that are consistently more variable than the model allows may signal a variance assumption that does not describe the data. The diagnostic task is not to make every residual equally small; it is to ask whether their locations and spread are compatible with the fitted model.
In a GLM, the mean is connected to a linear predictor through a link function, \(g(\mu_i)=\boldsymbol{x}_i^\mathsf{T}\boldsymbol{\beta}\). A residual pattern that changes systematically above and below zero can therefore point to a problem with the mean structure, such as an omitted nonlinear relationship. A pattern in residual spread, after accounting for the expected variance, raises a different question: does the assumed variance function describe the observations?
Expected Spread Depends on the Response Family
The variance function is not the same for every GLM. The following examples give common assumptions, conditional on the predictors and on the stated family. In the usual Poisson and binomial GLMs, the dispersion is fixed at \(\phi=1\). Gaussian and Gamma GLMs commonly estimate a dispersion parameter.
| Response model | Conditional mean | Conditional variance | Implication for raw residual spread |
|---|---|---|---|
| Poisson count | \(\mu_i\) | \(\mu_i\), with \(\phi=1\) | Standard deviation grows as \(\sqrt{\mu_i}\) |
| Binomial count out of \(n_i\) | \(n_i p_i\) | \(n_i p_i(1-p_i)\), with \(\phi=1\) | Spread depends on both group size and event probability |
| Gaussian response | \(\mu_i\) | \(\phi\), constant across fitted means | Constant conditional spread is assumed |
| Gamma response | \(\mu_i\) | \(\phi\mu_i^2\) | Standard deviation is proportional to the fitted mean |
The table describes model assumptions, not guaranteed properties of every data set with a particular type of response. For instance, the Poisson variance equals the Poisson mean under the Poisson model. If the observations vary more than that assumption allows, the residuals may show excess variation. The pattern is a diagnostic signal to investigate, not a conclusion about its cause.
Deriving the Pearson Residual Scale
The Pearson residual divides the raw residual by the conditional standard deviation specified by the model. This scaling has a simple justification. Suppose the conditional mean and variance are correctly specified and \(\phi V(\mu_i)>0\). Then
The conditional mean of the numerator is zero because \(E[Y_i\mid\boldsymbol{x}_i]=\mu_i\). The squared residual has conditional expectation
Thus, under the stated variance model, the Pearson residuals have a natural scale: their conditional squared size averages to one in expectation. This does not mean every residual should be near one in absolute value, or that every sample will have exactly unit residual variance. It explains why Pearson residuals are more useful than raw residuals for comparing spread across different fitted means. When parameters are estimated from the same data, these statements are approximate diagnostics rather than exact finite-sample guarantees.
Worked Examples
Worked Example: Comparing Poisson Residuals at Different Means
A hypothetical model for daily equipment-failure counts gives fitted means of 4 and 16 for two days. The observed counts are 7 and 22, respectively. Assuming independent Poisson responses and \(\phi=1\), calculate the raw and Pearson residuals. Does the larger raw residual on the second day indicate worse standardized fit?
Method. For a Poisson response, \(V(\mu_i)=\mu_i\), so the Pearson residual is \((y_i-\mu_i)/\sqrt{\mu_i}\).
For the first day, the raw residual is \(7-4=3\). The assumed standard deviation is \(\sqrt{4}=2\), giving \(r_{P,1}=3/2=1.5000\).
For the second day, the raw residual is \(22-16=6\). The assumed standard deviation is \(\sqrt{16}=4\), giving \(r_{P,2}=6/4=1.5000\).
The second raw residual is twice as large, but its model-standardized residual is the same. Both observed counts are 1.5 assumed standard deviations above their fitted means. The calculation is internally consistent: \(4+3=7\) and \(16+6=22\).
Common mistake avoided. Comparing raw residuals alone would label the second day as having worse fit because \(6>3\). That ignores the Poisson model’s larger expected spread at a larger mean. Standardize by the family’s conditional standard deviation before comparing residual size.
Worked Example: Binomial Spread Changes With Group Size
A hypothetical survey model assigns an event probability \(p_i=0.40\) to two groups. The first group has \(n_1=20\) trials and 13 observed events; the second has \(n_2=80\) trials and 42 observed events. Treat each response as a binomial count with \(\phi=1\). Calculate each Pearson residual and compare the raw differences.
Method. For a binomial count, the fitted mean is \(n_i p_i\), and the variance is \(n_i p_i(1-p_i)\).
For group 1, the fitted mean is \(20(0.40)=8\), and the raw residual is \(13-8=5\). The variance is \(20(0.40)(0.60)=4.8\), so the standard deviation is \(\sqrt{4.8}=2.1909\). Therefore, \(r_{P,1}=5/\sqrt{4.8}=2.2822\).
For group 2, the fitted mean is \(80(0.40)=32\), and the raw residual is \(42-32=10\). The variance is \(80(0.40)(0.60)=19.2\), so the standard deviation is \(\sqrt{19.2}=4.3818\). Therefore, \(r_{P,2}=10/\sqrt{19.2}=2.2822\).
The second group’s raw difference is twice as large, but both groups have the same Pearson residual. Each observed count is about 2.28 assumed standard deviations above its fitted mean. The arithmetic checks: the fitted means are 8 and 32, and the respective observed counts are 13 and 42.
Common mistake avoided. Applying the Bernoulli variance \(p_i(1-p_i)\) directly to a group count would omit the factor \(n_i\). That variance is for one binary trial. For a count of \(n_i\) independent trials, the variance is \(n_i p_i(1-p_i)\).
Worked Example: A Possible Excess-Variation Signal
A hypothetical Poisson model gives a fitted mean of 4 for each of eight independent policy-group counts. The observed counts are \(0,0,4,4,4,8,8,4\). Calculate the Pearson residuals and summarize what the spread suggests. Assume \(\phi=1\) as specified by the Poisson model.
Method. Each Pearson residual is \((y_i-4)/\sqrt{4}=(y_i-4)/2\). Thus the residuals, in the same order as the observations, are \(-2,-2,0,0,0,2,2,0\).
Their squared values are \(4,4,0,0,0,4,4,0\), which sum to \(16\). The mean squared Pearson residual is \(16/8=2\). Under the stated Poisson variance assumption, the conditional expectation of a squared Pearson residual is 1 when the mean is known. A mean squared value of 2 in this small illustration is a reason to investigate whether the observed spread exceeds the model’s assumed spread.
This calculation is a descriptive warning, not a formal test or a definitive diagnosis. The example has only eight observations, and in a fitted model the mean is generally estimated rather than known. The repeated fitted mean is 4, and the observed counts sum to \(32\), so their sample mean is also 4; this verifies that the example is not showing a shift in the average. It shows how spread can raise a separate diagnostic question.
Common mistake avoided. Calling the counts of 0 or 8 errors simply because their residuals are \(-2\) or \(2\) would confuse an unusual observation with a data mistake. Check the data and the model assumptions; do not remove a valid observation solely because its residual is large.
Reading Residual Patterns Carefully
A residual-versus-fitted plot can help separate a mean pattern from a spread pattern. Pearson residuals are often plotted against fitted means because they scale each raw difference using the model’s variance function. The earlier material on residuals versus predictors covers functional-form checks; here the emphasis is whether residual variability is consistent with the family’s assumed variance.
- A systematic curve or trend in the residual center—for example, residuals tending to be positive at low fitted values and negative at high fitted values—suggests that the mean predictions may be systematically too low in one region and too high in another. Consider the mean structure before attributing the pattern to variance.
- A widening or narrowing band in raw residuals is not automatically a problem. Poisson and Gamma models, for example, allow the conditional standard deviation to increase with the mean. Ask whether the change is already implied by the response family.
- Residual spread consistently greater than the assumed scale can indicate excess variation. Possible explanations include unmodeled subgroups, dependence among observations, omitted predictors, or a response distribution with a different variance structure. A plot alone does not identify which explanation is correct.
- A single large residual is a prompt to check the observation, its context, and the model—not proof of a recording error or a reason to delete it. A correctly specified probability model can still produce occasional observations far from the mean.
The deviance residual is another diagnostic measure introduced with Pearson residuals in the earlier material on choosing and deriving GLM residuals. It is a signed square root of a deviance contribution and can reveal departures from the fitted model. Pearson residuals are particularly direct for this tutorial’s question because their denominator explicitly uses the assumed variance. Neither residual type should be interpreted as a standalone verdict.
Common Traps
- Using raw residuals across different fitted means. Their expected spread may vary by family and mean. Standardize using \(\sqrt{\phi V(\mu_i)}\) when that variance model is appropriate.
- Assuming Pearson residuals must be between \(-1\) and \(1\). They are not bounded that way. A value beyond that range is possible and is not automatically a data error.
- Treating every wide residual cloud as variance misspecification. First check whether the family itself predicts changing spread, and whether a mean pattern is present.
- Equating one large residual with excess variation. Excess variation is about the overall spread relative to the model, not a label for one observation.
- Ignoring the conditions behind the variance formula. The family’s variance function is conditional on the model assumptions, including the response distribution and relevant dependence structure.
Check Your Understanding
For each problem, use the stated response family and assumptions. Report Pearson residuals to four decimal places where needed.
- A Poisson model has fitted mean \(\mu=9\), and the observed count is 15. Find the Pearson residual. (Answer: \(2.0000\))
- A binomial count has \(n=50\), fitted probability \(p=0.20\), and observed count 14. Find the fitted mean, variance, and Pearson residual. (Answer: mean \(10\), variance \(8\), residual \(1.4142\))
- A Gaussian model has fitted mean 25 and dispersion \(\phi=25\). An observation is 31. Find its Pearson residual. (Answer: \(1.2000\))
- A Gamma model has \(\mu=12\), \(\phi=0.25\), and observed response 18. Find the assumed variance and Pearson residual. (Answer: variance \(36\), residual \(1.0000\))
- Two Poisson observations have fitted means and observed counts \((2,0)\) and \((8,14)\). Calculate both raw residuals and Pearson residuals. Explain why the raw residuals alone do not compare standardized fit. (Answer: raw residuals \(-2\) and \(6\); Pearson residuals \(-1.4142\) and \(2.1213\); the model-implied standard deviations differ.)