Residuals Versus Fitted Means

Why Plot Residuals Against Fitted Means?

A dispersion ratio compresses many residuals into one number. A residual-versus-fitted-mean plot retains the observation-level detail: it can show whether departures from the fitted model are concentrated at particular mean levels, whether the residual center drifts, and whether a few observations dominate the pattern. For binomial and Poisson GLMs, this is useful because their prescribed variance changes with the mean.

For observation \(i\), let \(Y_i\) be the response and \(\hat{\mu}_i\) its fitted mean. In a Poisson model, \(\hat{\mu}_i\) is the fitted expected count. For grouped binomial data with \(m_i\) trials, it is the fitted expected number of successes, \(m_i\hat{p}_i\). The horizontal coordinate should be the fitted mean on the response scale, not the linear predictor or, when exposure varies, the fitted rate alone.

Residuals for Binomial and Poisson Fits

A Pearson residual measures the response's deviation from its fitted mean in units of the model's assumed standard deviation. For a Poisson observation, the variance is \(\mu_i\); for a grouped binomial observation, the variance is \(m_i p_i(1-p_i)\). The formulas below use fitted means and probabilities in the residuals.

The signed deviance residual expresses the same departure using the observation's contribution to the residual deviance. Its sign is the sign of \(y_i-\hat{\mu}_i\). For zero counts or zero failures, terms of the form \(0\log(0/a)\) are defined as zero.

In either case, the square of a deviance residual is that observation's contribution to the residual deviance, and the sum of the squared residuals is the residual deviance. The Pearson residuals similarly satisfy \(\sum_i r_{P,i}^2=X^2\), the Pearson chi-square statistic. These identities explain why plotting individual residuals complements, rather than replaces, the aggregate dispersion checks.

What a Well-Calibrated Plot Looks Like

If the mean and variance specifications are appropriate, residuals should generally fluctuate on both sides of zero without a smooth trend in their center as fitted means change. Pearson residuals are divided by the standard deviation specified by the model. Thus, under the model, their scale is intended to be roughly comparable across fitted means. A persistent increase in their vertical spread can be a clue that the assumed variance is too small in some region; persistently narrow spread can suggest the reverse.

“Roughly” matters. Residuals use estimated parameters, so they are not independent observations with exactly the same distribution. The fitting process can also constrain or correlate residuals. Further, responses are discrete: at small Poisson means, for example, there are only a few possible counts, and a residual plot can look uneven even when the model is adequate. The plot is diagnostic evidence, not a formal test by itself.

Deviance and Pearson residual plots usually tell a similar broad story, but their individual values need not agree. At fitted means where observed values are close to expected values, their magnitudes are approximately equal. For more substantial departures, or near probability boundaries in binomial data, the scales can differ. Plotting both can help identify whether a visual conclusion depends heavily on the residual definition.

Proof. First consider the residual with the true mean and probability, rather than fitted estimates. For Poisson data let \(R=(Y-\mu)/\sqrt{\mu}\). Since \(E[Y]=\operatorname{Var}(Y)=\mu\),

For a binomial response with \(m\) trials and success probability \(p\), let \(R=(Y-mp)/\sqrt{mp(1-p)}\). The binomial mean and variance give

If the true variance in either case is \(\phi\) times the specified variance while the mean remains correct, the same calculation gives \(\operatorname{Var}(R)=\phi\). This is a statement about a residual standardized using the true mean. It does not say that residuals using estimated fitted means are exactly centered, independent, or unit variance. \(\square\)

Reading Shapes and Unusual Points

Read the plot by asking separate questions about its center, spread, and individual observations. A smooth curve in the residual center means that residuals tend to be positive at some fitted means and negative at others, or the reverse. That is a systematic pattern in the response scale that the fitted mean has not captured. The plot identifies the pattern; it does not by itself identify its cause.

Changing vertical spread is different from a shift in the center. A widening cloud of Pearson residuals as fitted means increase can suggest that the variance grows faster than the specified binomial or Poisson variance in that region. A narrowing cloud can suggest that it grows more slowly. Check whether the feature is supported by multiple observations: one large residual is not evidence of a general change in variance.

Clusters may appear as several points concentrated in a similar part of the plot, perhaps with similar signs or magnitudes. A cluster can be worth investigating, but a plot against fitted means alone does not reveal whether it corresponds to a particular predictor value or group. An isolated large residual may identify an observation whose response is poorly predicted; it may also arise from ordinary chance, especially with sparse or discrete data. Do not remove it solely because it looks unusual.

There is no universal residual cutoff that turns a visual feature into a diagnosis. Lines at plus or minus two can be used as rough visual reference points, not as guaranteed significance thresholds: the distribution of a residual depends on the family, fitted mean, and estimation. At small fitted means, asymmetry and discreteness deserve particular attention. Pearson residuals can be large in magnitude for a low-probability count, while deviance residuals can represent that departure differently.

Derivation. For a Poisson observation, write \(y=\mu+e\), with \(|e|/\mu\) small. Its squared deviance residual is \(2\{y\log(y/\mu)-(y-\mu)\}\). Treating this expression as a function of \(e\), expand \(\log(1+e/\mu)\) at zero: \(\log(1+e/\mu)=e/\mu-e^2/(2\mu^2)+O(|e|^3/\mu^3)\). Substitution and cancellation of the first-order terms yield

For grouped binomial data, set \(q=y/m\) and \(p=\hat{p}\). The deviance is \(2m\{q\log(q/p)+(1-q)\log[(1-q)/(1-p)]\}\). As a function of \(q\), the quantity in braces is zero and has first derivative zero at \(q=p\); its second derivative there is \(1/[p(1-p)]\). If \(p\) stays away from 0 and 1 and \(q-p\) is small, Taylor expansion therefore gives

The leading term is \(r_P^2\). This approximation explains why the plots can agree near zero, while not implying that their residuals are interchangeable for large departures or probabilities near the boundaries. \(\square\)

Common Mistakes

• Reading every widening cloud as overdispersion. A visual change can be driven by a few points or by discreteness at low means. Look for a sustained pattern across observations and compare it with the model's mean-dependent variance.
• Calling a curved residual center a variance pattern. A positive-to-negative drift concerns the residual center; changing vertical spread concerns variation. These are distinct features.
• Using rates as the fitted means for unequal-exposure counts. The fitted mean count is exposure times the fitted rate. Use that expected count on the horizontal axis when the response is a count.
• Treating one extreme residual as a reason to delete an observation. First verify the response, denominator or exposure, and model setup. An unusual point is a prompt for investigation, not an automatic exclusion.
• Assuming fitted residuals are independent standard normal values. Standardization does not make discrete residuals normally distributed, and estimating model parameters affects their behavior.