Why Discrete Residual Plots Can Mislead

Why Discrete Residuals Need Careful Interpretation

Residual plots are useful because they can reveal patterns that a fitted mean alone conceals. But a familiar smooth cloud is not guaranteed for every generalized linear model. In binomial and count models, the response itself takes discrete values. Residuals calculated from those responses therefore often occupy a set of separated levels, with many observations tied at the same level. Their possible values and probabilities can also change with the fitted mean.

These features matter when reading a plot against fitted means, covariates, or observation order. Bands, gaps, skewness, and changing vertical spread can arise from the response distribution and the residual definition, even when the model’s mean and variance assumptions are reasonable. They are reasons to interpret the plot in context, not automatic proof of either a defect or a good fit.

Definition: For an observation with response $Y_i$, fitted mean $\hat{\mu}_i$, and fitted variance function $\hat{V}_i$, the Pearson residual is $$ r_{Pi}=\frac{Y_i-\hat{\mu}_i}{\sqrt{\hat{V}_i}}. $$ A signed deviance residual is the signed square root of that observation’s deviance contribution. Both are transformations of the observed response, so a discrete response generally gives a discrete set of possible residual values.

Discrete Support Produces Bands and Ties

For a binomial observation $Y_i\sim\operatorname{Bin}(m_i,p_i)$, the response can only be one of $0,1,\ldots,m_i$. If $m_i$ and the fitted probability are fixed, the Pearson residual can therefore take only $m_i+1$ possible values. When the denominator $m_i$ is one, there are just two possible values. With a common fitted probability, observations with the same response and denominator have exactly the same Pearson residual; tied fitted means can make the ties especially visible in a plot.

For a Poisson response, the possible counts are $0,1,2,\ldots$. There are infinitely many possible residual values, but the observed counts are still integers. At a fixed fitted mean, the residuals fall on corresponding discrete levels. In practice, fitted means vary across observations, so these levels may form curved bands rather than horizontal stripes. Rounding of plotted values can make near-ties look exact.

Worked Example: Two Levels for a Bernoulli Response

Suppose an observation is Bernoulli with fitted probability $\hat{p}=0.2$. Its response is either $Y=0$ or $Y=1$, and the fitted mean and variance are $\hat{\mu}=0.2$ and $\hat{V}=0.2(0.8)=0.16$. The standard deviation used in the Pearson residual is therefore $0.4$.

If $Y=0$, then $r_P=(0-0.2)/0.4=-0.5$. If $Y=1$, then $r_P=(1-0.2)/0.4=2$. There are no other possible values for this observation at this fitted probability.

Thus, a collection of Bernoulli observations with the same fitted probability can produce two horizontal residual bands, at $-0.5$ and $2$, rather than a continuous-looking cloud. The unequal distances from zero also show that the support is asymmetric.

Asymmetric Support and Changing Residual Shape

A binomial response is bounded both below and above, while a count response cannot be negative and has no fixed upper bound. These restrictions affect residuals too. For a Bernoulli response with fitted probability $p$, the Pearson residuals for outcomes zero and one are $-\sqrt{p/(1-p)}$ and $\sqrt{(1-p)/p}$, respectively. Unless $p=1/2$, these are not equally far from zero. For counts, there can be a substantial run of possible negative residual values only down to the value corresponding to a zero count, while positive values continue as the count increases.

The response distribution also changes shape as its mean changes. For a binomial response, both its probability and its variance $mp(1-p)$ depend on $p$; for a Poisson response, the variance is its mean $\mu$. Pearson scaling divides by the model-based standard deviation, but that scaling does not turn a discrete, skewed distribution into a normal one. In particular, equal variance does not imply equal shape.

Result: If $Y\sim\operatorname{Bin}(m,p)$, with $m\geq1$ and $0<p<1$, and the parameters are treated as known, then the Pearson residual $R=(Y-mp)/\sqrt{mp(1-p)}$ has mean zero and variance one. For $m=1$, its third standardized moment is $$ E[R^3]=\frac{1-2p}{\sqrt{p(1-p)}}. $$ So it is generally skewed despite having mean zero and unit variance.

To verify the result, the binomial mean and variance are $E[Y]=mp$ and $\operatorname{Var}(Y)=mp(1-p)$. Therefore $$ E[R]=\frac{E[Y]-mp}{\sqrt{mp(1-p)}}=0, \qquad \operatorname{Var}(R)=\frac{\operatorname{Var}(Y)}{mp(1-p)}=1. $$ For $m=1$, write $q=1-p$. The response equals one with probability $p$ and zero with probability $q$. Hence $$ E[(Y-p)^3]=p(1-p)^3+q(0-p)^3 =pq^3-qp^3 =pq(q-p). $$ Dividing by $[pq]^{3/2}$, the cube of the standard deviation, gives $E[R^3]=(q-p)/\sqrt{pq}=(1-2p)/\sqrt{p(1-p)}$. This is nonzero when $p\ne1/2$, demonstrating skewness directly. $\square$

Worked Example: Binomial Ties and Deviance Residuals

Let $Y\sim\operatorname{Bin}(4,0.5)$, with the probability treated as known. The fitted mean is $2$, and the fitted variance is $4(0.5)(0.5)=1$. The Pearson residual is therefore $r_P=Y-2$. For responses $Y=0,1,2,3,4$, its possible values are $-2,-1,0,1,2$. The probabilities of those responses are $1/16,4/16,6/16,4/16,1/16$, so repeats are possible and the middle level is the most probable.

The signed deviance residual for this binomial observation is $$ r_D=\operatorname{sign}(Y-2)\sqrt{2\left[ Y\log\!\left(\frac{Y}{2}\right) +(4-Y)\log\!\left(\frac{4-Y}{2}\right)\right]}, $$ using $0\log 0=0$. At $Y=0$, this gives $r_D=-\sqrt{8\log 2}$, approximately $-2.355$. At $Y=1$, it gives $$ r_D=-\sqrt{2\left[\log(1/2)+3\log(3/2)\right]}\approx-1.023. $$ At $Y=2$, the residual is zero. The values at $Y=3$ and $Y=4$ are $1.023$ and $2.355$, by symmetry at $p=0.5$. The deviance residuals, like the Pearson residuals, occupy just five possible levels; changing the residual definition does not remove discreteness.

Count Residuals: A Worked Poisson Calculation

Worked Example: A Poisson Response at a Small Fitted Mean

Suppose $Y\sim\operatorname{Poisson}(0.5)$, with fitted mean $\hat{\mu}=0.5$. The Pearson residual is $r_P=(Y-0.5)/\sqrt{0.5}$. For $Y=0,1,2,3$, its values are approximately $-0.707, 0.707, 2.121, 3.536$. The first four response probabilities are approximately $0.6065, 0.3033, 0.0758,$ and $0.0126$, respectively. Most observations are likely to have residual $-0.707$, while positive residuals can extend to larger values as the count increases.

For the same observation, the signed Poisson deviance residual is $$ r_D=\operatorname{sign}(Y-0.5)\sqrt{2\left[Y\log\!\left(\frac{Y}{0.5}\right)-(Y-0.5)\right]}, $$ again with $0\log 0=0$. For $Y=0$, it is $-\sqrt{1}=-1$. For $Y=1$, it is $\sqrt{2(\log 2-0.5)}\approx0.622$. For $Y=2$, it is $\sqrt{2(2\log 4-1.5)}\approx1.595$. The possible values are discrete and their probabilities are unequal. A vertical asymmetry in a plot at such means can therefore reflect the Poisson distribution, not necessarily a misspecified mean structure.

These calculations also show why different residual definitions need not display the same spacing. Pearson residuals scale the response difference linearly; deviance residuals use a likelihood-based transformation. Both retain the response’s discrete support. A plot that looks more balanced under one residual definition is not, by itself, evidence that the model is correct.

What a Residual Plot Can and Cannot Show

The vertical scale should be interpreted in light of the chosen residual. Raw response residuals $Y_i-\hat{\mu}_i$ can have changing spread because the model variance changes with the mean: for a Poisson model the variance is $\mu_i$, and for a binomial count it is $m_i p_i(1-p_i)$. A widening raw-residual band may therefore be expected under the model. Pearson residuals account for this variance scaling, but their distributions can still be discrete, skewed, and differently shaped at different fitted means.

When many points overlap, a plot may hide how often each discrete outcome occurred. A dense-looking band can consist of a few repeated values, and a gap between bands can be dictated by the response support. Conversely, a broad or asymmetric pattern may still warrant investigation: the diagnostic question is whether its structure is plausible under the fitted distribution and design, not whether it resembles an idealized normal cloud.

A normal quantile plot of ordinary discrete-response residuals also needs qualification. Ties produce repeated ordered values, so the plotted points may step rather than follow a smooth curve. A departure from a straight line can reflect discreteness or skewness as well as model inadequacy. Randomized quantile residuals are one alternative for discrete responses: their randomization addresses the jumps in the discrete distribution, but fitted parameters and finite samples still affect their reference behavior. No residual transformation removes the need to consider how the data were generated.

Worked Example: Reading Two Bands Without Overdiagnosing

Suppose a residual-versus-fitted-mean plot for Bernoulli observations contains two bands, and the fitted probabilities vary from $0.2$ to $0.4$. At a fixed probability of $0.2$, the Pearson residual values are $-0.5$ for $Y=0$ and $2$ for $Y=1$. At a fixed probability of $0.4$, they are $-\sqrt{0.4/0.6}\approx-0.816$ and $\sqrt{0.6/0.4}\approx1.225$.

Because the fitted probabilities vary, the two possible residuals move with the fitted mean; they do not form two perfectly horizontal lines across the entire plot. Still, their discrete outcomes can appear as two curved bands. A responsible interpretation first recognizes that the bands are compatible with a binary response. It then asks whether the observed pattern, such as systematic movement or an unexpected concentration of outcomes, is compatible with the fitted probabilities. Merely seeing bands is not a sufficient diagnosis of model failure.

Common Mistake: Treating Unit Variance as Normality

Incorrect reasoning: “The Pearson residuals have been divided by their model-based standard deviations, so they should be normally distributed. Any skew or banding means the GLM is wrong.”
Correction: Variance standardization sets a scale; it does not change a discrete response into a continuous normal response. Even with known parameters, Pearson residuals can have unit variance and still be skewed, tied, and asymmetrically supported. Look for structure that is unexpected under the fitted response distribution rather than requiring a normal-looking cloud.

Key takeaway: In binomial and count models, residual bands, ties, asymmetric support, and changing shape can be consequences of discreteness. Interpret them against the fitted distribution and the residual definition; do not treat a conventional normal-looking plot as a universal requirement.

Check Your Understanding

Use the response distribution and residual definition to guide each answer.

Why can observations with the same binomial denominator, fitted probability, and response have exactly tied Pearson residuals?
For a Bernoulli observation with fitted probability $p$, explain why its two Pearson residual values need not be equally distant from zero.
What does unit variance of a Pearson residual establish, and what does it not establish?
Why might raw Poisson response residuals show increasing spread as fitted means increase?
A deviance-residual plot has separated bands. Give one distributional explanation and one additional feature you would examine before concluding that the model is inadequate.