Response Residuals Across GLM Families

What a Response Residual Measures

A fitted GLM gives each observation a fitted mean on the response scale. Comparing the recorded response with that mean gives a direct, unstandardized measure of the discrepancy: how far the observation lies above or below what the model predicts. This simple comparison is useful precisely because it remains in the units of the response. A residual of 2 can mean two events, two percentage points, or two grams, depending on how the response was recorded.

Let $Y_i$ be the response for observation $i$, and let $\hat{\mu}_i$ denote its fitted conditional mean. The fitted mean is on the response scale, even if the model was fitted using a link function on another scale. The response residual is the observed response minus that fitted mean.

Definition: The response residual for observation $i$ is $$ e_i^{(R)}=y_i-\hat{\mu}_i, $$ where $y_i$ is the observed response and $\hat{\mu}_i$ is the fitted conditional mean $E[Y_i\mid x_i]$ under the fitted model. A positive residual means the observation is above its fitted mean; a negative residual means it is below. The residual has the same units as the response as recorded.

This tutorial uses $\hat{\mu}_i$ to mean a fitted expectation for the response actually represented by $y_i$. That detail matters for binomial data: if $y_i$ is a number of successes, its fitted mean is an expected number of successes; if $y_i$ is a proportion, its fitted mean is a probability. The model and the residual must refer to the same response representation.

Response Residuals by Family

The subtraction itself does not change across GLM families. What changes is the response scale and, for binomial observations, how the data are recorded. The examples below use specified conditional distributions and fitted means so that every residual can be calculated directly.

Worked Example: Gaussian Response in Measurement Units

Suppose a Gaussian model gives a fitted mean of 12.0 milligrams for an observation, and the observed amount is 15.2 milligrams. For instance, the response could have conditional distribution $Y_i\mid x_i\sim N(12,9)$; the variance is 9, but it is the mean 12 that enters the response residual.

Subtract the fitted mean from the observed response: $$ e_i^{(R)}=15.2-12.0=3.2. $$

The response residual is 3.2 milligrams. It is positive because the observed amount exceeds the fitted mean. The variance does not appear in this calculation; dividing by a standard deviation would produce a different kind of residual.

Worked Example: Poisson Count Residual

Suppose $Y_i\mid x_i\sim\operatorname{Poisson}(4.6)$, so the conditional mean is 4.6 events, and the observation records 7 events. Then $$ e_i^{(R)}=7-4.6=2.4. $$

The residual is 2.4 events: the observed count is above its fitted expected count. Counts are integers, but residuals need not be integers because a fitted mean such as 4.6 can be noninteger. The residual remains on the event-count scale; it is not a proportion or a standardized count.

Worked Example: Grouped Binomial Count and Proportion

In group $i$, suppose 20 trials are observed, with 8 successes. Let the model give a fitted success probability of $\hat{p}_i=0.30$. The fitted expected number of successes is $$ \widehat{E[Y_i\mid x_i]}=20(0.30)=6. $$

If the response is the success count, the response residual is $$ e_{i,\mathrm{count}}^{(R)}=8-6=2\ \text{successes}. $$

If the same data are represented as the observed proportion, the response is $y_i=8/20=0.40$, and its fitted mean is the fitted probability, $0.30$. The proportion-scale residual is $$ e_{i,\mathrm{prop}}^{(R)}=0.40-0.30=0.10. $$

These are two representations of the same discrepancy, with different units: the count residual is 2 successes, while the proportion residual is 0.10, or 10 percentage points. They are related by $2=20(0.10)$. For one Bernoulli trial, the response is either 0 or 1, and the residual is $y_i-\hat{p}_i$.

Worked Example: Gamma Response in Outcome Units

Suppose a positive response has a gamma conditional distribution with shape 4 and scale 2. Its conditional mean is $4(2)=8$. For an observation $y_i=10.5$, the response residual is $$ e_i^{(R)}=10.5-8=2.5. $$

If the response is measured in hours, the residual is 2.5 hours; if it is measured in milliliters, it is 2.5 milliliters. The gamma family specifies a distribution for a positive response, but the response residual is still an observed-minus-fitted-mean difference. It is not a ratio or a difference on a transformed scale.

Two Properties of the Raw Difference

The response residual has a straightforward interpretation, but two mathematical facts help prevent overinterpretation: its centering property refers to the true conditional mean, and its units depend on the response representation.

Result: If $E[Y_i\mid x_i]=\mu_i$ exists and is finite, then the residual formed using the true conditional mean, $Y_i-\mu_i$, has conditional expectation zero: $$ E[Y_i-\mu_i\mid x_i]=0. $$ This statement does not assert that a residual formed with an estimated fitted mean has conditional expectation exactly zero after fitting.

Proof. The value $\mu_i=E[Y_i\mid x_i]$ is fixed conditional on $x_i$. Linearity of conditional expectation therefore gives $$ E[Y_i-\mu_i\mid x_i] =E[Y_i\mid x_i]-E[\mu_i\mid x_i] =\mu_i-\mu_i =0, $$ where $E[\mu_i\mid x_i]=\mu_i$ because $\mu_i$ is fixed given $x_i$. Finiteness of the conditional mean ensures these expectations are defined. $\square$

In practice, $\hat{\mu}_i$ is estimated from the data. It is not generally a fixed true conditional mean, and estimation can introduce dependencies among fitted residuals. Thus the definition $y_i-\hat{\mu}_i$ is exact, but one should not infer from the result above that every fitted residual has conditional mean zero.

Result: For grouped binomial data with $r_i$ trials, observed success count $Y_i$, and success proportion $\bar{Y}_i=Y_i/r_i$, suppose the fitted success probability is $\hat{p}_i$. The count-scale and proportion-scale response residuals satisfy $$ Y_i-r_i\hat{p}_i =r_i\left(\frac{Y_i}{r_i}-\hat{p}_i\right). $$

Derivation. The fitted mean of the count is $r_i\hat{p}_i$, so its response residual is $Y_i-r_i\hat{p}_i$. The observed proportion has fitted mean $\hat{p}_i$, so its residual is $Y_i/r_i-\hat{p}_i$. Multiplying the latter by $r_i$ gives $$ r_i\left(\frac{Y_i}{r_i}-\hat{p}_i\right) =Y_i-r_i\hat{p}_i, $$ as claimed. This equality explains why changing the response representation changes the numerical residual and its units, even though the underlying observed-minus-expected discrepancy is the same. $\square$

What the Units Do—and Do Not—Tell You

A raw response residual is easy to read in context: its sign indicates direction and its magnitude is expressed in the response’s units. A positive Poisson residual is an excess number of events; a positive gamma residual is an excess amount of a continuous measurement. For grouped binomial data, the residual may be reported in successes or as a proportion, so the scale must be stated.

The units also limit comparisons. A residual of 2 events and a residual of 2 milligrams do not represent comparable discrepancies just because their numerical values match. Even within one family, observations can have different variability. Response residuals do not account for that variation, and their magnitudes are not standardized measures of unusualness. Variance standardization is a separate step.

A second practical check is whether the fitted mean matches the recorded response. If a table contains 8 successes from 20 trials, subtracting the fitted probability 0.30 directly from 8 mixes a count and a proportion. Either compare 8 with the expected count 6, or compare 0.40 with 0.30. The relation between the answers is multiplication by the trial count, not equality.

Common Mistakes

Subtracting on mismatched scales. A success count cannot be compared directly with a fitted probability. Convert the fitted probability to an expected count, or express the observation as a proportion.
Assuming residuals must be integers. The observed Poisson count is an integer, but its fitted mean need not be. Their difference can therefore be noninteger.
Calling a raw residual standardized. The difference $y_i-\hat{\mu}_i$ retains the response units and does not adjust for the observation’s variance.
Claiming fitted residuals always have conditional mean zero. The centering calculation applies to the true conditional mean. A fitted mean is estimated, so the distinction matters.

Key takeaway: A response residual is observed response minus fitted conditional mean on the response scale. Check how the response is recorded—especially for grouped binomial data—before interpreting its size or units.

Check Your Understanding

For each question, state the response scale before interpreting the residual.

A Poisson observation is 3 events and its fitted mean is 4.2 events. Calculate and interpret its response residual.
A group has 12 successes in 30 trials, and the fitted success probability is 0.35. Find both the count-scale and proportion-scale response residuals.
A gamma response is 6.8 liters and its fitted mean is 7.5 liters. What is the response residual, including its sign and units?
Why does the zero conditional expectation result for a residual using the true mean not establish the same claim for every fitted residual?
Two raw residuals have numerical values 2, one measured in events and one measured in hours. Why is their numerical equality not enough to compare their sizes meaningfully?