Working Residuals on the Linear-Predictor Scale

Why Put a Response Residual on the Linear-Predictor Scale?

A response residual $y_i-\hat\mu_i$ measures the discrepancy between an observed response and its fitted mean in response units. But the mean is connected to the predictors through the inverse link, so the same-sized response discrepancy can correspond to very different changes in the linear predictor at different fitted means. A working residual adjusts for this local relationship.

The key adjustment is the slope of the inverse link at the fitted linear predictor. A working residual is therefore not another response-scale discrepancy, and it is not automatically a standardized or studentized residual. Its scale is the scale of the linear predictor, in the sense of a local approximation.

Definition: For observation $i$, let $y_i$ be the observed response, $\hat\eta_i$ the fitted linear predictor, and $\hat\mu_i=h(\hat\eta_i)$ the fitted mean, where $h=g^{-1}$ is the inverse link. If $h$ is differentiable at $\hat\eta_i$ and $h'(\hat\eta_i)\ne 0$, the working residual is $$ r_i^W=\frac{y_i-\hat\mu_i}{h'(\hat\eta_i)}. $$

The numerator is the response residual. The denominator is the rate at which the fitted mean changes with the linear predictor at that observation. Thus, when a small change in the linear predictor produces a large change in the mean, the same response discrepancy gives a smaller working residual; when the mean changes only slowly, the working residual can be large.

Derivation: A Local Change in the Linear Predictor

The definition follows from a first-order approximation. Suppose the linear predictor changes from $\hat\eta_i$ to $\hat\eta_i+\delta$. Differentiability of $h$ at $\hat\eta_i$ gives

h(\hat\eta_i+\delta) = h(\hat\eta_i)+h'(\hat\eta_i)\delta+o(\delta) \qquad\text{as }\delta\to 0.

If the response changes from its fitted mean by $y_i-\hat\mu_i$, the linear approximation sets that change equal to $h'(\hat\eta_i)\delta$. Solving for $\delta$ gives $r_i^W$. This motivates describing the working residual as the response discrepancy expressed in local linear-predictor units. It is an approximation, not generally an exact transformation of the observed response.

Result: Suppose $h$ is twice continuously differentiable on the interval between $\hat\eta_i$ and $\hat\eta_i+d$, $h'(\hat\eta_i)\ne 0$, and $y_i=h(\hat\eta_i+d)$. Then $$ r_i^W=d+\frac{R_i}{h'(\hat\eta_i)}, \qquad |R_i|\le \frac{M d^2}{2}, $$ where $M$ is any upper bound for $|h''(t)|$ on that interval. In particular, $r_i^W=d+O(d^2)$ as $d\to0$.

Proof. Taylor's theorem with remainder gives

h(\hat\eta_i+d) = h(\hat\eta_i)+h'(\hat\eta_i)d+R_i, \qquad |R_i|\le \frac{M d^2}{2}.

Because $y_i=h(\hat\eta_i+d)$ and $\hat\mu_i=h(\hat\eta_i)$, subtracting the fitted mean yields $y_i-\hat\mu_i=h'(\hat\eta_i)d+R_i$. Divide by the nonzero derivative $h'(\hat\eta_i)$. The definition of $r_i^W$ then gives the stated identity, and the remainder bound implies that the difference between $r_i^W$ and $d$ is at most $M d^2/(2|h'(\hat\eta_i)|)$. For fixed $\hat\eta_i$, this is of second order in $d$, proving the local approximation. $\square$

This result applies when the response can be written as $h(\hat\eta_i+d)$. For responses outside the range of the inverse link, no such exact $d$ need exist. The working residual is still defined whenever the derivative condition holds, and it remains the first-order conversion of a response discrepancy into a linear-predictor discrepancy.

Calculating Working Residuals for Common Links

For each link, first write its inverse $h$, differentiate it, and evaluate that derivative at the fitted linear predictor. Then divide the response residual by that value. The fitted mean alone is not enough unless it determines the derivative; in the examples below it does.

Worked Example: A Bernoulli Response with a Logit Link

Suppose the fitted success probability is $\hat p_i=0.4$ and the observed Bernoulli response is $y_i=1$. The logit inverse link is $h(\eta)=e^\eta/(1+e^\eta)$, so $h'(\eta)=h(\eta)\{1-h(\eta)\}$. At the fitted value, $\hat\mu_i=\hat p_i=0.4$, and

h'(\hat\eta_i)=0.4(1-0.4)=0.24, \qquad r_i^W=\frac{1-0.4}{0.24}=\frac{0.6}{0.24}=2.5.

For the same fitted probability and an observed response $y_i=0$, the response residual is $0-0.4=-0.4$, so

r_i^W=\frac{-0.4}{0.24}=-\frac{5}{3}\approx -1.667.

The two residuals differ because their response discrepancies differ. In both cases, the derivative is positive: for this increasing inverse link, a positive working residual corresponds to an observation above its fitted mean.

For grouped binomial data, the response convention matters. If the response is the observed proportion $Y_i/m_i$, the fitted mean is $\hat p_i$ and the working residual is $(Y_i/m_i-\hat p_i)/\{\hat p_i(1-\hat p_i)\}$. If the response is the count $Y_i$, its fitted mean is $m_i\hat p_i$, whose derivative with respect to the linear predictor is $m_i\hat p_i(1-\hat p_i)$. The resulting count-scale residual is

\frac{Y_i-m_i\hat p_i}{m_i\hat p_i(1-\hat p_i)} = \frac{Y_i/m_i-\hat p_i}{\hat p_i(1-\hat p_i)}.

Thus, when the count mean and its derivative are both used, the working residual agrees with the proportion-based calculation.

Worked Example: A Count Response with a Log Link

Suppose a Poisson GLM with a log link has fitted mean $\hat\mu_i=4$, and the observed count is $y_i=7$. The inverse link is $h(\eta)=e^\eta$, so $h'(\eta)=e^\eta=h(\eta)$. Therefore the derivative at the fitted predictor is $4$, and

r_i^W=\frac{7-4}{4}=\frac{3}{4}=0.75.

If instead the observed count were $y_i=1$, the response residual would be $1-4=-3$, giving $r_i^W=-3/4=-0.75$. The derivative is positive, as expected for the increasing exponential inverse link. Here the residual is a local linear-predictor-scale version of the response discrepancy, not the log of the observed count divided by the fitted mean.

Worked Example: A Gaussian Response with an Identity Link

Suppose an identity-link model has fitted mean $\hat\mu_i=7.5$ and observed response $y_i=8.2$. Since $h(\eta)=\eta$, its derivative is $h'(\eta)=1$ everywhere. Hence

r_i^W=\frac{8.2-7.5}{1}=0.7.

With the identity link, the working residual equals the response residual exactly. This is not an approximation: dividing by the constant derivative of one leaves the residual unchanged.

Sign and Scale: What the Residual Does and Does Not Say

Result: If $h'(\hat\eta_i)>0$, then $r_i^W$ has the same sign as $y_i-\hat\mu_i$. If $h'(\hat\eta_i)<0$, then it has the opposite sign. If $h'(\hat\eta_i)=0$, the working residual in this definition is undefined.

Proof. The working residual is the response residual divided by $h'(\hat\eta_i)$. Division by a positive number preserves sign; division by a negative number reverses sign. Division by zero is undefined. $\square$

For the logit, log, and identity inverse links, the derivatives are positive wherever these links are defined, so their working residuals have the same sign as their response residuals. That interpretation is not universal across all GLM links. More generally, the size of a working residual depends on the magnitude of the derivative: a small nonzero value can produce a very large residual even when the response discrepancy is modest. Such a value signals that the local conversion is sensitive; it does not by itself establish that the observation is an outlier.

Common Mistakes

Dividing by the link derivative instead of the inverse-link derivative. The mean is $h(\eta)$, so the denominator is $h'(\hat\eta_i)=d\mu/d\eta$, not $g'(\hat\mu_i)$.
Reading the sign without checking the derivative. A positive working residual means the observation is above its fitted mean only when the inverse link is increasing. For a decreasing inverse link, the sign is reversed.
Calling the residual exact on the predictor scale. Except for special cases such as the identity link, it is a local linear approximation, not necessarily $g(y_i)-\hat\eta_i$.
Treating it as standardized. The definition divides by the inverse-link derivative, not by a residual standard deviation. No variance standardization is included.
Ignoring the response convention for grouped data. A count response and a proportion response have different response residuals and different derivatives; use a consistent mean and derivative for the chosen response.

Key takeaway: Compute $r_i^W$ by dividing $y_i-\hat\mu_i$ by $h'(\hat\eta_i)$. The derivative converts a response discrepancy to local linear-predictor units; its sign controls the sign interpretation, and its magnitude controls the scaling.

Check Your Understanding

For each question, identify the inverse-link derivative before interpreting or calculating the residual.

With a logit link, $\hat p_i=0.25$ and $y_i=1$. Calculate the working residual.
For a log-link model with $\hat\mu_i=6$ and $y_i=3$, calculate the working residual and state its sign relative to the response residual.
Why does the working residual equal the response residual under an identity link?
A model uses a decreasing inverse link and has $y_i>\hat\mu_i$. What determines whether its working residual is positive or negative?
Explain why a working residual can be large when the response residual is not large, and why that does not make it a standardized residual.