Signed Deviance Residuals and Their Contributions

From Likelihood Contributions to Residuals

A model deviance summarizes the gap between the fitted model and a saturated model, which can fit each observation separately. That total is useful for assessing overall fit, but it does not identify which observations contribute most or whether a fitted mean lies above or below its observed response. Signed deviance residuals retain both pieces of information: the size of an observation’s deviance contribution and the direction of its response error.

The construction is simple once the observation-level contributions are defined. Take the square root of each nonnegative contribution, then attach the sign of the observed response minus its fitted mean. The resulting residuals have a useful exact aggregation property: their squares add to the model deviance.

Definitions and Conventions

Assume independent observations $Y_1,\ldots,Y_n$ from a likelihood-based GLM. Write $y_i$ for the observed response and $\hat{\mu}_i$ for its fitted mean. For observation $i$, let $\ell_i^{\mathrm{sat}}(y_i)$ be the maximized log-likelihood under a model that fits that observation separately, and let $\ell_i(y_i;\hat{\mu}_i)$ be its log-likelihood contribution under the fitted model. The unit-deviance contribution is twice the saturated-minus-fitted log-likelihood difference, on the course’s unit-deviance scale. If there is a known prior weight $q_i>0$, the observation’s contribution to total deviance includes that weight.

Definition: Let $d(y_i,\hat{\mu}_i)$ be the unit deviance for observation $i$, and let $q_i$ be its prior weight (with $q_i=1$ when observations are unweighted). Its deviance contribution is $c_i=q_i d(y_i,\hat{\mu}_i)$. The signed deviance residual is $$ r_{D_i}=\operatorname{sign}(y_i-\hat{\mu}_i)\sqrt{c_i}, $$ where $\operatorname{sign}(u)=1$ for $u>0$, $-1$ for $u<0$, and $0$ for $u=0$.

Some presentations call $d(y_i,\hat{\mu}_i)$ itself the observation’s deviance contribution when there are no weights. Keeping the names separate makes the weighted case clear: the unit deviance is $d$, whereas the amount that enters the total is $c_i=q_i d$. If a common dispersion parameter is used, the unit deviance and total deviance are conventionally reported without dividing by that dispersion; the corresponding likelihood-ratio statistic is then dispersion-scaled.

The saturated likelihood for each observation must be at least as large as the fitted likelihood for that observation. When the saturated fit is attained only at a boundary, interpret it as the limiting saturated likelihood. Thus the contributions used here are nonnegative, and their square roots are real. A residual is zero when the contribution is zero. A positive residual means $y_i>\hat{\mu}_i$; a negative residual means $y_i<\hat{\mu}_i$.

Why the Squares Add to Deviance

For independent observations, the fitted log-likelihood is the sum of the individual log-likelihood contributions. The saturated model also separates observation by observation, because it allows each observation its own best-fitting mean. This factorization is what makes the observation-level residuals aggregate exactly.

Theorem: For independent observations whose total deviance is the sum of their weighted unit-deviance contributions, $$ D=\sum_{i=1}^{n} c_i, $$ the signed deviance residuals satisfy $$ \sum_{i=1}^{n}r_{D_i}^{\,2}=D. $$

Proof. By definition, $r_{D_i}=\operatorname{sign}(y_i-\hat{\mu}_i)\sqrt{c_i}$. Squaring gives $$ r_{D_i}^{\,2}=\operatorname{sign}(y_i-\hat{\mu}_i)^2c_i. $$ If $y_i-\hat{\mu}_i\ne 0$, the squared sign is $1$. If $y_i-\hat{\mu}_i=0$, the sign is $0$; in the usual GLM setting the observation is then at its saturated mean and $c_i=0$, so the identity still gives $r_{D_i}^{\,2}=c_i$. Therefore, for every observation $r_{D_i}^{\,2}=c_i$. Summing over observations gives $$ \sum_{i=1}^{n}r_{D_i}^{\,2}=\sum_{i=1}^{n}c_i=D. $$ $\square$

The likelihood factorization also verifies the sum defining $D$. If the independent-observation likelihood is a product of individual likelihoods, taking logs turns that product into a sum. Subtracting the fitted log-likelihood from the saturated log-likelihood therefore gives the sum of the observation-level differences; multiplying by two on the unit-deviance scale yields the total deviance. This identity concerns the fitted model’s deviance, not a sum of signed residuals: positive and negative residuals do not generally cancel.

Result: Subject to the saturated likelihood being at least as large as the fitted likelihood for each observation, $c_i\ge 0$, so every $r_{D_i}$ is real. Also, $r_{D_i}^{\,2}=c_i$; the sign changes the residual’s direction but not its contribution to total deviance.

Justification. For observation $i$, the saturated fit maximizes its likelihood over the permitted observation-specific mean, including a boundary limit if necessary. The fitted model supplies one permitted mean value. A maximum is at least as large as the value at that fitted mean, so $\ell_i^{\mathrm{sat}}(y_i)\ge \ell_i(y_i;\hat{\mu}_i)$. Their difference, and hence the corresponding unit-deviance contribution, is nonnegative. Since $q_i>0$, $c_i\ge0$, and its square root exists. Squaring the definition of $r_{D_i}$ gives $c_i$ when the sign is nonzero. When the sign is zero, $y_i=\hat{\mu}_i$; in a regular GLM with a unique saturated fit at that response, the contribution is zero as well. $\square$

Worked Examples

The examples use Gaussian GLMs with known dispersion $1$, so the calculations are particularly transparent. With prior weight $q_i$, observation $i$ has variance $1/q_i$. Its unit-deviance contribution including the weight is $c_i=q_i(y_i-\hat{\mu}_i)^2$. The focus here is how contributions become signed residuals and add up, rather than deriving formulas for other response families.

Worked Example: An Unweighted Intercept-Only Model

Let $Y_1,\ldots,Y_4$ be independent $N(\mu,1)$ observations in an intercept-only model. The observed values are $1,2,3,6$. The fitted mean is their average: $$ \hat{\mu}=\frac{1+2+3+6}{4}=3. $$

The response errors $y_i-\hat{\mu}$ are $-2,-1,0,3$. Since the weights are all $1$, the contributions are their squared errors: $c_1=4$, $c_2=1$, $c_3=0$, and $c_4=9$. Taking square roots and applying the signs gives $r_{D_1}=-2$, $r_{D_2}=-1$, $r_{D_3}=0$, and $r_{D_4}=3$.

The total deviance is $D=4+1+0+9=14$. The squared residuals give the same result: $(-2)^2+(-1)^2+0^2+3^2=14$. In particular, the signs distinguish the two observations below the fitted mean from the one above it.

Worked Example: Prior Weights Change the Contributions

Suppose two independent observations have distributions $Y_1\sim N(\mu,1/3)$ and $Y_2\sim N(\mu,1)$, corresponding to prior weights $q_1=3$ and $q_2=1$. Observe $y_1=0$ and $y_2=4$ in an intercept-only model. The weighted fitted mean is $$ \hat{\mu}=\frac{3(0)+1(4)}{3+1}=1. $$

The errors are $-1$ and $3$. The first observation’s contribution is $c_1=3(-1)^2=3$, and the second’s is $c_2=1(3)^2=9$. Thus the signed residuals are $r_{D_1}=-\sqrt{3}$ and $r_{D_2}=3$.

The model deviance is $D=3+9=12$, and the residual squares add to $(-\sqrt{3})^2+3^2=3+9=12$. The first observation has a smaller absolute raw error, but its weight affects the deviance contribution. The sign still comes from the unweighted response error $y_i-\hat{\mu}_i$.

Worked Example: Residual Signs in a Regression Through the Origin

Let $Y_i\sim N(\beta x_i,1)$ independently, with $x=(1,2,3)$ and observed responses $y=(1,1,5)$. The least-squares estimate in this regression through the origin is $$ \hat{\beta}=\frac{\sum_i x_i y_i}{\sum_i x_i^2} =\frac{1(1)+2(1)+3(5)}{1^2+2^2+3^2} =\frac{18}{14}=\frac{9}{7}. $$

The fitted means are $9/7,18/7,27/7$. Subtracting each fitted mean from its response gives errors $-2/7,-11/7,8/7$. With unit weights, the contributions are $4/49,121/49,64/49$, so the signed residuals are $-2/7,-11/7,8/7$.

Their squares sum to $$ \frac{4+121+64}{49}=\frac{189}{49}=\frac{27}{7}. $$ The total deviance is also $D=4/49+121/49+64/49=27/7$. The first two observations lie below their fitted means and have negative residuals; the third lies above its fitted mean and has a positive residual.

Interpretation and a Common Pitfall

The absolute value of a signed deviance residual is the square root of that observation’s contribution to deviance. A large absolute residual therefore marks an observation that contributes substantially to the model’s overall deviance. The sign supplies a direction: positive indicates an observed response above its fitted mean, while negative indicates one below. This directional information can help identify patterns across observations, but the sign by itself does not indicate whether a contribution is large.

Common mistake: treating the signed residual itself as the deviance contribution, or summing signed residuals to obtain total deviance. For example, in the first worked example the residuals sum to $-2-1+0+3=0$, but the deviance is $14$, not $0$. The correction is to square each signed residual: its square is the corresponding nonnegative contribution, and those squares—not the signed values—sum to $D$.

The identity is exact bookkeeping, not a claim that residuals have a particular reference distribution, are independent, or sum to zero. Their behavior also depends on the fitted model and its data. Signed deviance residuals are useful because they make the observation-level pieces of deviance visible while preserving the model’s overall deviance as the sum of their squares.

Check Your Understanding

Use the definitions and aggregation result to answer these questions.

If an observation has contribution $c_i=2.25$ and $y_i<\hat{\mu}_i$, what is its signed deviance residual?
Why do the squared signed deviance residuals, rather than the signed residuals, add to total deviance?
For an observation with prior weight $q_i=4$, unit deviance $d_i=0.5$, and $y_i>\hat{\mu}_i$, find its contribution and signed deviance residual.
Can two observations with equal absolute signed deviance residuals have different signs? Explain what the signs mean.
What does the identity between squared residuals and deviance establish, and what does it not establish about residual distributions?