Deriving Pearson and Deviance Residuals

Why GLMs Have More Than One Residual

A residual compares an observed response with the value predicted by a fitted model. In a generalized linear model (GLM), however, the response variance depends on its mean and on the response distribution. A raw difference of 2 therefore has a different scale when the expected count is 3 than when it is 300. Residuals account for this changing scale so that discrepancies can be assessed more meaningfully.

Two common choices are the Pearson residual, which scales the response error by its fitted standard deviation, and the deviance residual, which assigns each observation a signed version of its contribution to model deviance. They describe discrepancies in different ways; neither changes the fitted values or the underlying response.

Definition: For independent observations with unit prior weights, let $Y_i$ have fitted mean $\hat{\mu}_i$, variance function $V(\hat{\mu}_i)$, and dispersion estimate $\hat{\phi}$. The Pearson residual is $$ r_{P,i}=\frac{y_i-\hat{\mu}_i}{\sqrt{\hat{\phi}V(\hat{\mu}_i)}}. $$ The deviance residual is $$ r_{D,i}=\operatorname{sign}(y_i-\hat{\mu}_i)\sqrt{d_i}, $$ where $d_i$ is the observation’s nonnegative unit deviance contribution. For binomial data recorded as counts, both the variance and the unit deviance contribution must reflect the number of trials.

Here $y_i$ is the observed response. The variance function $V(\mu)$ describes how variance changes with the mean; for example, it is $\mu$ for a Poisson response. The dispersion parameter $\phi$ scales that variance. In common Poisson and binomial GLMs, $\phi$ is conventionally fixed at 1. In a Gaussian GLM, or a model that estimates dispersion, it need not equal 1.

Deriving the Pearson Residual

Start with the response-scale error $Y_i-\mu_i$, where $\mu_i=E[Y_i]$. If the model specifies \[ \operatorname{Var}(Y_i)=\phi V(\mu_i), \] then the standard deviation of $Y_i$ is $\sqrt{\phi V(\mu_i)}$. Dividing the error by this standard deviation gives a response error measured in standard-deviation units. Since the true mean and dispersion are not known, a fitted GLM substitutes $\hat{\mu}_i$ and $\hat{\phi}$.

\frac{Y_i-\mu_i}{\sqrt{\operatorname{Var}(Y_i)}} = \frac{Y_i-\mu_i}{\sqrt{\phi V(\mu_i)}} \quad\longrightarrow\quad r_{P,i}= \frac{y_i-\hat{\mu}_i}{\sqrt{\hat{\phi}V(\hat{\mu}_i)}}.

This derivation explains the scale adjustment: observations with larger fitted variance have a larger denominator. A Pearson residual is not necessarily exactly standard normal. Even if the model is correctly specified, its fitted mean and dispersion are estimated from the data, and the response distribution may be discrete or skewed.

Important distinction: Under the definition above, the sum of squared Pearson residuals is $$ \sum_i r_{P,i}^2 = \frac{1}{\hat{\phi}}\sum_i \frac{(y_i-\hat{\mu}_i)^2}{V(\hat{\mu}_i)}. $$ The usual unscaled Pearson statistic is $X_P^2=\sum_i (y_i-\hat{\mu}_i)^2/V(\hat{\mu}_i)$. Therefore, $\sum_i r_{P,i}^2=X_P^2/\hat{\phi}$; the two sums coincide only when $\hat{\phi}=1$. For a single observation, the same relationship holds between its squared Pearson residual and its unscaled Pearson contribution.

Deriving the Deviance Residual

The deviance compares a fitted model with a saturated model. A saturated model is a reference model with enough freedom to fit each observed response exactly. For an observation $i$, its unit deviance contribution $d_i$ measures the loss in fit from using the fitted mean instead of the saturated fit. It is nonnegative, and it is zero when the fitted value equals the observation.

For an exponential-dispersion GLM with a common dispersion parameter, the unscaled deviance is defined as the sum of unit deviance contributions. Twice the log-likelihood difference between the saturated and fitted models is the deviance divided by $\phi$. The residual uses the square root of the unscaled observation-level contribution, with the sign of the response error attached. The square root makes its magnitude correspond to the contribution on a residual scale; the sign shows whether the observation is above or below its fitted mean.

D=\sum_i d_i,\qquad r_{D,i}=\operatorname{sign}(y_i-\hat{\mu}_i)\sqrt{d_i}.

For a Poisson response, the unit deviance contribution is \[ d_i=2\left[y_i\log\left(\frac{y_i}{\hat{\mu}_i}\right)-(y_i-\hat{\mu}_i)\right]. \] When $y_i=0$, the term $y_i\log(y_i/\hat{\mu}_i)$ is defined by its limit as 0. For grouped binomial data with $m_i$ trials and $y_i$ successes, fitted success probability $\hat{p}_i$, and fitted expected count $\hat{\mu}_i=m_i\hat{p}_i$, the unit deviance is \[ d_i=2\left[ y_i\log\left(\frac{y_i}{m_i\hat{p}_i}\right) +(m_i-y_i)\log\left(\frac{m_i-y_i}{m_i(1-\hat{p}_i)}\right) \right]. \] A term with a zero count is interpreted as 0 by continuity. These formulas compare the fitted model with the observation-level saturated fit; they are not interchangeable with the Pearson formula.

Worked Example: Pearson and Deviance Residuals for a Poisson Count

A hypothetical GLM for daily equipment alerts has a Poisson response. For one day, the observed count is $y=5$, and the fitted mean is $\hat{\mu}=3$. Use $\hat{\phi}=1$. Calculate both residuals, rounding only final answers to four decimal places.

Method and Pearson calculation. For a Poisson response, $V(\mu)=\mu$. Thus $V(\hat{\mu})=3$, and the Pearson denominator is $\sqrt{1\cdot3}=\sqrt{3}=1.7321$. The response error is $5-3=2$, so \[ r_P=\frac{2}{\sqrt{3}}=1.1547. \] The positive sign indicates that the observed count exceeds the fitted mean.

Deviance calculation. Substitute $y=5$ and $\hat{\mu}=3$ into the Poisson unit deviance: \[ d=2\left[5\log\left(\frac{5}{3}\right)-(5-3)\right]. \] Using $\log(5/3)=0.5108256$, the bracket is $5(0.5108256)-2=2.5541280-2=0.5541280$. Therefore $d=2(0.5541280)=1.1082560$, and \[ r_D=+\sqrt{1.1082560}=1.0527. \] The sign is positive because $5-3>0$.

Check and trap avoided. The deviance contribution is nonnegative, and its square root has the sign of the response error. Using the Pearson denominator $\sqrt{\hat{\mu}}$ for the deviance residual would mix two different definitions.

Worked Example: Grouped-Binomial Residuals

A hypothetical screening study records 7 positive results among 10 people in one group. A fitted binomial GLM gives $\hat{p}=0.5$. Calculate Pearson and deviance residuals for the group, rounding final answers to four decimal places.

Method and Pearson calculation. The fitted expected number of positive results is $m\hat{p}=10(0.5)=5$, and the fitted expected number of negative results is also 5. The variance of the success count is $m\hat{p}(1-\hat{p})=10(0.5)(0.5)=2.5$. The observed-minus-fitted count is $7-5=2$, so \[ r_P=\frac{2}{\sqrt{2.5}}=\frac{2}{1.5811}=1.2649. \]

Deviance calculation. There are $10-7=3$ observed negative results. The unit deviance is \[ d=2\left[7\log\left(\frac{7}{5}\right)+3\log\left(\frac{3}{5}\right)\right]. \] Using $\log(1.4)=0.3364722$ and $\log(0.6)=-0.5108256$, the bracket is $7(0.3364722)+3(-0.5108256)=2.3553054-1.5324768=0.8228286$. Hence $d=1.6456572$, and \[ r_D=+\sqrt{1.6456572}=1.2828. \]

Check and trap avoided. The count variance is 2.5, not the variance of a single Bernoulli response, 0.25. The observation is a group of 10 trials, so its count-scale variance and deviance both use the group size.

Dispersion and the Pearson-Statistic Trap

The dispersion factor in the Pearson residual matters whenever it differs from 1. For a one-observation illustration, suppose $\hat{\phi}=4$, $V(\hat{\mu})=1$, and $y-\hat{\mu}=2$. The Pearson residual is $2/\sqrt{4}=1$, so its square is 1. The usual unscaled Pearson contribution is $2^2/1=4$. Thus the squared residual is the unscaled contribution divided by 4, not the unscaled contribution itself.

A common error is to write “summing squared Pearson residuals gives the Pearson statistic” while using a residual denominator that includes $\hat{\phi}$. The corrected calculation is \[ \hat{\phi}\sum_i r_{P,i}^2 = \hat{\phi}\sum_i \frac{(y_i-\hat{\mu}_i)^2}{\hat{\phi}V(\hat{\mu}_i)} = \sum_i\frac{(y_i-\hat{\mu}_i)^2}{V(\hat{\mu}_i)} =X_P^2. \] For Poisson and binomial models with $\hat{\phi}=1$, the distinction disappears. Always check whether a stated residual definition includes dispersion before relating its squared sum to a statistic.

Worked Example: Checking the Dispersion Scaling

A hypothetical Gaussian GLM has one observation with $y=9$, fitted mean $\hat{\mu}=7$, variance function $V(\hat{\mu})=1$, and estimated dispersion $\hat{\phi}=4$. Calculate the Pearson residual and its squared value, then compare with the unscaled Pearson contribution. Round final answers to four decimal places.

Method. The response error is $9-7=2$. The fitted variance is $\hat{\phi}V(\hat{\mu})=4(1)=4$, so its square root is 2. Therefore $r_P=2/2=1.0000$, and $r_P^2=1.0000$.

The unscaled Pearson contribution is $(9-7)^2/1=4.0000$. Multiplying the squared residual by the dispersion gives $4(1)=4.0000$, as required. For the Gaussian unit deviance, $d=(9-7)^2=4$, so the deviance residual is $+\sqrt{4}=2.0000$.

Check and trap avoided. The Pearson residual is not 2: that would omit the dispersion from its denominator. Nor is its square the unscaled Pearson contribution here. The displayed values satisfy $\hat{\phi}r_P^2=4.0000$.

How to Calculate Them Reliably

For an observation, first identify the response distribution and the scale on which it is recorded. Then use the matching variance function for Pearson residuals and the matching unit deviance for deviance residuals. A grouped count must be treated as a count, not as a single binary response. Keep the sign separate until the magnitude has been calculated.

Identify the response and fitted value.
Record $y_i$, $\hat{\mu}_i$, the response family, and whether the observation is individual or grouped.

Calculate the Pearson residual.
Use the appropriate $V(\hat{\mu}_i)$, include $\hat{\phi}$ if it is not fixed at 1, and divide the response error by the fitted standard deviation.

Calculate the deviance residual.
Evaluate the observation’s unit deviance contribution, take its square root, and assign the sign of $y_i-\hat{\mu}_i$.

Check scale and sign.
Confirm that the deviance contribution is nonnegative, that both residual signs agree with the response error, and that the dispersion convention is consistent.

The two residuals need not have the same magnitude. Pearson residuals are based on the variance function at the fitted mean. Deviance residuals are based on the likelihood difference between the fitted and saturated descriptions. Their shared sign is useful, but their numerical scales arise from distinct constructions.

Key takeaway: Pearson residuals are response errors divided by fitted standard deviations. Deviance residuals are signed square roots of observation-level unit deviance contributions. Under the dispersion-scaled Pearson definition, their squared sum equals the usual Pearson statistic divided by $\hat{\phi}$, not necessarily the statistic itself.

Check Your Understanding

For each problem, use the residual definitions in this tutorial. Round numerical answers to four decimal places unless the answer is exact.

A Poisson GLM gives $\hat{\mu}=2$ for an observation with $y=0$, with $\hat{\phi}=1$. Calculate its Pearson and deviance residuals. (Answer: $r_P=-1.4142,\ r_D=-2.0000$)
A binomial group has $m=8$ trials, $y=6$ successes, and fitted probability $\hat{p}=0.5$. Calculate the Pearson residual. (Answer: $r_P=1.4142$)
For the group in question 2, calculate the deviance residual. (Answer: $r_D=1.4467$)
A one-observation model has $\hat{\phi}=4$, $V(\hat{\mu})=1$, and $y-\hat{\mu}=2$. Give the Pearson residual, its square, and the unscaled Pearson contribution. (Answer: $r_P=1,\ r_P^2=1,$ unscaled contribution $=4$)
A Poisson observation has $y=4$ and $\hat{\mu}=4$. Calculate both residuals and explain the sign. (Answer: $r_P=0,\ r_D=0$; the response error and deviance contribution are both zero.)

What You'll Learn