Pearson Residuals from Variance Standardization

From Response Residuals to Pearson Residuals

A response residual $y_i-\hat{\mu}_i$ records the observed-minus-fitted difference on the response scale. But the same-sized difference can have different meanings when the responses have different variances. A discrepancy of 3 is substantial if the model-based standard deviation is 1, and much less so if it is 10. A Pearson residual puts the response discrepancy in standard-deviation units, using the variance specified by the model.

The key is to standardize using the variance for the response actually being modeled. For a grouped binomial model, that may be a success count or a proportion; their variances differ because the responses are on different scales. Dispersion and variance weights must also be included when the model’s variance specification calls for them.

Definition: Suppose $Y_i$ has fitted mean $\hat{\mu}_i$ and model-based variance $\widehat{\operatorname{Var}}(Y_i)$. The Pearson residual for observation $i$ is $$ r_{Pi}=\frac{y_i-\hat{\mu}_i}{\sqrt{\widehat{\operatorname{Var}}(Y_i)}}. $$ The numerator is the response residual. The denominator is the model-based standard deviation of the response, evaluated at the fitted mean and any specified dispersion or variance weights.

Dispersion and Variance Weights

A common GLM variance specification is

\operatorname{Var}(Y_i)=\frac{\phi V(\mu_i)}{w_i},

where $\mu_i=E[Y_i]$, $V(\mu_i)$ is the variance function, $\phi>0$ is a dispersion parameter, and $w_i>0$ is a variance weight under this convention. Substitution of fitted quantities gives the dispersion-scaled Pearson residual

r_{Pi}^{(\phi)}= \frac{y_i-\hat{\mu}_i} {\sqrt{\hat{\phi}V(\hat{\mu}_i)/w_i}} = \sqrt{\frac{w_i}{\hat{\phi}}} \frac{y_i-\hat{\mu}_i}{\sqrt{V(\hat{\mu}_i)}}.

When the dispersion is fixed at 1 and $w_i=1$, this reduces to $(y_i-\hat{\mu}_i)/\sqrt{V(\hat{\mu}_i)}$. Some presentations call that expression the Pearson residual and reserve “scaled Pearson residual” for the version divided by $\sqrt{\hat{\phi}}$. Here the formula always makes clear whether dispersion has been included.

The weight convention matters. In the formula above, a larger $w_i$ means a smaller variance for the same $V(\mu_i)$ and $\phi$, so it increases the magnitude of the standardized residual for a fixed response discrepancy. Software and model descriptions may use weights for different purposes, including frequency weights that represent replicated observations. Do not treat every weight as a variance weight without checking its definition.

Theorem: If $Y_i$ has finite mean $\mu_i$ and finite positive variance $\phi V(\mu_i)/w_i$, with $\phi>0$, $V(\mu_i)>0$, and $w_i>0$, then the residual standardized using the true mean and variance, $$ Z_i=\frac{Y_i-\mu_i}{\sqrt{\phi V(\mu_i)/w_i}}, $$ has $E[Z_i]=0$ and $\operatorname{Var}(Z_i)=1$.

Proof. The denominator is a positive constant for the distribution of $Y_i$. Therefore,

E[Z_i] = \frac{E[Y_i]-\mu_i}{\sqrt{\phi V(\mu_i)/w_i}} =0.

Likewise, scaling a random variable by a constant scales its variance by the square of that constant, so

\operatorname{Var}(Z_i) = \frac{\operatorname{Var}(Y_i)}{\phi V(\mu_i)/w_i} = \frac{\phi V(\mu_i)/w_i}{\phi V(\mu_i)/w_i} =1.

This result is about standardization using the true mean and variance. In a fitted GLM, the mean and sometimes the dispersion are estimated from data. The resulting fitted residuals need not have exact mean zero and variance one observation by observation.

Grouped Binomial Responses

For a standard binomial model, let $Y_i$ be the number of successes in $m_i$ trials and let $p_i$ be the success probability. Then $E[Y_i]=m_i p_i$ and $\operatorname{Var}(Y_i)=m_i p_i(1-p_i)$. The standard binomial model has dispersion fixed at $\phi=1$. If instead a quasi-binomial variance specification is used, its stated variance is $\phi m_i p_i(1-p_i)$; that scaled variance is not the variance of a standard binomial distribution when $\phi\ne 1$.

Worked Example: A Standard Grouped-Binomial Residual

Suppose a group has $m_i=20$ trials, with $y_i=13$ observed successes, and the fitted success probability is $\hat{p}_i=0.50$. Calculate the Pearson residual under the standard binomial model.

The fitted mean count is $\hat{\mu}_i=m_i\hat{p}_i=20(0.50)=10$. The response residual is $13-10=3$. Since this is a standard binomial model, $\phi=1$, and the fitted count variance is

\widehat{\operatorname{Var}}(Y_i) =m_i\hat{p}_i(1-\hat{p}_i) =20(0.50)(0.50)=5.

Thus the Pearson residual is

r_{Pi}=\frac{13-10}{\sqrt{5}} =\frac{3}{2.2361} \approx 1.342.

The observed count is about 1.342 fitted standard deviations above its fitted mean. No extra dispersion factor is introduced: for this standard binomial calculation, it is 1.

The same grouped data can be modeled as a proportion $\bar{Y}_i=Y_i/m_i$. Its fitted mean is $\hat{p}_i$, and under the standard binomial model its variance is $\hat{p}_i(1-\hat{p}_i)/m_i$. This produces the same Pearson residual, as the next derivation shows.

Result: For a grouped binomial observation with $Y_i$ successes in $m_i$ trials, the Pearson residual calculated from the count equals the Pearson residual calculated from the sample proportion, provided each is standardized using its corresponding binomial variance.

Derivation. Write $\bar{Y}_i=Y_i/m_i$. The count-based residual is

\frac{Y_i-m_i\hat{p}_i} {\sqrt{m_i\hat{p}_i(1-\hat{p}_i)}}.

The proportion-based residual is

\frac{\bar{Y}_i-\hat{p}_i} {\sqrt{\hat{p}_i(1-\hat{p}_i)/m_i}} = \frac{(Y_i-m_i\hat{p}_i)/m_i} {\sqrt{\hat{p}_i(1-\hat{p}_i)/m_i}}.

Because $m_i>0$, the denominator in the last expression is $\sqrt{\hat{p}_i(1-\hat{p}_i)}/\sqrt{m_i}$. Multiplying numerator and denominator by $\sqrt{m_i}$ yields

\frac{Y_i-m_i\hat{p}_i} {\sqrt{m_i\hat{p}_i(1-\hat{p}_i)}},

which is exactly the count-based residual. $\square$

Worked Example: A Quasi-Binomial Variance

Use the same data, $m_i=20$, $y_i=13$, and $\hat{p}_i=0.50$, but now suppose the model explicitly specifies a quasi-binomial variance with $\hat{\phi}=1.44$. Calculate its dispersion-scaled Pearson residual.

The fitted mean remains 10 and the response residual remains 3. The quasi-binomial variance is

\widehat{\operatorname{Var}}(Y_i) = \hat{\phi}m_i\hat{p}_i(1-\hat{p}_i) = 1.44(20)(0.50)(0.50) =7.2.

Therefore,

r_{Pi}^{(\phi)} = \frac{3}{\sqrt{7.2}} = \frac{3}{2.6833} \approx 1.118.

The residual is smaller than the standard-binomial value because the explicitly specified quasi-binomial variance is larger. This calculation describes the quasi-binomial variance specification; it does not assert that the response has a standard binomial distribution with dispersion 1.44.

Poisson Responses and Variance Weights

For a standard Poisson response, $Y_i\sim\operatorname{Poisson}(\mu_i)$, so its mean and variance are both $\mu_i$. The standard Poisson model has $\phi=1$ and, for an unweighted observation, $w_i=1$. Its Pearson residual is therefore $(y_i-\hat{\mu}_i)/\sqrt{\hat{\mu}_i}$. A scaled version is appropriate only when the model specifically uses a non-unit dispersion, as in a quasi-Poisson variance specification.

Worked Example: A Standard Poisson Residual

A fitted Poisson model gives $\hat{\mu}_i=4$ for an unweighted observation, and the observed count is $y_i=7$. The response residual is $7-4=3$, while the fitted variance is 4. Hence

r_{Pi}=\frac{7-4}{\sqrt{4}} =\frac{3}{2} =1.5.

The squared residual is $1.5^2=2.25$. It is the squared response discrepancy, $3^2=9$, divided by the fitted variance, $4$.

Worked Example: A Weighted, Scaled Poisson Variance

Suppose a model explicitly specifies $\operatorname{Var}(Y_i)=\phi V(\mu_i)/w_i$, with Poisson variance function $V(\mu_i)=\mu_i$. For one observation, let $y_i=11$, $\hat{\mu}_i=8$, $\hat{\phi}=1$, and variance weight $w_i=4$. Calculate the Pearson residual using this stated convention.

The specified fitted variance is

\widehat{\operatorname{Var}}(Y_i) = \frac{\hat{\phi}\hat{\mu}_i}{w_i} = \frac{1(8)}{4} =2.

The response residual is $11-8=3$, so

r_{Pi} = \frac{3}{\sqrt{2}} \approx 2.121.

This is the result for a variance weight of 4 under the stated formula. It should not be applied automatically to a frequency weight that represents four replicated counts; the meaning of the weight determines the variance used.

Squared Residuals and the Pearson Statistic

Squaring a Pearson residual expresses the discrepancy without its sign and in units of fitted variance. For observations with variances $\phi V(\mu_i)/w_i$, the corresponding squared contribution using fitted quantities is

\left(r_{Pi}^{(\phi)}\right)^2 = \frac{(y_i-\hat{\mu}_i)^2} {\hat{\phi}V(\hat{\mu}_i)/w_i} = \frac{w_i(y_i-\hat{\mu}_i)^2} {\hat{\phi}V(\hat{\mu}_i)}.

The sum of these squared residuals is often called the Pearson statistic (with the particular scaling convention made explicit). The algebra follows directly by squaring the residual definition; it does not require the observations to be normally distributed.

Result: With true means and variances, the expected squared standardized residual for each observation is 1. With fitted means or an estimated dispersion, the displayed squared residual is a fitted diagnostic contribution; it is not guaranteed to have expectation exactly 1.

Proof. For $Z_i=(Y_i-\mu_i)/\sqrt{\operatorname{Var}(Y_i)}$, the mean-zero property gives

E[Z_i^2] = \frac{E[(Y_i-\mu_i)^2]}{\operatorname{Var}(Y_i)} = \frac{\operatorname{Var}(Y_i)}{\operatorname{Var}(Y_i)} =1.

The denominator is finite and positive by assumption. Replacing $\mu_i$ or the variance with fitted quantities changes the random expression, so this exact expectation calculation no longer applies directly. $\square$

Common Mistake: Adding Dispersion to a Standard Binomial Model

A frequent error is to calculate a grouped-binomial variance as $\phi m_i p_i(1-p_i)$ and then call the result the variance of a binomial distribution for an arbitrary $\phi$. That is incorrect: a standard binomial variable has variance $m_i p_i(1-p_i)$, corresponding to $\phi=1$. A factor $\phi\ne1$ belongs only when the model specifies a scaled variance, such as a quasi-binomial model.

The correction is to identify the variance model before standardizing. Use $\operatorname{Var}(Y_i)=m_i p_i(1-p_i)$ for standard binomial data; use $\operatorname{Var}(Y_i)=\phi m_i p_i(1-p_i)$ only when the stated model includes that dispersion. Similarly, include weights only according to the variance convention they represent. These choices determine the denominator, and therefore the Pearson residual.

Key takeaway: A Pearson residual is a response residual divided by the model-based standard deviation of that response. Write down the response scale, variance function, dispersion, and weight convention first; then substitute fitted values and calculate.

Check Your Understanding

For each question, identify the variance specification before calculating or interpreting a residual.

A standard binomial model has $m_i=12$, $\hat{p}_i=0.25$, and $y_i=5$. What is the fitted count variance and the Pearson residual?
For the same grouped data, show that using the observed proportion and its standard binomial variance gives the same Pearson residual.
A quasi-Poisson model specifies $\operatorname{Var}(Y_i)=\phi\mu_i$. If $y_i=10$, $\hat{\mu}_i=6$, and $\hat{\phi}=2$, calculate the dispersion-scaled Pearson residual.
Under the convention $\operatorname{Var}(Y_i)=\phi V(\mu_i)/w_i$, what happens to the residual’s denominator as $w_i$ increases while other quantities remain fixed?
Why does the theorem about mean zero and variance one for a standardized response not imply that every fitted Pearson residual has exactly mean zero and variance one?