Pearson Residuals and Variance Scaling

Why Scale a GLM Response Residual?

A response residual is the observed response minus its fitted mean. Its size is in the response's original units: claims for a count model, dollars for a severity model, or dollars per exposure unit for a pure-premium model. But an error of 100 may be unusual when the fitted standard deviation is 20 and quite ordinary when it is 500. Pearson residuals address this by scaling each response residual by the standard deviation implied by the fitted model.

The variance model matters. In a Poisson model, variability increases with the fitted mean; in a Gamma model, it increases with the square of the mean; and in a Tweedie model, it follows a power of the mean. Therefore the same response error can produce different Pearson residuals under different distribution and dispersion assumptions. This tutorial is aligned with the CAS Exam 8 (Advanced Ratemaking) syllabus.

Definition: For observation $i$, let $Y_i$ be the response, $\mu_i=E[Y_i\mid\text{predictors}]$ its conditional mean, and $\hat{\mu}_i$ its fitted mean. If the assumed conditional variance is $\operatorname{Var}(Y_i\mid\text{predictors})=\phi V(\mu_i)$, the fitted Pearson residual is $$ r_{P,i}=\frac{y_i-\hat{\mu}_i}{\sqrt{\hat{\phi}V(\hat{\mu}_i)}}. $$ Here $V(\mu)$ is the variance function and $\hat{\phi}$ is the dispersion value used in fitting or evaluating the model. For a model with fixed dispersion, use that fixed value; for example, the ordinary Poisson model has $\phi=1$.

The numerator is the response residual. The denominator is its fitted standard deviation, not its fitted variance. Thus the Pearson residual is dimensionless. Its sign is the sign of the response error: positive means the observed response exceeds the fitted mean, and negative means it is below the fitted mean.

Why the Formula Uses a Standard Deviation

Suppose first that the model parameters are known, so that the conditional mean and variance are $\mu_i$ and $\phi V(\mu_i)$. Divide the response error by its standard deviation. The resulting squared quantity has conditional expectation one:

E\left[\left.\left(\frac{Y_i-\mu_i}{\sqrt{\phi V(\mu_i)}}\right)^2\right|\text{predictors}\right] = \frac{E[(Y_i-\mu_i)^2\mid\text{predictors}]}{\phi V(\mu_i)} = \frac{\operatorname{Var}(Y_i\mid\text{predictors})}{\phi V(\mu_i)} =1.

The first equality follows because the denominator is fixed conditional on the predictors. The second uses $E[Y_i\mid\text{predictors}]=\mu_i$, so the conditional mean squared deviation from $\mu_i$ is the conditional variance. This explains why dividing by a standard deviation puts errors from different fitted means onto a common scale. In practice, parameters are estimated, so fitted Pearson residuals need not have exactly this expectation or behave like independent standard normal values.

For a GLM written with variance $\phi V(\mu_i)$, the common variance functions are $V(\mu)=\mu$ for Poisson, $V(\mu)=\mu^2$ for Gamma, and $V(\mu)=\mu^p$ for Tweedie. Each formula still requires an appropriate response distribution or variance assumption; the formula is not a license to select whichever variance function gives a preferred residual.

Poisson Variance: Scale by the Fitted Count

For a Poisson count $N_i$, assume conditionally that $N_i\sim\operatorname{Poisson}(\mu_i)$. Then $E[N_i\mid\text{predictors}]=\mu_i$ and $\operatorname{Var}(N_i\mid\text{predictors})=\mu_i$, so $\phi=1$ and $V(\mu)=\mu$. The Pearson residual is the count error divided by the square root of the fitted expected count.

Worked Example: A Poisson Count With an Exposure Offset

A hypothetical portfolio cell has 0.75 policy-years of exposure. A fitted Poisson frequency model gives 0.8 expected claims per policy-year, and the cell has 2 observed claims. Calculate the Pearson residual. Assume independent Poisson counts conditional on the fitted rate, with no extra dispersion. No interest or mortality assumptions are relevant.

Step 1: Find the fitted count mean. The fitted frequency is a rate per policy-year, so multiply by the cell's exposure:

\hat{\mu}=0.75(0.8)=0.60\text{ claims}.

Thus the assumed count distribution for this cell is $N\sim\operatorname{Poisson}(0.60)$, with variance $0.60$.

Step 2: Find the response residual and its fitted standard deviation.

y-\hat{\mu}=2-0.60=1.40,\qquad \widehat{\operatorname{SD}}(N)=\sqrt{0.60}=0.7745967.

Step 3: Divide.

r_P=\frac{1.40}{0.7745967}=1.8074.

The positive value indicates more claims than fitted. As a check, squaring the unrounded residual gives $1.96/0.60=3.2667$, also equal to $1.8074^2$ up to rounding. The method avoids using the frequency rate itself as though it were the expected count.

Gamma Variance: Dispersion Multiplies the Squared Mean

For a positive continuous response under a Gamma GLM, the standard variance function is $V(\mu)=\mu^2$, giving $\operatorname{Var}(Y_i\mid\text{predictors})=\phi\mu_i^2$. The dispersion is essential: the fitted standard deviation is $\sqrt{\phi}\mu_i$, not simply $\mu_i$. One common equivalent parameterization is a Gamma distribution with shape $\alpha$ and scale $\theta$; its mean is $\alpha\theta$, its variance is $\alpha\theta^2$, and the GLM dispersion is $\phi=1/\alpha$. This assumes a positive response and a constant dispersion under the specified model.

Worked Example: Gamma Severity With a Fitted Dispersion

For a hypothetical severity class, an individual claim has observed severity $y=\$1{,}500$, fitted mean $\hat{\mu}=\$1{,}200$, and fitted dispersion $\hat{\phi}=0.25$. Assume a Gamma response with shape $\alpha=4$ and scale $\theta=\$300$, so the conditional mean is $\$1{,}200$. There is no discounting or mortality assumption. Calculate the Pearson residual.

Step 1: Verify the distribution parameters. For a Gamma shape-scale distribution,

E[Y]=\alpha\theta=4(300)=\$1{,}200,\qquad \operatorname{Var}(Y)=\alpha\theta^2=4(300^2)=\$360{,}000.

The GLM variance formula agrees: $\hat{\phi}\hat{\mu}^2=0.25(1{,}200^2)=0.25(1{,}440{,}000)=\$360{,}000$. The fitted standard deviation is $\sqrt{\$360{,}000}=\$600$.

Step 2: Compute the response residual.

y-\hat{\mu}=\$1{,}500-\$1{,}200=\$300.

Step 3: Scale by the fitted standard deviation.

r_P=\frac{\$300}{\$600}=0.5000.

The positive residual is half of the fitted standard deviation. As a check, $\$300^2/\$360{,}000=0.25=0.5000^2$. The method uses the specified Gamma dispersion as well as the squared mean.

Tweedie Variance: Use the Specified Power

A Tweedie GLM uses a power variance function $V(\mu)=\mu^p$, so its variance is $\phi\mu^p$. The value of $p$ is part of the model specification; it cannot be omitted when calculating a Pearson residual. For $1<p<2$, a Tweedie distribution has a compound Poisson-Gamma form, which is useful for responses such as aggregate loss per exposure that can be zero or positive. The mean, dispersion, and power together determine the variance. The residual calculation does not require separately recovering the compound distribution's Poisson and Gamma parameters.

Worked Example: Tweedie Pure Premium With a Noninteger Power

A hypothetical rating cell has observed pure premium $y=\$460$ per exposure unit. A fitted Tweedie model has mean $\hat{\mu}=\$400$, dispersion $\hat{\phi}=0.25$, and power $p=1.5$. Assume the conditional Tweedie variance is $\phi\mu^p$, with $1<p<2$. No interest or mortality assumptions are relevant. Calculate the Pearson residual.

Step 1: Calculate the fitted variance. Since $400^{1.5}=400\sqrt{400}=400(20)=8{,}000$,

\widehat{\operatorname{Var}}(Y)=0.25(400^{1.5})=0.25(8{,}000)=2{,}000.

The fitted standard deviation is $\sqrt{2{,}000}=\$44.7214$ per exposure unit.

Step 2: Calculate the response residual and scale it.

y-\hat{\mu}=\$460-\$400=\$60,\qquad r_P=\frac{60}{\sqrt{2{,}000}}=1.3416.

The observed pure premium is above the fitted mean by about 1.3416 fitted standard deviations. Squaring confirms the scale: $60^2/2{,}000=1.8=1.3416^2$, allowing for rounding. Using a Poisson variance or a Gamma variance here would substitute a different model for the specified Tweedie variance.

Exposure, Weights, and Common Traps

An exposure offset and a variance weight have different roles. An offset helps determine the fitted mean. For example, in the Poisson example, exposure converted a rate into an expected count. It did not mean that the residual denominator was the square root of the rate. The denominator followed the variance of the response actually modeled: the count with fitted mean $0.60$.

Some GLM implementations also allow prior or precision weights. Under the convention that an observation with weight $w_i$ has variance $\phi V(\mu_i)/w_i$, its Pearson residual is

r_{P,i}=\frac{y_i-\hat{\mu}_i}{\sqrt{\hat{\phi}V(\hat{\mu}_i)/w_i}}.

This weighted form applies only when the model defines the weight as changing the response variance in that way. An exposure used as an offset is not automatically such a weight. Always identify the modeled response and its variance before applying a weight adjustment.

Common exam trap: In the Poisson example, using $\sqrt{0.8}$ as the denominator gives the incorrect calculation $1.40/\sqrt{0.8}=1.5652$. The value $0.8$ is the fitted rate per policy-year, not the fitted mean count for the cell. The count mean is $0.75(0.8)=0.60$, so the correct denominator is $\sqrt{0.60}$ and the Pearson residual is $1.8074$. For Gamma, a related error is to omit $\phi$: that would use a standard deviation of $\$1{,}200$, rather than $\sqrt{0.25}(\$1{,}200)=\$600$.

Pearson residuals are useful because they scale response errors according to the variance assumptions of the chosen model. A residual is not, by itself, proof that a model is adequate or inadequate: interpretation depends on the observation, the fitted model, and how the residuals behave across the data. Also, a Pearson residual is not adjusted for the effect of estimating model parameters. It should not be treated as a leverage-adjusted residual or assumed to follow an exact standard normal distribution.

Name the response model.
Identify the response being modeled and the assumed variance function.

Find the fitted response mean.
Apply any offset or exposure conversion needed to put the fitted mean on the scale of the observed response.

Calculate the fitted variance.
Evaluate $\hat{\phi}V(\hat{\mu})$, including any applicable variance weight.

Divide the response error by its standard deviation.
Use $(y-\hat{\mu})/\sqrt{\widehat{\operatorname{Var}}(Y)}$, retain the sign, and round only after the division.

Check Your Understanding

For each problem, identify the variance assumption before calculating. Round final residuals to four decimal places.

A Poisson count model has exposure 1.25 policy-years and fitted rate 0.4 claims per policy-year. If 1 claim is observed and dispersion is fixed at 1, calculate the Pearson residual. (Answer: $0.7071$)
A positive severity response is Gamma with shape 6.25 and scale $\$160$. An observed claim is $\$900$. Calculate its Pearson residual using the model's implied mean and dispersion. (Answer: $-0.2500$)
A Tweedie response has observed value 0, fitted mean 100, dispersion 0.09, and power 1.5. Calculate the Pearson residual. (Answer: $-10.5409$)
An overdispersed Poisson count has observed value 7, fitted mean 4, and dispersion 1.5. Calculate the Pearson residual, including dispersion. (Answer: $1.2247$)
A Gamma response has observed value $\$1{,}300$, fitted mean $\$1{,}000$, and fitted Pearson residual $0.6000$. Using $\operatorname{Var}(Y)=\phi\mu^2$, infer the dispersion $\phi$. (Answer: $0.2500$)

What You'll Learn