Why a Poor Fit Needs More Than One Explanation
A generalized linear model (GLM) can fit poorly for different reasons. Its mean structure may be wrong: a predictor, nonlinear term, interaction, or other systematic feature of the response has been omitted or incorrectly specified. Alternatively, the mean may be reasonably specified while the responses vary more than the model’s assumed variance allows. The first problem is lack of fit in the mean structure; the second is overdispersion.
These problems can occur together, and a large residual statistic alone does not identify which one is present. A useful diagnosis combines residual patterns against predictors or fitted values, a dispersion estimate, and—when the models are nested—a comparison of their fitted mean structures. This tutorial is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.
What the Pearson Dispersion Estimate Can Tell You
In a Poisson GLM, the variance assumption is \(\operatorname{Var}(Y_i\mid x_i)=\mu_i\), so \(V(\mu_i)=\mu_i\) and the Pearson contribution is \((y_i-\hat{\mu}_i)^2/\hat{\mu}_i\). In a binomial GLM with \(m_i\) trials and success probability \(\pi_i\), the grouped response \(Y_i\) has mean \(m_i\pi_i\) and variance \(m_i\pi_i(1-\pi_i)\). Its Pearson contribution uses that entire binomial variance.
For a correctly specified model with independent observations and the assumed variance, \(\hat{\phi}_P\) is often expected to be near 1, subject to sampling variability and the adequacy of the approximation. A substantially larger value is evidence of extra variation relative to the assumed variance. It is not proof of a particular cause. Dependence, unmodeled heterogeneity, outliers, and an incorrect mean structure can all contribute to a large statistic.
The residual degrees of freedom are an approximation to the information remaining after parameter estimation. They are not a universal guarantee that \(X_P^2\) follows a chi-square distribution exactly. Sparse counts, small samples, influential observations, and poorly estimated means can make chi-square reference calculations unreliable. Treat the dispersion ratio as a diagnostic, and examine residuals and the design of the data as well.
Why Residual Patterns Point Toward Mean Misspecification
A mean model describes the expected response as a function of predictors. If that function omits a systematic feature, residuals can tend to be positive in some regions and negative in others. For example, a linear predictor may systematically underpredict at both ends of a predictor’s range and overpredict in the middle. Increasing the assumed variance does not, by itself, correct that pattern: it changes the scale assigned to residuals, not the fitted means.
A derivation explains why Pearson residuals are scaled as they are. If the conditional mean is \(\mu_i\) and the variance is \(\phi V(\mu_i)\), then the response error \(Y_i-\mu_i\) has variance \(\phi V(\mu_i)\). Dividing that error by its standard deviation gives a quantity with variance 1:
The ordinary Pearson residual omits \(\sqrt{\phi}\), because the GLM variance function is commonly written without a dispersion multiplier and \(\phi\) is then estimated from the residuals. Under a suitable fit, the squared standardized residuals should therefore have an average on the order of 1 after allowing for fitted parameters. A structured residual pattern, however, is a different kind of evidence: it suggests the conditional means themselves may be systematically wrong.
Worked Example: A Count Model With a Directional Residual Pattern
A hypothetical service center records claim counts at five branches with the same exposure. A Poisson GLM with an intercept only predicts the same expected count at each branch. The observed counts are 2, 4, 6, 8, and 10. Assess the residual evidence, using the stated model assumptions.
Method and assumptions. Assume independent Poisson counts with a common mean under the intercept-only model. The fitted mean is the sample average: \(\hat{\mu}=(2+4+6+8+10)/5=30/5=6\). There is one estimated coefficient, so the residual degrees of freedom are \(5-1=4\).
The raw residuals, observed count minus fitted count, are \(-4,-2,0,2,4\). Their signs change in order with branch index: the model overpredicts the first two counts, matches the middle count, and underpredicts the last two. That ordered pattern would be relevant if branch index represented a meaningful predictor or arrangement; it is not the appearance expected from unsystematic scatter alone.
For a Poisson model, \(V(\hat{\mu})=\hat{\mu}=6\). Thus the Pearson statistic is \((16+4+0+4+16)/6=40/6=6.6667\). The estimated dispersion is \(6.6667/4=1.6667\), or 1.67 to two decimals.
Conclusion. The residual ordering is evidence to investigate a missing mean trend. The dispersion estimate is also above 1, but it does not establish that extra-Poisson variation is the primary problem. A model that accounts for the ordered mean pattern should be assessed before attributing all the excess Pearson statistic to overdispersion.
Check and common mistake. The squared raw residuals sum to 40, but \(40\) is not the Pearson statistic: each Poisson contribution is divided by the fitted mean 6. Using 40 directly would overstate the statistic.
Use Nested Models to Test a Mean-Structure Change
A model is nested in another when the smaller model can be obtained by setting one or more coefficients in the larger model to specified values, usually zero. For example, a model with an intercept and a linear predictor is nested in a model that adds a quadratic term. When the response distribution and dispersion assumptions are appropriate, the difference in deviances between nested models provides evidence about whether the added terms improve fit.
When there is evidence of overdispersion, a plain Poisson or binomial likelihood-ratio calibration can be too optimistic: it treats the specified variance as correct. A common approximate adjustment for an extra-dispersion setting is to divide the deviance reduction by an estimated \(\phi\), then compare the scaled result with a chi-square reference. In quasi-likelihood practice, an approximate \(F\)-comparison may instead divide the scaled reduction per added parameter by a residual dispersion estimate. These are approximations, not exact distributional results, and the chosen dispersion estimate should be stated.
In particular, do not interpret a large deviance reduction as a clean mean-structure test without considering the variance assumption. Conversely, an elevated Pearson ratio in a reduced model can partly reflect its omitted mean terms. Refit the expanded model, inspect its residuals and dispersion, and distinguish what the evidence supports from what it cannot establish.
Worked Example: Excess Pearson Variation in Grouped Binary Data
A hypothetical study divides policy applications into six groups, with 20 applications in each group. The fitted intercept-only binomial model estimates a success probability of 0.50 in every group. The observed success counts, in the pre-specified group order, are 2, 18, 3, 17, 10, and 10. Calculate the Pearson statistic and dispersion estimate, then explain what they establish.
Method and assumptions. Treat the six grouped responses as independent binomial counts, each with \(m_i=20\) trials and fitted probability \(\hat{\pi}_i=0.50\). Under this model, each fitted success count is \(20(0.50)=10\), and each variance is \(20(0.50)(0.50)=5\). There are six grouped observations and one fitted coefficient, so the residual degrees of freedom are \(6-1=5\).
The observed-minus-fitted deviations are \(-8,8,-7,7,0,0\). The four nonzero squared contributions sum to \(64+64+49+49=226\). The two zero residuals contribute \(0^2+0^2=0\), so including all six groups still gives a sum of 226. Therefore, \(X_P^2=226/5=45.2\), and \(\hat{\phi}_P=45.2/5=9.04\).
Conclusion. The statistic is very large relative to the residual degrees of freedom, indicating substantial variation beyond the fitted binomial variance under this intercept-only model. Because the group order is specified, the alternating residuals do not display a simple monotone trend in that order. Still, the large dispersion estimate alone cannot distinguish unmodeled group effects from extra-binomial variation. Relevant predictors or group structure should be considered before concluding that the response variance itself is inflated.
Check and common mistake. The sum of squared deviations is 226, not the Pearson statistic: each group’s squared deviation must be divided by its binomial variance of 5. Also, the zero residuals are included in the six-group calculation, although they add nothing. Omitting their zero contributions leaves the sum at 226; it does not make that sum incorrect.
Read the Three Sources of Evidence Together
The most informative diagnosis comes from asking separate questions. Do residuals show a systematic pattern against a predictor, fitted value, or meaningful ordering? Is the Pearson dispersion materially above 1 after fitting the proposed mean model? Does adding a defensible set of nested mean terms produce a meaningful deviance reduction, and what does the expanded model’s residual behavior look like?
| Evidence | What it can suggest | What it cannot establish by itself |
|---|---|---|
| Structured residual pattern | The fitted means may omit a systematic feature. | That overdispersion is absent, or which added term is correct. |
| Large Pearson dispersion estimate | Observed variation exceeds the model’s assumed variance relative to its residual degrees of freedom. | Whether the cause is a wrong mean, dependence, heterogeneity, or another source. |
| Large nested-model deviance reduction | The added mean terms improve fit under the stated likelihood assumptions. | That ordinary likelihood-ratio calibration remains valid under substantial overdispersion. |
Worked Example: Interpreting a Nested Count-Model Comparison
An analyst fits two Poisson GLMs to the same hypothetical count data. The reduced model has two coefficients, residual deviance 48, and Pearson statistic 56. The full model adds two coefficients and has residual deviance 31 and Pearson statistic 34. There are 30 independent observations. Assess the evidence, without treating a dispersion correction as exact.
Method and assumptions. The models are nested, use the same observations and Poisson response specification, and differ by two added mean parameters. The reduced model has \(30-2=28\) residual degrees of freedom; the full model has \(30-4=26\).
The reduced-model dispersion estimate is \(56/28=2.00\). The full-model estimate is \(34/26=1.3077\), or 1.31 to two decimals. The deviance reduction is \(48-31=17\), on \(2\) degrees of freedom. Under the ordinary Poisson likelihood assumptions, the unadjusted comparison uses a chi-square reference with 2 degrees of freedom.
If the reduced model’s dispersion estimate is used for a rough scaled comparison, the adjusted deviance reduction is \(17/2.00=8.50\). This is still substantial relative to 2 degrees of freedom, so the added mean terms appear to improve fit even after that approximate adjustment. The full model’s Pearson ratio is lower than the reduced model’s, but remains above 1; some excess variation may remain, or the ratio may reflect sampling variability and model features not captured here.
Conclusion. There is evidence that the added terms improve the mean structure, and the expanded fit has less apparent dispersion. These summaries do not prove that all lack of fit has been removed or identify the cause of the remaining ratio. Residual plots and the validity of the Poisson variance assumption still matter.
Check and common mistake. The degrees of freedom for the comparison are the two added coefficients, not the full model’s 26 residual degrees of freedom. Comparing 17 directly with a chi-square value having 26 degrees of freedom would answer a different question.
Common Exam Traps
- Calling every large Pearson ratio overdispersion. An omitted mean pattern can inflate the statistic. Examine residuals against relevant predictors and revisit the mean model.
- Calling every residual pattern overdispersion. A pattern in the conditional mean is not repaired just by multiplying the variance. Consider a suitable mean-structure change.
- Using the wrong denominator. Pearson contributions divide squared errors by the GLM variance function, not by 1 or by a common arbitrary number. For grouped binomial data, use \(m_i\hat{\pi}_i(1-\hat{\pi}_i)\).
- Using the wrong degrees of freedom. The residual dispersion estimate uses the number of observations minus the number of estimated coefficients. A nested comparison uses the number of added parameters.
- Applying an ordinary likelihood-ratio reference without checking assumptions. If variance is substantially larger than assumed, the usual reference distribution can be misleading. Any scaling adjustment is approximate and should be identified as such.
- Treating a ratio as a diagnosis. A ratio above 1 is evidence about fit relative to a variance assumption, not a proof of a unique mechanism.
Check Your Understanding
For each problem, distinguish evidence about the mean structure from evidence about dispersion.
- A Poisson GLM has 18 observations, 3 estimated coefficients, and Pearson statistic 30. Calculate the residual degrees of freedom and Pearson dispersion estimate. (Answer: 15 residual degrees of freedom; \(\hat{\phi}_P=2.00\).)
- In a grouped-binomial model, one group has 12 trials, fitted success probability 0.25, and observed successes 6. Calculate its Pearson contribution. (Answer: \((6-3)^2/[12(0.25)(0.75)]=4\).)
- Two nested models differ by 3 coefficients. Their deviances are 62 and 50. State the deviance reduction and comparison degrees of freedom. (Answer: \(\Delta D=12\) on 3 degrees of freedom.)
- A reduced Poisson model has Pearson statistic 40 on 20 residual degrees of freedom. A nested model adds two terms and reduces deviance by 14. Using the reduced-model dispersion estimate for a rough scaled comparison, calculate the adjusted deviance reduction. (Answer: \(\hat{\phi}_P=2\), so the adjusted reduction is \(14/2=7\), on 2 added-parameter degrees of freedom.)
- A binary GLM has a large Pearson dispersion estimate, while residuals alternate above and below zero across an ordered predictor. What two explanations should be investigated, and why is the ratio alone insufficient? (Answer: Investigate a missing systematic mean effect and extra-binomial variation; the ratio measures excess variation relative to the fitted model but does not identify its cause.)