An Overdispersion Diagnostic Workflow and Report

Why Use a Workflow?

A large Pearson statistic or deviance ratio can be evidence of extra variation, but it does not identify its cause. A misspecified mean, dependence among observations, latent heterogeneity, sparse data, or a small number of influential cases can all produce a dispersion signal. A useful diagnostic therefore proceeds in a deliberate order: first check the mean model and residual patterns, then compare dispersion summaries, and finally examine dependence and influence before writing a conclusion.

The aim is not to find one threshold that settles the question. The Pearson and deviance statistics are summaries of discrepancies between observed responses and fitted means. Their interpretation depends on the model, the data structure, and the diagnostics around them. Where calibration is uncertain, simulation under the fitted model can help assess how unusual the observed statistics are.

Define the Checks Before Applying Them

Suppose $Y_i$ has fitted mean $\hat{\mu}_i$, for $i=1,\ldots,n$. In a Poisson model, the Pearson residual is the response discrepancy divided by its model-based standard deviation. In a grouped binomial model, the same idea uses the binomial variance. The Pearson statistic adds the squared Pearson residuals.

Definition: For independent Poisson responses, the Pearson residual and Pearson statistic are $$ r_{Pi}=\frac{y_i-\hat{\mu}_i}{\sqrt{\hat{\mu}_i}}, \qquad X_P^2=\sum_{i=1}^n r_{Pi}^2 =\sum_{i=1}^n\frac{(y_i-\hat{\mu}_i)^2}{\hat{\mu}_i}. $$ For independent grouped binomial responses $Y_i\sim\operatorname{Binomial}(m_i,p_i)$, replace the variance in the denominator by $m_i\hat p_i(1-\hat p_i)$, so that $$ X_P^2=\sum_{i=1}^n\frac{(y_i-m_i\hat p_i)^2}{m_i\hat p_i(1-\hat p_i)}. $$

The deviance compares the fitted model’s log-likelihood with the log-likelihood of a saturated model, which fits each observation as closely as possible. For independent Poisson data, its observation-level contributions are $d_i=2\{y_i\log(y_i/\hat{\mu}_i)-(y_i-\hat{\mu}_i)\}$. For grouped binomial data they are $d_i=2\{y_i\log[y_i/(m_i\hat p_i)]+(m_i-y_i)\log[(m_i-y_i)/(m_i(1-\hat p_i))]\}$. In both formulas, a term of the form $0\log(0/a)$ is defined to be zero.

Definition: The deviance is $D=\sum_i d_i=2\{\ell_{\mathrm{sat}}-\ell(\hat{\theta})\}$, where $\ell_{\mathrm{sat}}$ is the saturated-model log-likelihood and $\ell(\hat{\theta})$ is the fitted-model log-likelihood. A commonly examined descriptive summary is a dispersion ratio: $X_P^2/(n-p)$ or $D/(n-p)$, where $p$ is the number of fitted regression coefficients. These ratios are not automatic tests; their reference behaviour depends on the model, fitting, and data conditions.

Two Facts That Guide Interpretation

The first fact is simply the connection between residual contributions and the overall statistic. It is useful because a total alone can hide whether the evidence is spread across observations or dominated by a few.

Result: For either model above, the Pearson statistic is the sum of the squared Pearson residuals, and the deviance is the sum of the observation-level deviance contributions.

Proof. In the Poisson case, the definition $r_{Pi}=(y_i-\hat{\mu}_i)/\sqrt{\hat{\mu}_i}$ gives $r_{Pi}^2=(y_i-\hat{\mu}_i)^2/\hat{\mu}_i$. Summing over $i$ gives the displayed expression for $X_P^2$. In the grouped binomial case, the Pearson residual is $(y_i-m_i\hat p_i)/\sqrt{m_i\hat p_i(1-\hat p_i)}$, and squaring and summing gives its stated statistic. By definition, the total deviance is twice the fitted-versus-saturated log-likelihood difference. For independent observations, the log-likelihood is a sum of observation-level log-likelihood contributions. The saturated-versus-fitted difference therefore also sums observation by observation, yielding $D=\sum_i d_i$. $\square$

A second fact explains why Pearson statistics are often compared with residual degrees of freedom, while also showing why that comparison is only approximate after fitting.

Result: If the means $\mu_i$ are known, the responses are independent, and each response has variance $\mu_i$, then $E[X_P^2]=n$ for the Pearson statistic formed using those true means.

Proof. Define $r_{Pi}=(Y_i-\mu_i)/\sqrt{\mu_i}$. Since $E[Y_i]=\mu_i$ and $\operatorname{Var}(Y_i)=\mu_i$, $$ E[r_{Pi}^2] =\frac{E[(Y_i-\mu_i)^2]}{\mu_i} =\frac{\operatorname{Var}(Y_i)}{\mu_i} =1. $$ The statistic is a finite sum, so linearity of expectation gives $E[X_P^2]=\sum_{i=1}^n E[r_{Pi}^2]=n$. This argument assumes known means; it does not establish an exact expectation or a chi-squared distribution for a statistic using estimated means. Estimating regression coefficients changes the residual behaviour, and small or sparse data may further affect calibration. $\square$

A Sequence for Diagnosis

Use the following sequence to organize evidence. It is not a mechanical pass-or-fail test: a warning at one stage is a reason to investigate its explanation before moving to a more specific conclusion.

Check the response and mean structure.
Confirm the response type, exposure or binomial denominator, link, and included predictors. Look for a plausible functional form and missing structure. A systematic mean error can inflate residual variation.

Inspect residual patterns.
Plot residuals against fitted means, predictors, and relevant groupings. Look for trends, curvature, changing spread, clusters, or a small number of conspicuous residuals. A pattern points toward a model or data-structure issue, not automatically toward a single dispersion explanation.

Compare Pearson and deviance summaries.
Calculate $X_P^2$, $D$, and their ratios using the same fitted model. Compare the overall summaries with the observation-level contributions and with an appropriate reference or simulation calibration.

Check dependence and influence.
Ask whether observations share a subject, site, time, or other source of dependence. Examine whether one or a few cases dominate the statistics, and assess how refitting without them changes the results.

Report the evidence and its limits.
State the model, summaries, patterns checked, and material sensitivity findings. Distinguish evidence of excess variation from evidence about its cause, and name important remaining uncertainties.

Worked Diagnostic Examples

Worked Example: Comparing Pearson and Deviance Evidence

A Poisson regression with two fitted coefficients is used for four independent counts. The fitted means and observed counts are shown below. This example focuses on calculating and comparing the residual summaries, not on estimating the regression coefficients.

Observation	$y_i$	$\hat{\mu}_i$
1	2	2
2	4	3
3	3	4
4	9	6

For the Pearson statistic, the four contributions are $0$, $(4-3)^2/3=1/3$, $(3-4)^2/4=1/4$, and $(9-6)^2/6=9/6=1.5$. Thus $$ X_P^2=0+\frac13+\frac14+1.5=2.0833. $$

The Poisson deviance contributions are $0$, $2[4\log(4/3)-1]\approx0.30146$, $2[3\log(3/4)+1]\approx0.27391$, and $2[9\log(9/6)-3]\approx1.29837$. Therefore $D\approx1.87374$. With $n-p=4-2=2$, the descriptive ratios are $2.0833/2\approx1.0417$ and $1.87374/2\approx0.9369$. These values are close to one another and do not, by themselves, give a strong signal of excess variation. The last observation contributes most to both totals, so its residual and influence deserve attention even though neither overall ratio is large.

Worked Example: A Grouped Binomial Check

Suppose three independent groups each contain $m_i=10$ trials, and a binomial model with a common fitted probability is used. The observed numbers of successes are $1,2,3$. The fitted probability is the pooled proportion, $\hat p=(1+2+3)/(10+10+10)=6/30=0.2$, so each fitted success count is $10(0.2)=2$.

The variance for each group is $10(0.2)(0.8)=1.6$. The Pearson contributions are $(1-2)^2/1.6=0.625$, $0$, and $(3-2)^2/1.6=0.625$. Hence $X_P^2=1.25$.

For the deviance, the group with two successes contributes zero because its observed count equals its fitted count. The other contributions are $2[\log(1/2)+9\log(9/8)]\approx0.73380$ and $2[3\log(3/2)+7\log(7/8)]\approx0.56335$. Thus $D\approx1.29715$. There are $n=3$ grouped observations and $p=1$ fitted coefficient, so the ratios are $1.25/(3-1)=0.625$ and $1.29715/(3-1)\approx0.6486$. These summaries do not suggest excess variation in this small example; with only three groups, however, they provide limited diagnostic information.

Worked Example: Interpreting a Pattern, Dependence, and Influence

A Poisson regression with two coefficients is fitted to counts from ten patients at five clinics. The analyst obtains $X_P^2=12.0$ and $D=11.4$. With $n-p=10-2=8$, the ratios are $12.0/8=1.50$ and $11.4/8=1.425$. Residuals plotted against fitted means have no clear curve, but residuals within several clinics tend to have the same sign. The analyst therefore checks whether clinic-level dependence or an omitted clinic effect could explain the pattern; the ratios alone cannot distinguish these possibilities.

One clinic has a large residual contribution. After removing that clinic’s two observations and refitting the same model, the reported Pearson statistic is $6.1$. The refit has $n=8$, $p=2$, and hence $n-p=6$, giving a new ratio $6.1/6\approx1.02$. A fitted coefficient changes from $0.55$ to $0.18$ on this refit. The reduction in the ratio and the coefficient change show that the initial evidence is sensitive to those observations. They do not establish that the clinic should be excluded: its data and measurement process should be checked, and the dependence concern remains. A fair report presents the original and sensitivity results rather than choosing whichever result supports a preferred conclusion.

What a Useful Report Includes

A concise report should let another analyst understand what was checked and why the conclusion is appropriately qualified. Include the response model and the observation unit; whether the data were individual or grouped; the fitted mean structure; Pearson and deviance statistics and the denominator used for their ratios; and the main residual patterns. If simulation was used to calibrate a dispersion check, state the simulation setup and the resulting evidence rather than treating a generic cutoff as universal.

If diagnostics identify dependence or influential cases, report how those findings were assessed and whether conclusions changed under a justified sensitivity analysis. Do not describe a high ratio as proof of a particular cause, or a ratio near one as proof that the model is correct. A model can have an acceptable overall ratio while still showing a structured residual pattern, and a large ratio can arise from a mean-model problem rather than unexplained extra variation.

Common Mistake

Wrong reasoning: “The Pearson ratio is above one, so the data are overdispersed; I can report that as the cause of the poor fit.” This treats a descriptive statistic as both a definitive test and an explanation.

Correction: First inspect the fitted mean and residual patterns, then compare Pearson and deviance evidence, and then check dependence and influential observations. State what the diagnostics support, what alternative explanations remain plausible, and how sensitive the conclusion is to reasonable refits or calibration choices.

Key takeaway: Overdispersion diagnosis is a sequence of checks, not a single ratio. A defensible conclusion brings together the mean model, residual patterns, Pearson and deviance evidence, dependence, influence, and the uncertainty that remains after those checks.

Check Your Understanding

Use the workflow to explain both the calculation and the limits of a dispersion conclusion.

Why should residual patterns be examined before interpreting a large Pearson ratio as evidence of extra variation?
For independent Poisson responses with known means, explain why the expected Pearson statistic is $n$. Why does this result not give an exact reference distribution after fitting regression coefficients?
What can be learned by comparing Pearson and deviance contributions observation by observation rather than comparing only their totals?
If residuals from patients at the same clinic tend to have the same sign, what alternative explanation should be investigated before attributing the pattern to overdispersion?
A dispersion ratio falls substantially after a few observations are removed. What should be reported, and why is this not by itself a reason to discard those observations?

Observation	\(y_i\)	\(\hat{\mu}_i\)
1	2	2
2	4	3
3	3	4
4	9	6