Interpreting Pearson and Deviance Disagreements

Why Two Goodness-of-Fit Statistics Can Disagree

Pearson and deviance statistics summarize the discrepancy between observed responses and fitted means, but they measure that discrepancy on different scales. When most observations are close to their fitted means and those means are away from the edges of their possible ranges, the statistics often tell a similar story. A few unusual observations or fitted means near a boundary can make them diverge sharply.

The useful diagnostic question is not only whether the two totals are large, but which observations contribute to each total and why. In particular, a rare outcome can be many variance units away from its fitted mean while producing a more moderate deviance contribution. Looking at the contributions helps distinguish that pattern from many small discrepancies spread across the data.

Definitions and Observation-Level Contributions

Consider independent responses \(Y_1,\ldots,Y_n\) from a GLM, with fitted means \(\hat{\mu}_i\). For the formulas below, assume unit weights and write \(V(\mu_i)\) for the variance function, so that \(\operatorname{Var}(Y_i)=\phi V(\mu_i)\) when the model includes a dispersion parameter \(\phi\). The unscaled Pearson statistic below uses \(V(\hat{\mu}_i)\), not \(\phi V(\hat{\mu}_i)\).

A saturated model fits each observation exactly. Thus the deviance contribution measures the loss in log likelihood from using the fitted mean instead of that exact fit. For Poisson observations,

and \(d(0,\mu)=2\mu\). The convention \(0\log(0/\mu)=0\) makes this formula continuous at \(y=0\). For a Bernoulli observation \(y\in\{0,1\}\) with fitted success probability \(p\in(0,1)\), the Pearson and deviance contributions are

where zero-times-log terms are interpreted as zero. These are observation-level quantities. In grouped data, the formulas and the contribution attached to a row depend on the response representation; that distinction matters when interpreting a particular data table.

Why the Measures Usually Agree Locally

For a regular exponential-dispersion GLM with a positive variance function \(V\), its unit deviance has the representation

This formula applies when the integral is defined between the observed response \(y\) and the mean \(\mu\); the orientation of the integral gives a nonnegative value on either side of \(\mu\). It follows, for example, by differentiating the unit deviance with respect to \(\mu\): its derivative is \(2(\mu-y)/V(\mu)\), and its value at \(\mu=y\) is zero.

To verify the result, put \(\delta=y-\mu\) and change variables in the integral using \(t=\mu+u\delta\). This gives

By continuity and positivity of \(V\) at \(\mu\), \(1/V(\mu+u\delta)\) converges uniformly over \(0\leq u\leq1\) to \(1/V(\mu)\) as \(\delta\to0\). Therefore the integral converges to

Substitution yields \(d(y,\mu)/\delta^2\to1/V(\mu)\), which proves the expansion. The Pearson contribution is exactly \(\delta^2/V(\mu)\), so the two contributions have the same leading term. The interior and continuity conditions are important: this local approximation does not promise close agreement for large residuals or near a boundary where \(V(\mu)\) approaches zero. \(\square\)

Rare Outcomes and Boundary Fitted Means

A Bernoulli observation makes the boundary issue especially clear. If \(y=1\), then

As \(p\) approaches zero, the fitted mean is near the boundary for a success. The Pearson contribution grows at order \(1/p\), while the deviance grows only at order \(\log(1/p)\). Their ratio diverges: \(X^2/d(1,p)=(1-p)/\{2p\log(1/p)\}\to\infty\). The same pattern occurs for an observed failure when \(p\) approaches one: writing \(q=1-p\), the contributions are \(p/(1-p)\) and \(-2\log(1-p)\), respectively, and their ratio also diverges.

A Poisson example shows the same mechanism. When \(y=1\) and \(\mu=0.001\), the outcome is rare under the fitted mean. The two contributions are

The calculations do not mean that deviance is a universal correction for rare observations, or that Pearson should be ignored. They show that the two statistics encode different magnitudes for an observation that is far from a small fitted mean. In practice, inspect the response, fitted mean, and both contributions before deciding what the discrepancy indicates.

How to Interpret a Disagreement

Pearson and deviance goodness-of-fit statistics are commonly compared with reference distributions under specified large-sample conditions, as discussed in the earlier material on residual degrees of freedom and reference distributions. A difference between their values is not, by itself, a formal test comparing two models or a rule that one statistic is correct and the other is not. Their reference approximations also need not be equally reliable in every data configuration.

When the conclusions differ, begin with the observation-level contributions. A large Pearson statistic relative to deviance may be driven by one or more responses far from small fitted means, especially rare binary outcomes or counts. Several moderate discrepancies can also add up differently because Pearson squares standardized residuals, whereas deviance reflects a log-likelihood difference. The residual and fitted-mean context matters; the totals do not identify a cause on their own.

Common Mistake

A common mistake is to conclude that the statistic with the larger numerical value must provide the more trustworthy fit assessment, or that the smaller one shows the model fits adequately. This reasoning ignores both the distinct scales of the contributions and the conditions behind the reference distributions. A Pearson total dominated by a single rare outcome may tell a different diagnostic story from a deviance total spread across many observations.

The correction is to examine the contribution-level pattern, then interpret it in the context of the response type, fitted means, and applicable reference assumptions. Similar statistics for small interior residuals are expected from their common second-order approximation. A substantial disagreement can be informative, but it is a prompt for diagnosis, not a verdict by itself.