Pearson's Chi-Square for GLM Fit

Why Use Pearson's Goodness-of-Fit Statistic?

A generalized linear model (GLM) describes a response through a fitted mean and a specified variance function. The Pearson goodness-of-fit statistic summarizes how far the observed responses lie from those fitted means, measured relative to the response variability the model expects. A discrepancy of one unit matters differently when the response variance is small than when it is large, so the variance assumption is essential to the calculation.

The statistic is useful as an overall summary of fit, but it is not a guarantee that the model is appropriate. Its interpretation relies on assumptions about the response distribution, mean structure, variance, and residual degrees of freedom. This tutorial develops the calculation and its approximate reference distribution; it does not treat the statistic as a substitute for examining the model in context.

Definition: For independent responses \(Y_1,\ldots,Y_n\) with fitted means \(\hat{\mu}_i\) and GLM variance function \(V(\mu_i)\), the unscaled Pearson goodness-of-fit statistic is

X_P^2=\sum_{i=1}^{n}\frac{(y_i-\hat{\mu}_i)^2}{V(\hat{\mu}_i)}.

The \(i\)th summand is the squared Pearson residual when the residual is defined as \(r_i^P=(y_i-\hat{\mu}_i)/\sqrt{V(\hat{\mu}_i)}\). If \(\operatorname{Var}(Y_i)=\phi V(\mu_i)\), then \(\phi\) is the dispersion parameter: the statistic standardized using the full variance is \(X_P^2/\phi\).

The Variance Denominator and the Reference Distribution

The denominator is the model's variance function evaluated at the fitted mean. It is not the squared fitted mean by default, nor is it the observed response. For a Poisson response, \(V(\mu)=\mu\) and \(\phi=1\). For a binomial count \(Y_i\) out of \(m_i\) trials with success probability \(p_i\), \(\mu_i=m_ip_i\) and \(V(\mu_i)=m_ip_i(1-p_i)\), again with \(\phi=1\). For a gamma response, a common parameterization has \(V(\mu)=\mu^2\) and \(\phi>0\), so the variance is \(\phi\mu^2\).

With \(p\) fitted regression parameters and a full-rank model matrix, the residual degrees of freedom are ordinarily \(n-p\), where \(n\) is the number of independent response observations (rows). Under a correctly specified model and suitable large-sample conditions, \(X_P^2/\phi\) is approximately chi-square with \(n-p\) degrees of freedom. In the special case \(\phi=1\), this gives the familiar approximation \(X_P^2\sim\chi^2_{n-p}\). If \(\phi\) is unknown, the ratio \(X_P^2/(n-p)\) is often used as an estimate of dispersion; it is not itself a proof that the model is correct.

This chi-square calibration is approximate, not an exact finite-sample result. It is most useful when the response distribution and variance function are appropriate, observations are independent, the mean model is correctly specified, parameters are estimated regularly, and the data contain enough information for the approximation to be credible. Sparse counts or very small samples can make a reference p-value unreliable. For grouped binomial data, \(n\) counts the independent groups, not the total number of trials.

Derivation: The degrees-of-freedom adjustment follows from the local weighted least-squares form of GLM fitting. After standardizing response deviations by their variances, fitting \(p\) parameters uses the data to estimate \(p\) directions of systematic variation. Locally, the remaining residual vector is approximately a projection of a standardized error vector onto a space of dimension \(n-p\). If the standardized errors have mean zero and variance \(\phi\), the expected squared length of that residual projection is approximately \(\phi(n-p)\). Thus \(E[X_P^2]\) is approximately \(\phi(n-p)\), and when the chi-square approximation is suitable, \(X_P^2/\phi\) has approximately \(n-p\) degrees of freedom. This argument is approximate because GLM variances and weights depend on fitted means, and the residual projection need not behave exactly like a fixed projection in a small sample.

Worked Calculations

Worked Example: Poisson Responses With a Known Unit Dispersion

A hypothetical insurer models the number of roadside assistance calls for four independent service territories using a Poisson GLM. The model has two fitted regression parameters, including the intercept, and produces fitted means \((4,4,5,7)\) calls. The observed counts are \((3,5,4,8)\). Calculate the Pearson statistic, its residual degrees of freedom, and the approximate reference distribution.

Method and assumptions. For a Poisson response, \(Y_i\sim\operatorname{Poisson}(\mu_i)\), so \(E[Y_i]=\mu_i\), \(\operatorname{Var}(Y_i)=\mu_i\), and \(\phi=1\). The observations are assumed independent, and the stated fitted means come from the specified two-parameter model.

For territory 1, the squared Pearson residual is \((3-4)^2/4=1/4=0.25\). For territory 2, it is \((5-4)^2/4=1/4=0.25\). For territory 3, it is \((4-5)^2/5=1/5=0.20\). For territory 4, it is \((8-7)^2/7=1/7\approx0.142857\).

Adding the four contributions gives \(X_P^2=0.25+0.25+0.20+0.142857=0.842857\), or \(0.8429\) to four decimal places. There are \(n=4\) response observations and \(p=2\) fitted parameters, so the residual degrees of freedom are \(4-2=2\). Because \(\phi=1\), the approximate reference is \(\chi^2_2\).

Check and interpretation. Each contribution is nonnegative, and the total is at least as large as its largest contribution, \(0.25\). The statistic is \(0.8429\) on only two residual degrees of freedom. The sample is too small to treat the approximate reference as decisive evidence about the model. The common mistake avoided here is dividing each squared difference by \(\hat{\mu}_i^2\): a Poisson variance is \(\hat{\mu}_i\), not \(\hat{\mu}_i^2\).

Worked Example: Grouped Binomial Responses and Residual Degrees of Freedom

A hypothetical safety program records the number of successful equipment checks in four independent groups, each containing 12 checks. A logistic model with an intercept and one slope gives fitted success probabilities \(1/5,1/3,1/2,\) and \(2/3\); these probabilities correspond, for example, to fitted logits that increase by \(\log 2\) between consecutive groups. The observed success counts are \(2,5,7,\) and \(9\). Calculate the Pearson statistic and residual degrees of freedom.

Method and assumptions. For group \(i\), \(Y_i\sim\operatorname{Binomial}(m_i,p_i)\), with independent groups, \(m_i=12\), mean \(\mu_i=m_ip_i\), and variance \(m_ip_i(1-p_i)\). There are four response observations and two fitted parameters.

The fitted means are \(12(1/5)=12/5=2.4\), \(12(1/3)=4\), \(12(1/2)=6\), and \(12(2/3)=8\). The corresponding variances are \(12(1/5)(4/5)=48/25=1.92\), \(12(1/3)(2/3)=8/3\), \(12(1/2)(1/2)=3\), and \(12(2/3)(1/3)=8/3\).

The four contributions are therefore \((2-2.4)^2/1.92=0.16/1.92=1/12\approx0.083333\), \((5-4)^2/(8/3)=3/8=0.375\), \((7-6)^2/3=1/3\approx0.333333\), and \((9-8)^2/(8/3)=3/8=0.375\). Their sum is \(1/12+3/8+1/3+3/8=14/12=7/6\approx1.1667\), to four decimal places.

The residual degrees of freedom are \(4-2=2\), and the binomial dispersion is \(\phi=1\). Thus the approximate reference is \(\chi^2_2\). The key trap is using \(48-2=46\) degrees of freedom because there are 48 individual checks. In this grouped model, the four group counts are the four response observations; the number of trials affects each variance, not the count of fitted response rows.

Worked Example: A Gamma Variance Function With Known Dispersion

A hypothetical model predicts positive repair costs in thousands of dollars using a gamma GLM. For three independent claims, the observed costs are \(12,9,\) and \(10\) thousand dollars, and the fitted means are \(10,12,\) and \(8\) thousand dollars. Assume the gamma variance is \(\operatorname{Var}(Y_i)=\phi\mu_i^2\) with known \(\phi=0.5\), and the model has two fitted parameters. Calculate the unscaled Pearson statistic and the statistic standardized by the full variance.

Method and assumptions. For the specified gamma model, \(V(\mu_i)=\mu_i^2\) and the full variance is \(0.5\mu_i^2\). First calculate the unscaled statistic using \(V(\hat{\mu}_i)\); then divide by \(\phi\) for the statistic calibrated to the full variance.

For the first claim, the unscaled contribution is \((12-10)^2/10^2=4/100=0.04\). For the second, it is \((9-12)^2/12^2=9/144=0.0625\). For the third, it is \((10-8)^2/8^2=4/64=0.0625\). Hence \(X_P^2=0.04+0.0625+0.0625=0.165\).

The full-variance-standardized statistic is \(X_P^2/\phi=0.165/0.5=0.33\). With \(n=3\) responses and \(p=2\) parameters, the residual degrees of freedom are \(3-2=1\), so the approximate reference for the standardized statistic is \(\chi^2_1\). All monetary values use the stated unit of thousands of dollars, so the same unit cancels from each squared residual divided by its squared-mean variance function.

Check and interpretation. The full-variance statistic is twice the unscaled statistic because \(\phi=0.5\). This example avoids the mistake of silently treating a gamma model's dispersion as one. With just one residual degree of freedom, the approximation should be interpreted cautiously.

Common Traps and Practical Interpretation

Using the wrong variance. The contribution is a squared difference divided by variance, not by standard deviation. For Poisson data the denominator is \(\hat{\mu}_i\); for binomial counts it is \(m_i\hat{p}_i(1-\hat{p}_i)\); for the stated gamma parameterization it is \(\hat{\mu}_i^2\) in the unscaled statistic.
Mixing up scaled and unscaled statistics. If the variance is \(\phi V(\mu)\), the unscaled sum \(X_P^2\) uses \(V(\hat{\mu}_i)\). The sum based on the full variance is \(X_P^2/\phi\). State which quantity is being compared with a chi-square reference.
Using the wrong number of degrees of freedom. Use the number of independent response rows minus the rank of the fitted model matrix. In grouped binomial data, do not substitute the total number of trials for the number of response rows.
Reading a summary as a verdict. A value near the residual degrees of freedom is consistent with the expected scale under the model assumptions, but does not establish that the mean structure, independence, or distribution is correct. A large or small value may warrant further investigation; it does not identify the cause by itself.

The Pearson statistic differs from residual deviance: Pearson's calculation uses squared response deviations divided by the fitted variance function, whereas deviance is based on a likelihood comparison with a saturated model. Their values need not match. Both summarize overall discrepancies, and neither removes the need to understand the model's assumptions.

Key takeaway: Calculate each Pearson contribution with the variance function appropriate to the response, sum over independent response observations, and interpret the result using the correct dispersion and residual degrees of freedom. The chi-square comparison is approximate and depends on the GLM assumptions and the amount of information in the data.

Check Your Understanding

For each problem, use the stated variance model and count response rows—not individual trials—when determining residual degrees of freedom.

A Poisson model has observed counts \(2,6,5\), fitted means \(3,5,5\), and two fitted parameters. Calculate \(X_P^2\) and the residual degrees of freedom. (Answer: \(X_P^2=1/3+1/5+0=0.5333\); 1 residual degree of freedom.)
Two independent binomial groups each have 10 trials. Their observed success counts are \(3,7\), and their fitted probabilities are \(0.4,0.6\). A model has one fitted parameter. Calculate the unscaled Pearson statistic and residual degrees of freedom. (Answer: \(X_P^2=1/6+1/6=1/3\approx0.3333\); 1 residual degree of freedom.)
A gamma GLM uses \(V(\mu)=\mu^2\) and known \(\phi=2\). One response is \(15\), its fitted mean is \(12\), and the model has \(n=5\) response rows and \(p=2\) parameters. Find this response's contribution to the unscaled statistic, the full-variance-standardized contribution, and the residual degrees of freedom. (Answer: \(9/144=0.0625\); \(0.0625/2=0.03125\); 3 residual degrees of freedom.)
A grouped binomial model has three independent rows, each with 20 trials, and estimates an intercept and slope. Its unscaled Pearson statistic is \(2.4\), with \(\phi=1\). State the residual degrees of freedom and the approximate reference distribution. (Answer: \(3-2=1\) residual degree of freedom; approximately \(\chi^2_1\), not \(\chi^2_{60-2}\).)
A gamma model has \(X_P^2=4.8\), known dispersion \(\phi=1.2\), 12 independent response rows, and a model matrix of rank 4. Calculate the statistic standardized by the full variance and state its approximate reference distribution. (Answer: \(4.8/1.2=4.0\); approximately \(\chi^2_8\), since \(12-4=8\).)

What You'll Learn