Interpreting Residual Deviance and GLM Goodness of Fit

What Residual Deviance Can Tell You

A GLM output may report a residual deviance and residual degrees of freedom. Comparing these quantities is a compact way to ask whether the fitted model leaves substantially more lack of fit than would be expected under its assumptions. The comparison is useful, but it is not a diagnosis: a single number cannot reveal which predictor, functional form, or distributional assumption may be responsible.

The key is to interpret residual deviance relative to the model, response family, and observation structure. A ratio near one can be reassuring in some settings, but it does not prove that a model is adequate. A large ratio can be a warning, but it does not by itself identify a particular defect.

Definition: The residual deviance $D$ is twice the difference between the maximized log-likelihood of the saturated model and that of the fitted model: $$ D=2\{\ell_{\mathrm{sat}}-\ell_{\mathrm{fit}}\}. $$ The saturated model fits the observed response in each observational unit as closely as the response family allows. The residual degrees of freedom are usually $n-r$, where $n$ is the number of independent observational units represented in the likelihood and $r$ is the rank of the fitted design matrix. An offset is not an estimated coefficient and does not increase $r$.

The design-matrix rank counts estimable coefficients, not merely the number of terms written in a formula. For example, a categorical predictor with four levels represented by an intercept and three indicator columns contributes four columns in total, provided they are linearly independent. If a column is redundant, it does not add to the rank.

Why Compare Deviance With Its Degrees of Freedom?

Under regularity conditions, a likelihood-ratio statistic comparing a correctly specified fitted model with a saturated model has an approximate chi-square distribution. Its degrees of freedom are the difference in model dimensions. For a GLM with $n$ independent response units and design-matrix rank $r$, the saturated model has roughly one free fitted mean per unit, so that difference is $n-r$. This motivates comparing $D$ with a chi-square distribution with $n-r$ degrees of freedom.

The approximation is not an exact identity. It works best when the likelihood-based large-sample approximation is credible and the data structure supports it. Sparse counts, fitted probabilities near zero or one, small numbers of groups, or individual binary outcomes can make the reference distribution unreliable. The chi-square comparison is therefore evidence to weigh, not a mechanical pass-or-fail rule.

\text{residual deviance per residual degree of freedom} = \frac{D}{n-r}.

For some fixed-dispersion GLMs, such as standard Poisson and binomial models, a ratio around one is often a convenient rough reference: the deviance is on a scale whose expected value is approximately its degrees of freedom under a suitable model and approximation. A ratio far above one can indicate lack of fit or extra variation relative to the assumed model. A ratio below one is not proof of exceptional fit; random variation and approximation error can produce it.

For families with an estimated dispersion parameter, interpretation needs an additional qualification. In a Gamma GLM, for example, the variance is commonly written $\operatorname{Var}(Y_i)=\phi\mu_i^2$. The relevant scaled deviance is related to $D/\phi$; comparing the raw $D$ directly with residual degrees of freedom as though $\phi=1$ can be misleading. The family and dispersion convention must be identified before interpreting the ratio.

Identify the observational units.
Use the independent units represented in the likelihood. For grouped binomial data, these are generally the groups, not the total number of trials across groups.

Find the design-matrix rank.
Count the linearly independent estimated coefficient directions. Do not count an offset as an estimated coefficient.

Calculate residual degrees of freedom and the ratio.
Use $n-r$, then calculate $D/(n-r)$ with the reported residual deviance.

Check whether the reference is credible.
Consider the family, dispersion treatment, amount of data, grouping, and whether sparse outcomes weaken the chi-square approximation.

State a bounded conclusion.
Describe whether the summary raises concern, then note that it does not locate the source of any misfit.

Worked Examples

Worked Example: Grouped Binomial Goodness of Fit

A hypothetical insurer groups policyholders into 12 risk cells. In each cell, the response is the number of claims among 100 policies. A logistic model has four estimable coefficients, including the intercept, and its reported residual deviance is 21.6. Assume the groups are independent and the fitted probabilities are not close to zero or one. Calculate the residual degrees of freedom and assess the summary.

Method. For grouped binomial data, count the 12 independent groups as the observational units. The total of 1,200 policies is not the number of rows in the grouped likelihood for this residual-df calculation.

The model rank is $r=4$, and the number of groups is $n=12$. Therefore, $$ \text{residual df}=n-r=12-4=8. $$

The deviance-to-df ratio is $$ \frac{D}{\text{residual df}}=\frac{21.6}{8}=2.7. $$

This ratio is substantially above one, so it raises concern about fit under the assumed binomial model. If the chi-square approximation is reasonable, the upper-tail probability for a chi-square statistic of 21.6 on 8 degrees of freedom is approximately 0.0057. That would be evidence against treating the observed deviance as typical under that approximation. However, there are only 12 groups, so the asymptotic reference still deserves caution. The summary does not say whether the concern comes from an omitted predictor, an inadequate link or mean structure, dependence between groups, or another issue.

Check. The calculation uses groups, not individual policies: $12-4=8$, and $21.6/8=2.7$. A common mistake would be to use $1{,}200-4$ degrees of freedom, which does not match the grouped likelihood units.

Worked Example: A Poisson Model With an Exposure Offset

A hypothetical analyst models annual claim counts for 30 independent territories using a Poisson GLM with a log link. The model has five estimable coefficients, and a log-exposure offset is included. The reported residual deviance is 22.4. Interpret the deviance relative to its residual degrees of freedom.

Method. The offset adjusts the mean for exposure but is not an estimated coefficient. Thus $n=30$ and $r=5$.

The residual degrees of freedom are $$ 30-5=25. $$ The deviance-to-df ratio is $$ \frac{22.4}{25}=0.896. $$

The ratio is close to one, and the deviance is slightly below its residual degrees of freedom. If the Poisson assumptions and chi-square approximation are reasonable, this summary does not raise the same concern as a ratio far above one. It does not establish that the model is correct: for example, a deviance summary cannot rule out a particular omitted pattern or a problematic observation.

Check. Counting the offset as a sixth coefficient would give $30-6=24$ degrees of freedom and $22.4/24\approx0.9333$, but that is the wrong rank. The offset is fixed by the exposure information; it is not estimated from the response.

Worked Example: Individual Binary Responses and a Tempting Ratio

A logistic regression is fitted to 200 individual binary outcomes, with four estimable coefficients. Software reports residual deviance 183.4. Calculate the nominal residual degrees of freedom and ratio, then explain why a ratio near one is not enough to conclude that fit is adequate.

Method. For the individual-response likelihood, $n=200$ and the model rank is $r=4$. The nominal residual degrees of freedom are $$ 200-4=196. $$

The ratio is $$ \frac{183.4}{196}=0.935714\ldots\approx0.9357. $$ The result is below one and numerically close to it.

The numerical comparison alone is not decisive. With individual binary data, the saturated model gives one fitted probability per observation, and the usual chi-square goodness-of-fit approximation can be unreliable. The ratio therefore cannot certify adequate fit. A candidate should report the calculation accurately, qualify the reference approximation, and avoid claiming that the summary identifies—or rules out—a specific source of misfit.

Check. Rounding to four decimal places gives 0.9357. The incorrect value 0.9367 does not follow from dividing 183.4 by 196.

What the Summary Cannot Diagnose

Residual deviance compresses the discrepancy between observed outcomes and the fitted model into one total. Different problems can produce similar totals. A high value could reflect an inadequate mean structure, an unsuitable response family, dependence, extra variation, or a small number of unusual contributions. A value near its degrees of freedom can also coexist with a localized issue that is offset by other observations contributing less deviance.

This is why a goodness-of-fit statement should distinguish evidence about overall fit from explanation of the fit problem. Residual deviance can flag a concern, especially when its scale is large relative to the appropriate reference. It cannot tell you which observation or structure caused the concern. The later task of choosing residuals and studying their behavior requires additional information; it is not contained in the deviance-to-df ratio.

Common Exam Traps

Using the wrong number of units. In grouped binomial data, use the groups represented by the likelihood, not the total number of trials. In individual binary data, the rows are the individual response units, though the chi-square approximation may be weak.
Counting an offset as a coefficient. A fixed offset changes the linear predictor but does not increase the fitted model rank.
Treating $D/\text{df}\approx1$ as proof. It is a rough reference, not a universal acceptance criterion. The response family and data structure matter.
Assuming a large ratio names the problem. It indicates a discrepancy relative to a reference, not its cause.
Ignoring dispersion. For a model with estimated dispersion, raw deviance may not be directly comparable with residual df as if the dispersion were one.

Key takeaway: Calculate residual degrees of freedom from the likelihood units and fitted model rank, then interpret residual deviance in the context of the family and the quality of the chi-square approximation. The ratio is a compact warning signal, not a complete goodness-of-fit diagnosis.

Check Your Understanding

For each problem, calculate the requested quantities before reading the answer in parentheses.

A grouped binomial model uses 15 independent cells and has six estimable coefficients. Its residual deviance is 18.0. Find its residual degrees of freedom and deviance-to-df ratio. (Answer: 9 residual degrees of freedom; ratio $18.0/9=2.000$.)
A Poisson model has 40 independent observational units, rank 7, residual deviance 30.5, and a fixed exposure offset. Find the residual degrees of freedom and ratio. (Answer: 33 residual degrees of freedom; $30.5/33\approx0.9242$.)
A grouped binomial data set has 800 trials arranged in 20 independent groups. A model has rank 5 and residual deviance 36.0. Which count determines the usual residual degrees of freedom, and what is the ratio? (Answer: Use 20 groups; residual df $20-5=15$; ratio $36.0/15=2.4$.)
A logistic model for 150 individual binary responses has rank 3 and residual deviance 142.2. Calculate the nominal ratio and state one qualification. (Answer: Residual df $150-3=147$; ratio $142.2/147\approx0.9673$; the usual chi-square approximation for individual binary outcomes may be unreliable, so this ratio alone does not establish adequate fit.)
A Poisson model uses 25 independent units, has rank 4, includes a fixed offset, and reports residual deviance 42.0. A candidate says the ratio is $42/(25-5)=2.1$ because the offset counts as a parameter. Correct the calculation and explain what the result can establish. (Answer: The offset is not an estimated coefficient, so residual df $=25-4=21$, and the ratio is $42/21=2.0$; this raises concern relative to the rough Poisson reference but does not identify the source of misfit.)

What You'll Learn