Why Compare Nested GLMs?
A generalized linear model (GLM) can be extended by adding predictors or terms. The larger model will generally fit the observed data at least as well as the smaller one, but a better fit alone does not establish that the added terms improve the model enough to justify their complexity. A nested-model likelihood-ratio test quantifies that improvement relative to the amount of model complexity added.
The central calculation is the difference between the reduced model’s deviance and the full model’s deviance. For an appropriate pair of models, this difference is a likelihood-ratio statistic whose null distribution is approximately chi-square. The comparison requires more than subtracting two reported numbers: the models must be genuinely nested, fit to the same response data, and compared under suitable likelihood and regularity conditions.
Deviance Difference and the Likelihood-Ratio Test
Let \(\ell_{\text{sat}}\) denote the maximized log-likelihood of the saturated model, which fits each observational unit as freely as the response structure permits. Let \(\ell_R\) and \(\ell_F\) be the maximized log-likelihoods for the reduced and full models. Deviance measures the model’s loss of fit relative to the saturated model:
Because the full model includes the reduced model as a special case, maximizing over the full model cannot produce a lower maximized log-likelihood. Thus \(\ell_F\geq \ell_R\), and correspondingly \(D_F\leq D_R\). The likelihood-ratio statistic for the comparison is:
This identity is the reason a deviance difference tests the added terms. The saturated-model log-likelihood cancels when the two deviances are subtracted. Under the null hypothesis and standard regularity conditions, \(G^2\) is approximately distributed as a chi-square random variable with degrees of freedom equal to the number of independent parameter restrictions.
The degrees of freedom are not automatically the number of named predictors. They are the number of independent parameters added: a factor with several levels may add several coefficients, while constraints, aliasing, or redundant terms may reduce the number of independent parameters. Count freely estimated parameters in each model on the same basis, including any intercept or nuisance parameters that differ.
Conditions for a Valid Comparison
Before calculating a p-value, verify that the comparison is meaningful. The reduced model must be obtainable by restrictions on the full model. Both models must use the same response observations, response definition, likelihood family, weights, and offsets. For example, comparing deviances from different subsets of observations does not isolate the effect of the additional terms.
The chi-square reference is an asymptotic result. It is most reliable when observations are independent under the model, parameters are identifiable, the null parameter values are not on problematic boundaries, and the likelihood is correctly specified. Sparse data, separation in logistic regression, or a variance component tested at zero can undermine the usual approximation. A small p-value is evidence against the restrictions tested, not proof that the full model is substantively correct.
For binomial and Poisson GLMs with their usual fixed dispersion convention, the deviance difference commonly supplies the likelihood-ratio statistic directly. If dispersion is estimated, take care: the raw difference in unscaled residual deviances need not be a chi-square likelihood-ratio statistic. In normal linear models with unknown variance, for example, the standard nested-model comparison is usually expressed as an \(F\)-test. For other families, identify the actual likelihood-ratio statistic and its scale treatment rather than applying the chi-square rule mechanically.
State the restrictions defining the null model and identify which terms are added in the full model.
Confirm that both fits use the same response data and compatible likelihood, weights, offsets, and dispersion convention.
Compute \(G^2=D_R-D_F\), then count the independent parameters added to obtain \(\nu\).
Use the upper-tail probability \(P(\chi^2_\nu\geq G^2)\), or compare \(G^2\) with a stated chi-square critical value.
Explain whether the added terms improve fit sufficiently to reject their joint null restrictions; do not claim that the test establishes overall model adequacy.
Worked Applications
Worked Example: Testing Three Added Terms in a Poisson GLM
A hypothetical insurer models the number of roadside-assistance calls per policy using independent Poisson responses and a log link. Both models use the same policy records and the same fixed exposure offset. The reduced model has 4 freely estimated regression parameters and deviance \(D_R=28.4\). The full model adds three independent slope parameters, has 7 freely estimated parameters, and deviance \(D_F=20.9\). At the 5% level, test whether the three added terms are jointly needed.
Method and hypotheses. Use the nested-model likelihood-ratio test. The null hypothesis sets all three added slope parameters to zero; the alternative allows them to vary. The responses are Poisson, the dispersion convention is fixed, and the models are fitted to the same observations with the same offset.
Test statistic. Subtract the full-model deviance from the reduced-model deviance:
Degrees of freedom. The full model adds \(7-4=3\) freely estimated parameters, so \(\nu=3\). Under the null, compare \(7.5\) with a \(\chi^2_3\) distribution. Its upper-tail probability is:
The 5% critical value is approximately \(7.815\), and \(7.5<7.815\). The p-value also exceeds \(0.05\), so do not reject the joint null hypothesis at the 5% level. The data do not provide sufficient evidence that these three added terms improve fit. This does not prove that their effects are exactly zero.
Check and common mistake avoided. The full model has the smaller deviance, as expected for nested models, and the parameter-count difference is three. Using a p-value for one or two degrees of freedom would be incorrect; the reference distribution must match the number of independent added parameters.
Worked Example: Comparing Logistic Models for Grouped Outcomes
A hypothetical clinic models the number of patients who return for follow-up among groups of patients with the same recorded characteristics. For group \(j\), the return count has distribution \(Y_j\sim\operatorname{Binomial}(m_j,p_j)\), independently across groups, and the logit link is used. Both models use the same groups and denominators \(m_j\). A reduced model has 3 freely estimated parameters and deviance \(31.6\); a full model that adds a two-parameter categorical factor has 5 parameters and deviance \(22.4\). Test the factor at the 5% level.
Method and hypotheses. The null hypothesis sets both factor contrasts to zero. The alternative allows both contrasts to vary. This is a grouped-binomial likelihood comparison with a common data set and fixed binomial dispersion.
Statistic and degrees of freedom.
For two degrees of freedom, the chi-square upper-tail probability is \(e^{-G^2/2}\). Therefore:
Since \(0.0101<0.05\), reject the null hypothesis. There is evidence that at least one of the two factor contrasts contributes to fit, conditional on the rest of this model. The test does not establish which contrast matters; that would require a separate, appropriately specified analysis.
Check and common mistake avoided. The factor contributes two independent parameters, so the test has 2 degrees of freedom, not 1 merely because the factor is a single named variable. The deviance falls from \(31.6\) to \(22.4\), consistent with the full model’s greater flexibility.
Worked Example: A One-Parameter Poisson Comparison Near the Decision Boundary
A hypothetical transit operator models incident counts by route using independent Poisson responses, a log link, and a fixed offset for route exposure. The reduced model has 6 parameters and deviance \(18.70\). A full model adds one prespecified slope, has 7 parameters, and deviance \(14.70\). Test the added slope at the 5% level.
Method. Under the null, the added slope is zero. Since one independent parameter is added and the fixed-dispersion Poisson likelihood is used for both fits, use a likelihood-ratio statistic with one degree of freedom.
Calculation.
For one degree of freedom, \(\chi^2_1\) is the square of a standard normal variable. Thus:
The p-value is below \(0.05\), so reject the null at the 5% level. The evidence is modest: the result is close to the decision threshold, so rounding the deviance difference prematurely could change the reported decision. Use the unrounded model output when available.
Check and common mistake avoided. The statistic is positive because the full-model deviance is lower. Do not confuse \(G^2=4.00\) with a one-sided normal statistic of 4; the chi-square upper tail with one degree of freedom is equivalent to a two-sided standard-normal tail beyond 2.
Common Exam Traps and Interpretation
- Reversing the subtraction. Use \(D_R-D_F\), not \(D_F-D_R\). A negative result signals a calculation or comparability problem for genuinely nested maximum-likelihood fits.
- Counting terms instead of parameters. A factor may add multiple independent contrasts. Use the difference in model ranks or freely estimated parameter counts.
- Ignoring the upper tail. Large positive deviance differences are evidence against the reduced model, so the p-value is \(P(\chi^2_\nu\geq G^2)\).
- Comparing fits on different data. If missingness causes the models to use different observations, their deviance difference does not isolate the added terms.
- Using chi-square automatically. Check the family and dispersion treatment. In particular, a standard \(F\)-comparison is generally appropriate for nested normal linear models with unknown variance; do not substitute an unscaled deviance difference into a chi-square test without justification.
- Overstating the result. Rejection says the restrictions are inconsistent with the data to the degree measured by the test. It does not guarantee predictive performance, causal meaning, or satisfactory overall fit.
Check Your Understanding
For each comparison, identify the statistic and degrees of freedom, then use the stated reference distribution and conditions to reach the requested conclusion.
- A hypothetical Poisson log-link model with fixed exposure offsets has reduced-model deviance \(40.2\) and 6 parameters. The nested full model has deviance \(34.2\) and 8 parameters. Find \(G^2\), degrees of freedom, and the 5% decision. (Answer: \(G^2=6.0\), \(\nu=2\), \(p=e^{-3}\approx0.0498\); reject at 5%.)
- For comparable nested Poisson GLMs, the reduced model has 4 parameters and deviance \(16.0\); the full model has 7 parameters and deviance \(11.5\). Find the chi-square p-value and decision at 5%. (Answer: \(G^2=4.5\), \(\nu=3\), \(p=e^{-2.25}(3.25)\approx0.3425\); do not reject.)
- Two comparable grouped-binomial logistic models have deviances \(25.7\) and \(20.1\), respectively. The full model adds two independent parameters. Find the p-value and decision at 5%. (Answer: \(G^2=5.6\), \(\nu=2\), \(p=e^{-2.8}\approx0.0608\); do not reject.)
- Two nested Poisson models are fitted to the same observations with the same offset and fixed dispersion. The reduced model has deviance \(12.8\), and the full model also has deviance \(12.8\); the full model adds two independent parameters. What is the test statistic and upper-tail p-value? (Answer: \(G^2=0\), \(\nu=2\), \(p=1.0000\).)
- A reduced normal linear model and a nested full normal linear model have unknown variance estimated from the data. Is it automatically valid to compare their raw deviance difference to a chi-square distribution? State the standard comparison. (Answer: No; the usual nested-model comparison is an \(F\)-test when variance is unknown.)