Likelihood-Ratio Tests From Deviance Differences

Why Compare Deviances?

A generalized linear model (GLM) with additional predictors can usually fit the observed responses at least as well as a nested model with fewer predictors. The practical question is whether the improvement is large enough to count as evidence against the simpler model, rather than merely the expected gain from estimating additional parameters. A likelihood-ratio test answers this question by comparing the models’ maximized likelihoods.

For comparable GLMs, the reported deviance difference gives the same comparison without requiring the saturated model’s likelihood to be calculated separately. The key tasks are to identify which model is reduced, subtract deviances in the correct order, and count the additional freely estimated parameters. This tutorial focuses on calculating and interpreting that comparison. The conditions for using a chi-square reference distribution are a separate issue; the examples below specify the reference distribution to use.

Definition: Let $M_0$ be a reduced model and $M_1$ a full model, where $M_0$ is nested in $M_1$. Write $\ell_0$ and $\ell_1$ for their maximized log likelihoods, and $D_0$ and $D_1$ for their deviances, calculated for the same response data and likelihood convention. The likelihood-ratio statistic is $$ G^2=2(\ell_1-\ell_0)=D_0-D_1. $$ If the models have $p_0$ and $p_1$ freely estimated parameters, respectively, the usual degrees of freedom for the comparison are $p_1-p_0$, when the nesting restrictions are independent.

The null hypothesis says that the restrictions defining the reduced model are adequate: the additional parameters in the full model are zero or otherwise satisfy the specified restrictions. The alternative is that at least one restriction does not hold. Rejecting the null supports the full model’s added terms as a group; it does not establish that every added coefficient is individually important.

Why the Deviance Difference Is the Test Statistic

Deviance measures the gap between a fitted model and a saturated model, which fits the observed responses as closely as the likelihood allows. For the same response data and likelihood convention, both nested models are compared with the same saturated model. Consequently, the saturated-model contribution cancels when their deviances are subtracted.

Write each deviance.
For model $M_j$, $D_j=2(\ell_{\mathrm{sat}}-\ell_j)$, where $\ell_{\mathrm{sat}}$ is the saturated-model log likelihood and $\ell_j$ is the fitted model’s maximized log likelihood.

Subtract reduced from full in the likelihood comparison.
$D_0-D_1=2(\ell_{\mathrm{sat}}-\ell_0)-2(\ell_{\mathrm{sat}}-\ell_1)$.

Cancel the shared term.
$D_0-D_1=2(\ell_1-\ell_0)$, which is the likelihood-ratio statistic $G^2$.

This identity explains both the subtraction order and the expected sign. When maximum likelihood fitting is performed correctly and the models are nested, the full model can reproduce the reduced model by satisfying its restrictions. Its maximized likelihood therefore cannot be smaller, so $\ell_1\geq\ell_0$ and $D_0-D_1\geq0$. A materially negative difference is a warning to check which model is which, whether the models are actually nested, and whether the reported deviances are comparable.

A Practical Testing Procedure

Before calculating a test, verify that both models use the same response observations, response distribution, and likelihood convention. The reduced model must be obtainable from the full model by imposing restrictions on its parameters. A change in the data rows, response family, or likelihood scale can make a raw deviance subtraction inappropriate. Different predictor names alone do not establish nesting.

Count parameters, not merely predictor labels. For example, a categorical predictor with four levels typically contributes three coefficients when one level is the reference. An interaction can contribute several coefficients. An offset is specified rather than estimated, so it does not add a freely estimated coefficient. More generally, use the difference in the models’ parameter dimensions, accounting for constraints and rank.

For a decision, compare $G^2$ with the supplied critical value for the stated reference distribution, or use the corresponding p-value. In common regular large-sample settings, a likelihood-ratio statistic is compared with a chi-square distribution having degrees of freedom equal to the number of independent added parameters. That reference is not automatic for every GLM or data structure; the examples explicitly give the reference to use.

Worked Comparisons

Worked Example: Added Rating Factors in a Poisson Model

A hypothetical insurer models the number of roadside-assistance calls per policy using a Poisson GLM and the same 240 policies in both fits. The reduced model includes vehicle age. The full model adds two numerical predictors, annual mileage and region indicator, each with one coefficient. The fitted deviances are 18.6 for the reduced model and 10.2 for the full model. Use a chi-square reference with 2 degrees of freedom; the supplied 5% critical value is 5.991. Test whether the added terms improve fit.

Method and hypotheses. Use a likelihood-ratio test. The null hypothesis is that both added coefficients are zero; the alternative is that at least one is nonzero.

Calculate the statistic. The reduced model is $M_0$, so subtract the full-model deviance from the reduced-model deviance: $G^2=D_0-D_1=18.6-10.2=8.4$.

Degrees of freedom and decision. Two independent coefficients were added, so the comparison has 2 degrees of freedom, as specified. Since $8.4>5.991$, reject the null hypothesis at the 5% level. The data provide evidence that adding the terms improves fit as a group.

Check and common mistake avoided. The full model has the lower deviance, as expected, and the statistic is positive. Subtracting in the reverse order would give $-8.4$, which is not the likelihood-ratio statistic for this comparison.

Worked Example: Testing a Predictor in a Logistic Model

A hypothetical benefits administrator models whether an application is complete at first submission, using a binomial logistic GLM fitted to the same 180 applications. A reduced model includes application channel. A full model adds one coefficient for a reminder indicator. The deviances are 42.7 and 35.9, respectively. Use a chi-square reference with 1 degree of freedom and a 5% critical value of 3.841.

Method and hypotheses. The likelihood-ratio test compares the model with the reminder coefficient fixed at zero against the model estimating that coefficient. The null says the reminder term adds no effect on the model’s linear predictor; the alternative allows a nonzero coefficient.

Calculate the statistic. $G^2=42.7-35.9=6.8$. One freely estimated coefficient was added, so the degrees of freedom are $1$.

Decision and interpretation. Because $6.8>3.841$, reject the null at the 5% level. The full model provides evidence of improved fit. This test concerns the added term’s contribution to fit; it does not by itself establish a causal effect of reminders.

Check and common mistake avoided. The subtraction uses the reduced-model deviance first, and the degrees of freedom are one because the reminder indicator has one coefficient. Counting the intercept or the existing channel terms again would incorrectly count parameters that appear in both models.

Worked Example: Several Added Terms Do Not Clear the Threshold

A hypothetical water utility models the number of service interruptions per monitored site with a Poisson GLM. Both models use the same 96 sites. The reduced model includes exposure as an offset and a baseline location term. The full model adds a three-level equipment category represented by two coefficients and one seasonal indicator coefficient. The reduced and full deviances are 27.4 and 21.1. Use a chi-square reference with 3 degrees of freedom and a 5% critical value of 7.815.

Method and hypotheses. Test jointly whether the two equipment coefficients and the seasonal coefficient are all zero. The offset is fixed, not estimated, and contributes no parameter to this comparison.

Calculate the statistic. $G^2=27.4-21.1=6.3$. The full model adds $2+1=3$ freely estimated coefficients, giving 3 degrees of freedom.

Decision and interpretation. Since $6.3<7.815$, do not reject the joint null hypothesis at the 5% level. The added terms have reduced the deviance, but the reduction is not large enough for rejection at the stated threshold.

Check and common mistake avoided. There are three added coefficients, not two: the three-level category requires two coefficients and the seasonal indicator requires one. Counting the offset as a fourth estimated parameter would give the wrong reference degrees of freedom.

Common Traps and Interpretation

Reversing the difference. Use $D_{\mathrm{reduced}}-D_{\mathrm{full}}$, not the reverse. The statistic measures the full model’s gain in fit and should be nonnegative, apart from possible numerical rounding.
Counting predictor names rather than parameters. A multi-level factor can add several coefficients, while a fixed offset adds none. Determine the actual independent restrictions being tested.
Comparing nonnested models. A smaller deviance in one of two nonnested models does not make their difference a standard nested-model likelihood-ratio statistic.
Ignoring changed observations or likelihood conventions. Deviances from different samples or incompatible scales cannot generally be subtracted to form this test.
Overstating the conclusion. A significant joint test says the added terms improve fit as a group under the specified model comparison. It does not show that each term is significant, that the model is well calibrated, or that the relationship is causal.

Key takeaway: For nested GLMs fitted to the same data under a comparable likelihood, calculate $G^2=D_{\mathrm{reduced}}-D_{\mathrm{full}}$. Use the number of independent added parameters for the degrees of freedom, then make the decision using the reference distribution specified for the test.

Check Your Understanding

For each problem, identify the deviance difference, the added-parameter degrees of freedom, and the test decision using the stated reference.

A hypothetical Poisson model has reduced deviance 31.2 and full deviance 25.7 after adding one coefficient. Use a chi-square reference with 1 degree of freedom and critical value 3.841. (Answer: $G^2=5.5$, 1 degree of freedom; reject at 5%.)
Two nested logistic models have deviances 58.0 and 51.6. The full model adds a four-level factor represented by three coefficients. Use a chi-square reference with 3 degrees of freedom and critical value 7.815. (Answer: $G^2=6.4$, 3 degrees of freedom; do not reject at 5%.)
A Poisson model’s deviance falls from 44.5 to 35.0 when two independent coefficients are added. A test using 2 degrees of freedom has a supplied critical value of 5.991. What is the conclusion? (Answer: $G^2=9.5$, 2 degrees of freedom; reject at 5%.)
Two models use the same response observations. The reduced model has deviance 16.8; the full model has deviance 18.1 after adding two terms. What should be checked before reporting a likelihood-ratio test? (Answer: The computed difference is $16.8-18.1=-1.3$; check model nesting, fitting convergence or numerical issues, and whether the deviances use the same likelihood convention. A negative value is inconsistent with a correctly maximized nested full model apart from minor rounding.)
A gamma GLM comparison has reduced deviance 12.4 and full deviance 7.9, with two added coefficients. The problem states that the applicable test uses a chi-square reference with 2 degrees of freedom and critical value 5.991. Decide at 5%. (Answer: $G^2=4.5$, 2 degrees of freedom; do not reject at 5%.)

What You'll Learn