Why a Workflow Matters
A deviance test is not justified merely because software reports two deviances. Before subtracting them, establish that the models are nested, that the fits use the same observations and likelihood, and that the difference in model dimensions matches the restrictions being tested. Only then does the numerical difference have the intended likelihood-ratio interpretation.
The workflow below is for a regular nested comparison. Its final reference distribution is generally an asymptotic result, not a guarantee that every small-sample or boundary problem has a chi-square calibration. Such qualifications matter: a carefully computed statistic can still be paired with an unsuitable reference distribution.
Write \(\ell_R\) and \(\ell_F\) for the maximized log-likelihoods under the reduced and full models, respectively. If both are fits to the same observed data under the same likelihood, nesting implies \(\ell_F\geq\ell_R\): the full model can always reproduce every fit available to the reduced model. The likelihood-ratio statistic is \(2(\ell_F-\ell_R)\).
The Comparison, Step by Step
Identify the restrictions defining the reduced model and state the alternative as parameter values in the full model that do not satisfy those restrictions.
Check that the response observations, sampling assumptions, likelihood, offsets, and any conditioning choices are common to both fits. Also check that the reduced model is genuinely contained in the full model.
Determine the dimensions of the identifiable parameter spaces and subtract: \(q=d_F-d_R\). For regular nested models this is the number of independent restrictions.
Subtract maximized log-likelihoods, or subtract comparable deviances in the correct order. The statistic must be nonnegative apart from numerical optimization error.
For a regular likelihood-ratio test, compare the statistic with a chi-square distribution on \(q\) degrees of freedom. State the assumptions and the conclusion in context.
For a common saturated-model reference, define the likelihood deviance of model \(M\) as \(D_M=2(\ell_{\mathrm{sat}}-\ell_M)\), where \(\ell_{\mathrm{sat}}\) is the maximized log-likelihood for the same data in a saturated model. This definition uses the same likelihood scale for both models. Some GLM software reports a unit deviance or a scaled deviance; when a dispersion parameter changes the likelihood-ratio scale, use the appropriate scaling rather than treating every reported deviance difference as an unscaled likelihood-ratio statistic.
To verify the identity, substitute the definition of deviance for each model:
The saturated log-likelihood cancels precisely because it is common. If the reference likelihood or the observations differ, this cancellation does not establish a likelihood-ratio test. \(\square\)
Because \(A\) has rank \(q\), the rank-nullity theorem gives \(\dim\{\beta:A\beta=0\}=p-q\). If \(A\beta=b\) has a solution \(\beta_0\), every solution has the form \(\beta_0+v\), where \(v\) lies in the null space of \(A\). This affine solution set therefore also has dimension \(p-q\), and the full-minus-reduced dimension is \(p-(p-q)=q\). Shared nuisance parameters add the same dimension to both models and do not change the difference. \(\square\)
Under the regularity conditions of Wilks' theorem, the statistic has an approximate \(\chi^2_q\) distribution under \(H_0\) as the sample size increases. The usual conditions include an identifiable, smooth likelihood; a true null parameter in the interior of the relevant parameter space; nonsingular information; a fixed finite number of parameters; and a regular nested null of codimension \(q\). Boundary parameters, nonidentifiability, or sparse data can invalidate this reference. The degrees of freedom are not automatically the number of terms printed in a model formula: they are the number of independent restrictions between the identifiable models.
Worked Comparisons
Worked Example: A Group Effect in a Grouped-Binomial GLM
Two groups each contribute 10 independent binary outcomes. The unexposed group has 4 successes and the exposed group has 8. Compare a reduced logistic model with a common success probability \(p\) against a full model with separate probabilities \(p_0\) and \(p_1\). The response data and grouped-binomial likelihood are the same in both fits.
The hypotheses are \(H_0:p_0=p_1\) against \(H_1:p_0\ne p_1\). The reduced estimate pools the successes, so \(\hat p=12/20=0.6\). In the full model, \(\hat p_0=4/10=0.4\) and \(\hat p_1=8/10=0.8\). These estimates are interior, and the full model includes the reduced model when its two probabilities are equal.
The binomial coefficients are identical across the two models and cancel in the log-likelihood difference. Omitting those common terms, the maximized log-likelihoods are
The reduced model has one parameter and the full model has two, so \(q=2-1=1\). The statistic is
Equivalently, this is the reduced-minus-full deviance difference when both deviances use the same saturated binomial likelihood. Against \(\chi^2_1\), the asymptotic upper-tail \(p\)-value is approximately \(0.063\). At the 0.05 level, this test does not reject equality of the group probabilities. This conclusion describes the test evidence; it does not establish that the probabilities are equal.
Worked Example: Equal-Rate Test for Three Poisson Counts
Suppose three independent Poisson counts are \(Y_1=2\), \(Y_2=4\), and \(Y_3=6\). Compare a reduced model with one common mean \(\mu\) against a full model with three separate means \(\mu_1,\mu_2,\mu_3\). Both models use all three counts and the product of their Poisson likelihoods.
The hypotheses are \(H_0:\mu_1=\mu_2=\mu_3\) against the full model with at least one unequal mean. Under the reduced model, the common-mean maximum-likelihood estimate is the total count divided by the number of observations, \(\hat\mu=12/3=4\). Under the full model the estimates are \((2,4,6)\). The parameter dimensions are 1 and 3, so \(q=2\).
For independent Poisson observations, \(\ell=\sum_i[y_i\ln(\mu_i)-\mu_i-\ln(y_i!)]\). The factorial terms are common and cancel. Since both fitted models have total fitted mean 12, the log-likelihood difference reduces to
Therefore \(G^2=2(1.04650)=2.09300\). The \(\chi^2_2\) upper-tail probability is \(\exp(-2.09300/2)\), approximately \(0.351\). Under the regular asymptotic calibration, the data do not provide strong evidence against a common Poisson mean. The calculated statistic is nonnegative, as it must be for nested maximum-likelihood fits.
Worked Example: Why Fits on Different Observations Cannot Be Subtracted
Consider independent Poisson counts \(2,4,6\). A common-mean fit to all three has \(\hat\mu=4\). Suppose a separate-mean fit is mistakenly made only to the first two counts, giving estimates \(2\) and \(4\). These are not nested fits to a common likelihood: one likelihood contains three Poisson observations and the other contains two.
The all-three reduced-model log-likelihood, including the factorial terms, is
The two-observation separate-means log-likelihood is
Subtracting these values and comparing the result with a chi-square distribution is invalid. The observations and likelihoods differ, so the subtraction is not a likelihood-ratio statistic for the stated three-count null. To make a valid comparison on the first two counts, refit both models to those same observations. The common-mean estimate is then \((2+4)/2=3\), while the separate estimates remain 2 and 4. The valid statistic is
That first-two-count comparison has one restriction and hence one degree of freedom. The computation is valid because both fits now use the same two observations and likelihood. It answers a different question from the intended comparison using all three counts.
Reporting and Common Pitfalls
A concise report should let a reader reconstruct the inferential comparison. Name the models or restrictions, state the statistic and its degrees of freedom, give the reference distribution and \(p\)-value, and interpret the result in context. Also identify any important calibration qualification, such as use of a large-sample approximation. If the likelihood scale involves dispersion, state the scale used for the statistic and why the selected reference is appropriate.
- Overlapping hypotheses. The alternative is not simply “the parameter is in the full model” when the null is contained there. State \(H_1\) as the full-model parameter space excluding the null.
- Subtracting incomparable deviances. Deviances must be based on the same data and compatible likelihood scale. Check observations, response definition, weights or offsets, and likelihood assumptions.
- Counting terms instead of restrictions. Use the dimension difference, accounting for redundant or aliased parameters. A restriction contributes one degree of freedom only when it is independent.
- Reversing the subtraction. Use reduced minus full deviance, equivalently \(2(\ell_F-\ell_R)\). A negative result beyond rounding signals a fit, nesting, or optimization problem.
- Treating the chi-square approximation as automatic. Wilks' reference requires regularity. Small samples, sparse data, or boundary hypotheses may need a different calibration.
Check Your Understanding
For each comparison, identify the checks that establish whether the reported test is justified.
- A full model has five identifiable parameters and its reduced model imposes two independent interior linear restrictions. What are the asymptotic degrees of freedom, and what additional conditions are needed for a chi-square reference?
- Why is \(H_1:\theta\in\Theta_F\) incomplete when \(\Theta_R\subseteq\Theta_F\)? State the disjoint alternative.
- Two models use the same observations and likelihood, and their deviances use a common saturated reference. Which subtraction gives the likelihood-ratio statistic?
- A model fit omits observations that are included in the other model. Can the two reported deviances be subtracted for a standard nested likelihood-ratio test? Explain.
- A deviance difference is negative by a substantial amount even though the reduced model is said to be nested in the full model. Give two checks to make before reporting a test.