Why Likelihood-Ratio Tests Require Nested Hypotheses

Why Nesting Matters

A likelihood-ratio statistic compares the best fit allowed by a null hypothesis with the best fit allowed by a larger model. The word larger matters: if every distribution allowed by the null is also allowed by the alternative, the two maximizations have a fixed ordering. That ordering gives the statistic its usual nonnegative form and is part of the setting in which Wilks' theorem supplies a chi-square calibration.

For two models that are not nested, one may still calculate and compare their maximized likelihoods. But the comparison is not automatically the usual likelihood-ratio test. There is no general containment-based ordering, and subtracting the two maxima does not, on its own, produce a statistic with a chi-square distribution. The issue is not whether the arithmetic can be done; it is whether the hypotheses and assumptions justify the proposed reference distribution.

Nested Models and the Likelihood-Ratio Statistic

In a regular parametric model, the null is often expressed as a restriction on a parameter vector: \(\theta\in\Theta_0\subseteq\Theta_1\). For example, an unrestricted model may estimate a probability \(p\), while the null fixes it at a specified value. The restricted fit and unrestricted fit use the same observed data and likelihood, but maximize over different parameter sets.

Proof. Every parameter value permitted by the null is also permitted by the alternative. Thus the null maximizes over a subset of the alternative's allowable values. A supremum over a subset cannot exceed the supremum over the full set, so \(L_0\leq L_1\). Dividing by the positive \(L_1\) gives \(\Lambda\leq1\), while likelihoods are nonnegative, so \(\Lambda\geq0\). For \(L_0>0\), taking logarithms gives \(\log\Lambda\leq0\), hence \(T=-2\log\Lambda\geq0\). If \(L_0=0\), the statistic is interpreted as infinite. \(\square\)

This ordering is a structural consequence of nesting; it is not a feature that must be checked after observing the data. For regular nested hypotheses, Wilks' theorem states that under the null, \(T\) converges in distribution to a chi-square random variable with degrees of freedom equal to the number of independent restrictions. The standard result requires, among other conditions, an identifiable model, an interior true parameter value, suitable differentiability and information conditions, and a smooth restriction of the stated rank. Boundary points, nonidentifiability, and other failures of regularity can invalidate that calibration.

What Changes for Nonnested Models

If neither model contains the other, their maximized likelihoods have no guaranteed order. One model may fit the observed data better than the other, but reversing which model is subtracted reverses the sign of the log-likelihood difference. As a result, there is no automatically defined nonnegative deviance with the same null-versus-expanded-model interpretation.

This does not mean that likelihoods cannot be compared, or that no test can compare nonnested hypotheses. It means that a test and its calibration require a specified testing problem and a distributional argument of their own. For two simple hypotheses, for instance, a likelihood ratio can be used in a Neyman–Pearson test with a designated null and alternative. That fact does not make the statistic a Wilks chi-square statistic.

A useful way to understand why a likelihood comparison can favor either model is to examine its expectation under a specified data-generating distribution. The following inequality gives that expectation a precise interpretation.

Proof. For any \(t>0\), \(\log t\leq t-1\). Apply this with \(t=q(x)/p(x)\) wherever \(p(x)>0\). Multiplying by \(p(x)\) and integrating gives \(\int p(x)\log\{q(x)/p(x)\}\,dx\leq\int q(x)\,dx-\int p(x)\,dx=0\). Changing the sign yields \(\int p(x)\log\{p(x)/q(x)\}\,dx\geq0\). Equality in \(\log t\leq t-1\) holds only at \(t=1\); therefore equality in the integral requires \(p=q\) almost everywhere. For independent observations \(X_1,\ldots,X_n\), the log likelihood ratio is the sum of the \(n\) individual log ratios. Linearity of expectation then gives \(nE_P[\log\{p(X)/q(X)\}]\). If \(P\) assigns positive probability to a region where \(q=0\), the expected log ratio is infinite rather than a finite negative value. \(\square\)

This result explains why the direction of a nonnested comparison depends on which distribution generates the data: the expected log likelihood ratio favors the model closer to that distribution in this particular divergence sense. But it does not give a chi-square null distribution for the observed statistic. That distribution depends on the hypotheses, how their parameters are estimated, and the sampling behavior under the null.

How to Use the Comparison Responsibly

Before assigning a chi-square reference distribution to a likelihood difference, check the model relationship rather than relying on the statistic's name. The relevant questions are:

• Is the null distribution set contained in the alternative distribution set?
• Are both likelihoods based on the same observed data and compatible probability models?
• If the models are nested, are the regularity conditions for Wilks' theorem satisfied?
• If they are not nested, what testing procedure and null distribution justify the chosen statistic?

A common mistake is to reason, “Both models have a maximized likelihood, so twice their log-likelihood difference must be chi-square.” This skips the key hypotheses of the usual result. For the nested test, the alternative maximizes over a set containing the null set, and the null imposes a specified number of regular restrictions. For nonnested models, neither this containment nor the corresponding degrees-of-freedom argument is available. A chi-square approximation must not be borrowed solely because the expression resembles \(2(\ell_1-\ell_0)\).

Nesting is therefore more than a convenient way to organize model fits. It fixes the direction of the usual likelihood-ratio statistic, gives a meaningful null-versus-expanded-model comparison, and—together with regularity conditions—connects that statistic to Wilks' chi-square limit. Without nesting, likelihood values remain informative descriptive quantities, but a test requires its own justification.