Why Compare Nested Nonlinear Models?
A nonlinear likelihood model may contain several parameters whose joint contribution is difficult to judge by inspecting individual estimates. A likelihood-ratio (LR) comparison asks a focused question: does the data provide evidence against a specified set of restrictions on those parameters? The comparison is useful only when the models are genuinely nested and their likelihoods are maximized on the same data under compatible assumptions.
For instance, a flexible nonlinear response curve might include an offset parameter, while a simpler version fixes that offset to zero. The restricted curve is then a special case of the flexible curve. The LR test measures how much the best attainable log-likelihood improves when the restriction is removed. It does not establish that either model is a causal explanation, nor does it say that the larger model is scientifically preferable in every respect.
Statistic, Restrictions, and Reference Distribution
Let \(\ell(\theta)\) be the log-likelihood for a common observed data set, and let \(\ell_1\) and \(\ell_0\) be its maximum under the unrestricted (larger) and restricted (smaller) models, respectively. Since the restricted parameter values are available to the larger model, \(\ell_1\) cannot be smaller than \(\ell_0\), apart from numerical optimization error.
Here \(L\) is the likelihood, and \(\Theta_0\subseteq\Theta_1\) are the restricted and unrestricted parameter spaces. The factor of 2 gives the standard asymptotic scale for the statistic. If the null model imposes \(q\) independent restrictions, the usual reference distribution under the null hypothesis is chi-squared with \(q\) degrees of freedom.
The parameter-count difference is the right degrees of freedom only when the restrictions are locally independent and the models satisfy the regularity conditions for this result. For linear restrictions this often amounts to counting independent equations. In nonlinear models, restrictions can be redundant or locally dependent, so simply counting their written form may give the wrong answer. Confirm that the restricted parameter space has the expected dimension within the unrestricted model.
The standard calibration also requires, in substance, a correctly specified likelihood, an identifiable true parameter, a null value in the interior of the parameter space, and sufficient smoothness and nonsingular information for the likelihood's local quadratic approximation. The observations must be handled under the same sampling and likelihood conventions in both fits. A nominal parameter count alone does not guarantee a chi-squared result.
Why the Chi-Squared Approximation Appears
The main justification is Wilks' theorem. Near a regular true parameter, the log-likelihood can be approximated by a quadratic function. Maximizing this local approximation without restrictions allows movement in all parameter directions; maximizing it under the null allows movement only along the restricted set. The difference between these two maximized values is asymptotically a squared distance in the \(q\) directions excluded by the null.
More explicitly, the score (the vector of first derivatives of the log-likelihood) has an asymptotic normal distribution, and the information matrix supplies the local curvature of the log-likelihood. After rescaling by that curvature, the \(q\) independent directions removed by the null behave like \(q\) independent standard normal coordinates. Their squared length has a \(\chi^2_q\) distribution. This argument depends on the local quadratic approximation being valid and on the null not lying at a boundary or at a point where parameters are unidentified.
For a test at significance level \(\alpha\), reject \(H_0\) when \(LR\) exceeds the upper \((1-\alpha)\) quantile of \(\chi^2_q\), or equivalently when its upper-tail p-value is below \(\alpha\). The p-value is a probability calculated under the null reference distribution: it is the probability of obtaining an LR statistic at least as large as the observed one, if the null and calibration assumptions hold. It is not the probability that the null is true.
Worked Example: Two Restrictions in a Nonlinear Gaussian Model
Suppose 60 hypothetical observations are fitted with two nested nonlinear mean specifications and Gaussian errors with a common unknown variance. The restricted mean has two free parameters; the unrestricted mean has four. The restricted fit has residual sum of squares \(SSE_0=90\), and the unrestricted fit has \(SSE_1=82\). Assume both fits converge to the relevant maxima, all mean parameters are identified and interior, and the Gaussian likelihood assumptions are appropriate.
With an unknown variance estimated by maximum likelihood, the maximized log-likelihood for a Gaussian regression fit with residual sum of squares \(SSE\) is \(\ell=-\frac{n}{2}\{\log(2\pi)+1+\log(SSE/n)\}\). Thus:
The two models share the estimated variance parameter, so the difference in total free-parameter counts is \(q=(4+1)-(2+1)=2\). The likelihood-ratio statistic is
Under the regular null, use \(\chi^2_2\). Its upper-tail probability at 5.5854 is \(\exp(-5.5854/2)\approx0.0613\). At the 5% level, this is not sufficient evidence to reject the two restrictions. The result does not prove that the smaller model is true; it says that the observed improvement is not unusually large by the stated reference distribution.
Checking Nesting Before Calculating
A sound LR comparison begins with the model definitions, not the software output. Verify that the restricted specification is produced by imposing explicit restrictions on the unrestricted one, with the same outcome, observations, and likelihood basis. The models should differ only through the restrictions being tested. For example, changing the sample because one model drops observations makes the likelihood values unsuitable for a direct LR comparison.
Also verify the parameter dimensions. If a model has \(k_1\) free parameters and its restricted version has \(k_0\), the proposed degrees of freedom are \(q=k_1-k_0\), provided that the restrictions are independent and the regularity conditions hold. Include nuisance parameters estimated in both models when counting parameters: they cancel in the difference, but can matter if they are present in only one model.
Express the smaller model as parameter constraints inside the larger model, and check that every restricted distribution is available in the larger model.
Use the same observations and likelihood definition. Check convergence and whether each reported solution is a maximum rather than an optimization failure.
Find the dimension difference, accounting for shared nuisance parameters and any dependent constraints.
If the null is regular and interior, compare the statistic with \(\chi^2_q\); otherwise do not assume that reference distribution applies.
Give the restrictions, \(LR\), degrees of freedom, p-value, and a conclusion tied to the specified null and significance level.
Worked Example: Testing an Offset in an Exponential Curve
Consider a hypothetical nonlinear response model with mean curve \(\mu(t)=a\exp(-bt)+c\), where \(a\), \(b\), and \(c\) are estimated from the same observations. The restricted model imposes \(c=0\), leaving \(\mu(t)=a\exp(-bt)\). Suppose maximum likelihood gives \(\ell_0=-41.20\) under the restriction and \(\ell_1=-39.85\) without it.
The restriction removes one free parameter, so \(q=1\). Assume the error likelihood is correctly specified, the estimates are finite and identified, and \(c=0\) lies in the interior of the parameter space rather than at a boundary. Then the regular chi-squared calibration is appropriate:
For \(\chi^2_1\), the upper-tail probability is approximately \(0.1003\). At \(\alpha=0.05\), the data do not provide sufficient evidence to reject \(c=0\). In context, the test does not establish that the response truly has no offset; it indicates that this likelihood comparison did not detect a statistically significant improvement from estimating the offset.
When the Usual Reference Distribution Is Unreliable
The chi-squared approximation can fail when the geometry or local behavior of the model violates the conditions behind Wilks' theorem. A common case is a null value on the boundary of the parameter space. For example, if an added variance component is constrained to be nonnegative, testing whether it is zero places the null at the boundary. The statistic may then have a mixture distribution rather than an ordinary chi-squared distribution. The precise mixture depends on the model and restrictions; it should not be guessed from parameter counts alone. Boundary parameters and nonstandard likelihood inference require special care.
Another problem is nonidentification under the null. If a parameter has no effect on the distribution when a different parameter is set to zero, that otherwise-present parameter may be unidentified under the null. The ordinary parameter-count difference then need not give the degrees of freedom. Weak identification or a flat likelihood can also make the large-sample quadratic approximation poor, especially in finite samples. Numerical convergence alone does not resolve these issues.
Misspecification is another warning. If the likelihood does not describe the data-generating distribution, the ordinary LR statistic need not have the standard chi-squared null distribution. A robust covariance estimator does not, by itself, make the classical LR calibration valid. The validity of the reference distribution depends on the statistical setting and test, not just on the optimizer or the apparent size of the sample.
Worked Example: A Boundary Restriction Changes Calibration
Suppose a hypothetical model adds a nonnegative parameter \(\gamma\geq0\), and the null hypothesis is \(H_0:\gamma=0\). Assume there is one such boundary parameter, the other parameters are regular nuisance parameters, the model is otherwise correctly specified and identified, and the relevant asymptotic result is the usual one-parameter boundary mixture: probability one-half at zero and probability one-half on \(\chi^2_1\).
If the observed likelihood-ratio statistic is \(LR=2.71\), a naive interior-parameter calculation would use the \(\chi^2_1\) upper tail, approximately \(0.0998\). Under the stated boundary mixture, however, the upper-tail probability for this positive statistic is half that value:
The conclusions at the 5% level differ: the ordinary chi-squared calibration would not reject, while this mixture calibration is just below 0.05. The mixture is justified here only because its specific boundary conditions were stipulated. Other boundary problems can have different limiting distributions, so the standard \(\chi^2_q\) rule should not be applied automatically.
Common Mistakes and Practical Interpretation
- Using a parameter count without checking nesting. Two specifications with different parameter counts are not necessarily nested. The restricted model must be obtainable by constraints within the larger one.
- Using the number of written equations as the degrees of freedom. Restrictions that are redundant or dependent do not each remove an independent dimension. Count independent restrictions.
- Using a chi-squared distribution just because software reports an LR statistic. A boundary null, unidentified parameter, weak identification, or misspecified likelihood can invalidate the usual calibration.
- Comparing log-likelihoods from incompatible fits. The observations and likelihood basis must match; otherwise the difference need not represent the improvement from relaxing the stated restrictions.
- Interpreting a non-significant result as proof of the null. Failure to reject means the evidence is insufficient at the chosen level, not that the restricted model has been established as true.
An LR comparison is most informative when its target is explicit: name the restriction, state why the models are nested, and justify the reference distribution. Report the maximized log-likelihoods or the LR statistic, the degrees of freedom, and the p-value. A small p-value is evidence against the restrictions under the assumed model and calibration; it does not guarantee that the larger model is well specified or substantively useful.
Check Your Understanding
Answer each question by identifying the model comparison and its assumptions, not just by naming a formula.
- A restricted model has three free parameters and the unrestricted model has six. Under regular conditions, what degrees of freedom would you use, and what must be true about the restrictions?
- The restricted and unrestricted models were fitted to different sets of observations. Why is their log-likelihood difference not automatically a valid LR statistic?
- If \(\ell_0=-52.4\) and \(\ell_1=-49.9\), calculate \(LR\). What does its nonnegative value express about nested maximum likelihood fits?
- Why can testing a nonnegative parameter equal to zero invalidate the ordinary \(\chi^2_1\) reference distribution?
- State a correct interpretation of an LR-test p-value of 0.03 under a valid 5% test.