Applying Likelihood Ratio Comparisons

When a Likelihood Ratio Comparison Is Appropriate

A likelihood ratio test (LRT) asks whether the improvement in fit from a more flexible model is larger than would be expected if the restrictions defining a simpler model were true. In pharmacometrics, it can provide a formal comparison of two fitted runs, but only when the runs represent suitable nested likelihood models. A lower objective value alone is not enough: the models, data, estimation, and parameter restrictions must support the test.

The comparison is directional. Call the simpler model the restricted model and the more flexible model the full model. The restricted model must be obtainable from the full model by imposing specified restrictions on its parameters. For example, setting a covariate effect to zero may reduce a model with that effect to one without it. Two models are not nested merely because one has fewer parameters or a higher objective function.

Write the reported objective values as \(\mathrm{OFV}_R\) and \(\mathrm{OFV}_F\). Then

Because the full model includes all parameter values permitted by the restricted model, its maximized likelihood cannot be lower when both optimizations are successful and use the same likelihood. Thus \(D\) should be nonnegative, apart from small numerical discrepancies. The degrees of freedom, \(q\), are the number of independent restrictions imposed by the restricted model. Under the null hypothesis that those restrictions hold, and subject to the regularity conditions described below, \(D\) is approximately distributed as \(\chi^2_q\). The p-value is the upper-tail probability \(P(\chi^2_q \ge D_{\mathrm{observed}})\).

Conditions to Check Before Calculating a P-Value

A standard chi-square LRT relies on more than a pair of objective values. Check each of the following before treating the reference distribution as appropriate.

• Genuine nesting: The restricted model must be a special case of the full model, obtained by fixing or restricting parameters. Identify the restrictions explicitly.
• Same observations and likelihood basis: Both runs must use the same analysis records and compatible likelihood definitions. Different records, exclusions, or likelihood contributions can make the objective difference unsuitable as an LRT statistic.
• Comparable objective values: The difference must represent twice a log-likelihood difference. Additive constants common to both models cancel, but a penalized objective, posterior objective, or differently defined criterion is not automatically a likelihood-ratio statistic.
• Reliable maximization: Both fits must have converged to appropriate maxima. Check run status, parameter estimates, gradients or other convergence diagnostics, and whether multiple starts suggest a better maximum. A failed or inferior optimization can distort the difference.
• Regularity: The null parameter values should be identifiable and lie in the interior of the full model's permissible parameter space. The likelihood should be sufficiently smooth near those values, with no relevant singularity or unidentifiable direction.
• Enough information for the approximation: The chi-square result is generally an asymptotic approximation. With sparse data, weakly identified parameters, or complex random-effects structures, that approximation may be poor.

In pharmacometric work, also verify that the fitting procedure's reported objective is a suitable likelihood-based quantity. If it is an approximation to the likelihood, the usual LRT interpretation depends on that approximation being adequate for this comparison. Do not assume that two numbers labelled “objective function” have the required relationship without checking how they were calculated.

Why the Chi-Square Reference Appears

Under the null hypothesis, the restricted model is adequate in the sense that the extra \(q\) parameters in the full model do not improve the population likelihood beyond their null values. For a regular, identifiable model with enough information, the log likelihood near its maximum is approximately quadratic in small parameter changes. After accounting for parameters shared by both models, the improvement from releasing \(q\) restrictions is approximately the squared length of a \(q\)-dimensional standardized random component.

Each component is approximately standard normal under the null. The sum of their squares therefore has a chi-square distribution with \(q\) degrees of freedom. This gives the large-sample result

This argument also explains why the degrees of freedom count restrictions, not the number of observations, the number of estimated parameters shared by both models, or the numerical size of the objective difference. It explains as well why boundary values and non-identifiability matter: they can invalidate the regular quadratic approximation or change the distribution of the improvement.

Worked Comparisons With More Than One Restriction

Boundary Parameters and Other Failure Modes

A standard LRT can fail when the null value lies on the boundary of the full model's parameter space. For instance, a variance component is constrained to be nonnegative, so testing whether it equals zero tests a boundary value. In such cases, the ordinary \(\chi^2_q\) reference distribution is not generally justified. A particular simple variance-component test may have a mixture reference distribution, but that result depends on its assumptions and does not automatically apply to more complex random-effects or covariance comparisons. Use a justified reference distribution for the specific model, or consider an appropriate calibration method such as a parametric bootstrap.

Other warning signs include a negative \(D\) that is too large to be explained by rounding, substantially different data rows, a full model that is not actually a superset of the restricted model, unstable estimates, or a parameter that changes dramatically with starting values. Investigate these conditions before interpreting a p-value. A lower objective does not repair a convergence problem or make non-nested models suitable for this test.

Interpreting the Result Alongside Other Comparisons

The LRT is a hypothesis test for specified restrictions in nested likelihood models. Its p-value describes how often a statistic at least as large as the observed \(D\) would arise under the null model and the test assumptions. It is not the probability that the null hypothesis is true, and a small p-value is not a measure of effect size or clinical relevance.

Information criteria, such as those used in the earlier material on comparing information criteria across runs, answer a different model-comparison question by balancing fit and complexity. They can be used in settings where a nested-model LRT is unavailable, subject to their own assumptions and purpose. Predictive evaluation may also be relevant when the goal is performance on new data. These approaches are not interchangeable: use the LRT when the nested restriction and likelihood assumptions are defensible, and use a method aligned with the actual comparison question otherwise.

A practical comparison record should preserve the model relationship, restricted and full objective values, the subtraction used, the restriction count, the reference distribution, the p-value, and any qualification about convergence or regularity. This makes the inference auditable and helps distinguish a valid test from a simple descriptive comparison of runs.

Common Mistakes

• Subtracting in the wrong direction. Use restricted minus full objective. A materially negative difference calls for investigation, not a negative chi-square statistic.
• Using the number of added parameters without checking restrictions. Degrees of freedom are the number of independent restrictions, which may not equal the apparent parameter count when constraints or identifiability issues are present.
• Applying the test to any two runs. Different data, non-nested model structures, or non-likelihood objectives do not meet the standard LRT setup.
• Using the ordinary chi-square distribution at a boundary. A variance fixed at zero, for example, requires a boundary-aware reference distribution or another justified method.
• Treating a non-significant result as proof of equality. Failure to reject a restriction means the comparison did not provide sufficient evidence against it at the chosen threshold; it does not demonstrate equivalence.
• Equating statistical evidence with scientific importance. A p-value does not replace evaluation of parameter magnitude, uncertainty, model adequacy, or clinical relevance.