Why a Search Needs a Cutoff
When two nested population pharmacokinetic models are fitted to the same data, adding a covariate cannot make the maximized likelihood worse: the expanded model can reproduce the reduced model by setting its added effect to zero. The expanded model will therefore often fit at least slightly better, even when the improvement is mostly sampling variation. A decision rule needs to distinguish an improvement large enough to meet a prespecified statistical criterion from one that does not.
For a likelihood-ratio test, that rule can be expressed as a cutoff on the objective-function difference. The cutoff is not universal. It depends on the significance level \(\alpha\) and on the number of independent restrictions being tested, expressed as the degrees of freedom \(df\). The same objective-function improvement can pass one test and fail another if their degrees of freedom or prespecified significance levels differ.
The null hypothesis \(H_0\) says that all added covariate-effect parameters are zero, so the reduced model is sufficient for this comparison. The alternative hypothesis \(H_a\) says that at least one added effect differs from zero. The number of degrees of freedom is the number of independent restrictions imposed by \(H_0\), usually the number of freely estimated effect parameters added. It is not necessarily the number of covariate names: a categorical covariate may require several parameters, while a model term may have constraints that reduce the number of independent parameters.
From Significance Level to an OFV Cutoff
Under the null hypothesis and standard regularity conditions, the likelihood-ratio statistic is approximately distributed as a chi-square random variable with \(df\) degrees of freedom. This is an asymptotic result: it becomes a useful approximation when the likelihood is correctly specified and the models are identifiable and regular. For a chosen significance level \(\alpha\), the critical value is the upper-tail chi-square quantile \(c\) satisfying \(P(\chi^2_{df}\ge c)=\alpha\).
The decision rule is to reject \(H_0\) for this test if \(\Delta OFV\ge c\). Equivalently, the likelihood-ratio p-value is the upper-tail probability \(P(\chi^2_{df}\ge\Delta OFV)\), and the test meets the prespecified criterion when that p-value is at most \(\alpha\). These are two ways to express the same rule, not two separate tests.
The logic follows directly from the test definition. Under \(H_0\), a statistic in the upper \(\alpha\) tail is an event with probability \(\alpha\), according to the reference chi-square distribution. Setting the boundary at the quantile with upper-tail probability \(\alpha\) therefore gives the rejection rule. Because the likelihood-ratio statistic equals \(\Delta OFV\), that same boundary is the objective-function-change cutoff. This controls the Type I error rate at approximately \(\alpha\) for this one test, when the assumptions and approximation hold.
| Degrees of freedom | Cutoff at \(\alpha=0.05\) | Cutoff at \(\alpha=0.01\) |
|---|---|---|
| 1 | 3.841 | 6.635 |
| 2 | 5.991 | 9.210 |
| 3 | 7.815 | 11.345 |
| 4 | 9.488 | 13.277 |
These values are reference points, not general-purpose OFV thresholds. For instance, 3.841 applies to a one-degree-of-freedom test at \(\alpha=0.05\); using it for a three-parameter joint covariate test would apply the wrong reference distribution.
Worked Applications
Worked Example: A One-Parameter Genotype Effect
Hypothetical data. A modeler tests whether a binary genotype is associated with clearance by adding one genotype coefficient. The reduced model has \(OFV=1268.4\), and the expanded model has \(OFV=1263.9\). The comparison is prespecified at \(\alpha=0.05\).
Both fits use the same observations and likelihood, and the expanded model adds one freely estimated coefficient. The models are nested, so \(df=1\). Assuming the coefficient is identifiable, the optimum is not on a problematic boundary, and the likelihood fit is reliable, the usual chi-square approximation is applicable.
Calculate the change: \[ \Delta OFV=1268.4-1263.9=4.5. \] For \(df=1\) and \(\alpha=0.05\), the critical value is \(3.841\). Since \(4.5>3.841\), the added effect meets the single-test, 0.05 likelihood-ratio criterion. The corresponding upper-tail p-value is approximately \(0.0339\).
The result supports retaining this genotype effect under the stated decision rule; it does not establish that genotype causes a change in clearance. The test concerns the model comparison and its assumptions, not a causal claim.
Worked Example: A Three-Level Covariate
Hypothetical data. A three-level renal-function category is added to a model for clearance using two coefficients relative to a reference category. The reduced model has \(OFV=912.7\), and the expanded model has \(OFV=907.1\). The joint test is prespecified at \(\alpha=0.05\).
The null hypothesis sets both added coefficients to zero. The models are nested, both coefficients are estimable, and the same likelihood and observations are used. Thus, \(df=2\). The cutoff is \(5.991\).
\[ \Delta OFV=912.7-907.1=5.6. \] Since \(5.6<5.991\), the improvement does not meet the prespecified 0.05 criterion. For two degrees of freedom, the upper-tail probability is approximately \(e^{-5.6/2}=e^{-2.8}=0.0608\).
The conclusion is about the joint test of the two coefficients. It would be incorrect to use the one-degree-of-freedom cutoff of 3.841 simply because the covariate is one named variable, or to decide from one coefficient’s individual test instead of the specified joint comparison.
Worked Example: A More Stringent Prespecified Level
Hypothetical data. A single continuous biomarker effect on apparent clearance is tested with one added coefficient. The reduced model has \(OFV=780.0\), and the expanded model has \(OFV=772.9\). Before fitting, the analysis plan specifies \(\alpha=0.01\) for this comparison.
The models are nested, use the same data and likelihood, and differ by one estimable coefficient, so \(df=1\). At \(\alpha=0.01\), the cutoff is \(6.635\), not \(3.841\).
\[ \Delta OFV=780.0-772.9=7.1. \] Because \(7.1>6.635\), this result meets the prespecified 0.01 criterion. Its p-value is approximately \(0.0077\). Had the change instead been \(5.0\), it would have exceeded the 0.05 cutoff for one degree of freedom but failed the stated 0.01 rule. Choosing the threshold after seeing the result would not preserve the intended interpretation of a prespecified significance level.
Assumptions, Diagnostics, and Limitations
The chi-square cutoff is meaningful only when the comparison is a likelihood-ratio comparison under conditions supporting its reference distribution. Check that the reduced model is obtained from the expanded model by imposing the stated null restrictions, that both models use the same observations and likelihood definition, and that both fits have converged to credible optima. A positive \(\Delta OFV\) is expected when the expanded model is properly optimized; a negative value can indicate a fitting, coding, or comparison problem.
The usual chi-square approximation also relies on regularity: parameters are identifiable, the likelihood behaves smoothly near the true values, and the null parameter values are in the interior of the parameter space. Sparse subgroup data, near-confounding among covariates, weakly identified effects, or estimates on parameter boundaries can undermine these conditions. In such settings, the nominal chi-square cutoff may not yield its advertised Type I error rate. A large sample approximation should not be treated as a guarantee for a sparse or unstable fit.
In practice, inspect parameter estimates and uncertainty, subgroup representation, convergence information, and whether the objective-function values are reproducible under reasonable refitting. A boundary estimate, implausibly large effect, or substantial sensitivity to starting values is a warning that the regular approximation or the model comparison may be unreliable. These checks do not replace the formal test; they help assess whether its assumptions are credible.
| Approach | What sets the decision | Important distinction |
|---|---|---|
| Likelihood-ratio cutoff | \(\alpha\), \(df\), and the observed \(\Delta OFV\) | Tests nested models using their maximized likelihoods. |
| Likelihood-ratio p-value | The upper-tail probability for the same statistic and reference distribution | Equivalent to comparing with the corresponding cutoff when \(\alpha\) is fixed. |
| One fixed OFV threshold for every test | A constant chosen without adapting to \(df\) or \(\alpha\) | Does not generally correspond to the intended significance level. |
| Wald or score test | A different test statistic and its reference distribution | Not the same calculation as comparing two maximized likelihoods. |
A significance-based cutoff answers a specific question: whether a nested-model improvement is large enough for a chosen single-test error rate under the assumed reference distribution. It does not account automatically for repeated screening of many covariates. If many tests are conducted, the overall chance of one or more false-positive findings can exceed the single-test \(\alpha\). Nor does passing the cutoff prove that an effect is clinically important, biologically plausible, or predictive in new data. Statistical evidence, effect size, uncertainty, and scientific context should be considered together.
Common Mistakes
- Using 3.84 for every covariate. That value is the approximate 0.05 cutoff only for one degree of freedom. Use the cutoff matching the number of independent restrictions.
- Reversing the OFV subtraction. Define \(\Delta OFV\) as reduced minus expanded. Subtracting in the opposite order makes an improvement appear negative.
- Confusing an OFV difference with an OFV value. The test uses the difference between the two fitted models, not either model’s absolute objective function.
- Changing \(\alpha\) after looking at the results. The stated error rate is meaningful as a decision rule when specified in advance, not selected to make a particular result pass.
- Reading a failed cutoff as proof of no effect. Failure to reject \(H_0\) means the comparison did not meet the chosen criterion; it does not establish that the covariate effect is exactly zero.
Check Your Understanding
For each problem, identify the relevant cutoff and show the objective-function arithmetic before making a decision.
- A one-parameter covariate test has reduced and expanded OFVs of 540.2 and 536.5. At \(\alpha=0.05\), does it meet the cutoff?
- A joint test adds three estimable parameters. At \(\alpha=0.05\), what cutoff should be used, and does \(\Delta OFV=8.0\) pass?
- For a two-degree-of-freedom test at \(\alpha=0.01\), the OFVs are 400.0 and 391.2. State the decision and approximate p-value.
- Why is it not valid to conclude that a covariate has no effect just because its objective-function change is below the cutoff?
Solutions. (1) \(\Delta OFV=540.2-536.5=3.7\). The one-degree-of-freedom cutoff is 3.841, so it does not pass the 0.05 criterion. (2) Use \(df=3\), giving a cutoff of 7.815. Since \(\Delta OFV=8.0>7.815\), it passes the single-test criterion. (3) \(\Delta OFV=400.0-391.2=8.8\). The \(df=2\), \(\alpha=0.01\) cutoff is 9.210, so it does not pass. The upper-tail probability is \(e^{-8.8/2}=e^{-4.4}\approx0.0123\), which is greater than 0.01. (4) The test may lack power, and the observed data may be compatible with a range of effects. Not crossing the threshold is a failure to reject the null at the chosen criterion, not proof that the effect equals zero.