Why Compare Nested Covariate Models?
A covariate model adds a structured explanation for variation in a typical PD parameter. The key question is not merely whether the larger model fits the observed data better: with additional parameters, its maximized likelihood cannot be lower. The question is whether the improvement is large enough, relative to the extra flexibility, to be inconsistent with the smaller model under specified assumptions.
A likelihood-ratio test (LRT) makes this comparison for nested models. The reduced model is obtained from the full model by fixing one or more of its parameters to specified null values. For example, if a typical PD parameter is related to a centered continuous covariate \(z\) by \(\theta_i=\theta_{\mathrm{typ}}\exp(\beta z_i)\), the no-covariate model is nested within the covariate model at \(H_0:\beta=0\). The coefficient \(\beta\) describes the change on the log scale per unit of \(z\); its units and interpretation depend on how \(z\) is measured.
The models must use the same observations, response definition, likelihood, and treatment of the other model components. The full model must contain every parameterization allowed by the reduced model. In population PD analysis, this means being especially careful that residual-error and random-effects structures have not changed at the same time as the covariate term. Otherwise, the comparison does not isolate the covariate effect.
Reference Distribution and Degrees of Freedom
Under the null hypothesis, the usual large-sample reference distribution is chi-square. Its degrees of freedom are the difference in the number of freely estimated parameters between the full and reduced models—not the number of model files, covariates considered, or observations.
Here \(k_F\) and \(k_R\) are the numbers of free parameters in the full and reduced models, and \(q\) is the number of independent restrictions imposed by the null hypothesis. For a single unrestricted covariate coefficient tested against zero, \(q=1\). For a categorical covariate with four levels and one reference level, the global no-effect test ordinarily sets three contrast coefficients to zero, so \(q=3\). A covariate represented by a spline or another multi-parameter function can likewise require more than one degree of freedom.
The chi-square approximation relies on regularity conditions. The models must be nested; the null parameter value must be identifiable and, in the usual case, in the interior of the parameter space; and the likelihood should be sufficiently smooth near the true parameter values. The information about the tested parameters must be adequate, and nuisance parameters must be estimable under the null. In a nonlinear mixed-effects model, the likelihood is generally the likelihood of the observations after accounting for the distribution of individual random effects. If software approximates that likelihood, the approximation must be adequate for the difference in objective function values to be trusted.
Why the Chi-Square Approximation Arises
A short justification helps clarify both the test and its limits. Near the parameter value that maximizes the likelihood under the null, a smooth log likelihood can be approximated by a quadratic function. After nuisance parameters have been accounted for, the gain from freeing \(q\) parameters is approximately a quadratic form in their estimated departures from the null values.
Here \(\mathbf{U}\) is the efficient score vector for the \(q\) tested parameters under the null, and \(\mathbf{I}\) is its information matrix. Under the regularity conditions, the standardized score \(\mathbf{I}^{-1/2}\mathbf{U}\) is approximately distributed as \(N(\mathbf{0},\mathbf{I}_q)\). The sum of squares of \(q\) independent standard normal variables has a \(\chi^2_q\) distribution. This gives the reference distribution for the LRT. If a null value is on the boundary, the local quadratic approximation is restricted to only part of the parameter space, so the ordinary chi-square result may fail.
Worked Example: One Continuous Covariate Coefficient
Hypothetical data: Two population PD models use the same response observations and the same random-effects and residual-error structures. The full model adds a body-weight effect on a typical PD parameter: \(\theta_i=\theta_{\mathrm{typ}}\exp(\beta z_i)\), where \(z_i\) is body weight centered and scaled in units of 10 kg. The reduced model fixes \(\beta=0\). Suppose the reduced and full objective function values are 1240.0 and 1233.4, respectively. In the full model, \(\hat{\beta}=0.420\) per 10 kg, with \(SE=0.160\).
Check the comparison. The full model is the reduced model plus one freely estimated coefficient, and both models use the same data and likelihood. Thus they are nested and \(q=1\). The null value \(\beta=0\) is not a boundary of the unrestricted real-valued coefficient parameter space.
Calculate the test statistic.
Under \(H_0\), use \(\chi^2_1\). The upper-tail p-value is \(P(\chi^2_1\geq 6.60)\approx0.0102\). At \(\alpha=0.05\), this is evidence against the no-covariate model under the stated assumptions. It is not the probability that \(H_0\) is true.
Describe magnitude and uncertainty. The estimate corresponds to a multiplicative change of \(\exp(0.420)=1.52\) in the typical parameter for a 10 kg increase in body weight, under this model. A rough Wald 95% interval for \(\beta\) is \(0.420\pm1.96(0.160)=(0.106,0.734)\) per 10 kg. Exponentiating the endpoints gives an approximate ratio interval of \((1.11,2.08)\) per 10 kg. This describes the estimated model effect and its uncertainty; the LRT alone does not establish clinical importance or causation.
Global Tests for Categorical Covariates
For a categorical covariate, the relevant LRT often tests the whole covariate as a group. Testing each contrast separately answers different questions and can miss the fact that the global hypothesis is a set of restrictions. The degrees of freedom must reflect the number of coefficients jointly removed in the reduced model.
Worked Example: A Three-Level Covariate
Hypothetical data: A full population PD model includes a three-level disease-severity covariate, represented by two contrasts against a reference level. A reduced model omits both contrasts and otherwise has the same structure and data. Suppose \(OFV_R=890.0\) and \(OFV_F=884.8\).
The full model adds two free coefficients, so \(q=2\). The LRT statistic is \(LR=890.0-884.8=5.20\). For two degrees of freedom, the upper-tail probability is \(P(\chi^2_2\geq5.20)=\exp(-5.20/2)=\exp(-2.60)\approx0.0743\). At \(\alpha=0.05\), this global test does not reject the no-effect null hypothesis. This is not proof that severity has no effect: the data may provide limited precision, and the test does not identify which level contrasts are most uncertain.
The LRT addresses whether the two coefficients jointly improve fit beyond what would be expected under the null model. To describe the estimated effect, report the contrast estimates, their units or scale, and suitable uncertainty intervals as well. A global p-value does not provide those quantities.
Boundary Cases
The standard \(\chi^2_q\) reference distribution generally assumes the null value is in the interior of the parameter space. A common exception in mixed-effects modeling is a variance component: a variance must be nonnegative, and testing whether it is zero places the null on the boundary. A similar issue arises if a covariate coefficient is deliberately constrained to be nonnegative and the null fixes it to zero. A boundary test should not automatically be judged using the ordinary chi-square reference distribution.
In a simple, regular case with one parameter on a boundary, a commonly applicable asymptotic reference distribution is a 50:50 mixture of a point mass at zero and \(\chi^2_1\). Under this mixture, the upper-tail p-value for a positive observed statistic is half the usual \(\chi^2_1\) upper-tail probability. This result does not apply indiscriminately: multiple boundary parameters, dependence among constraints, or other irregularities can produce different reference distributions. The appropriate null distribution may require a problem-specific result or simulation.
Worked Example: A Single Constrained Effect at Zero
Hypothetical data: Suppose a prespecified PD model constrains a covariate coefficient to be nonnegative, and the null model fixes that coefficient to zero. The models are nested and differ by one coefficient. Assume the other parameters are identifiable under the null and that the simple single-boundary mixture applies. The observed LRT statistic is \(2.71\).
Because the alternative permits only nonnegative values, zero is on its boundary. The appropriate upper-tail probability under the stated mixture is \(0.5P(\chi^2_1\geq2.71)\). Since \(P(\chi^2_1\geq2.71)\approx0.100\), the mixture p-value is approximately \(0.050\). This is borderline evidence at the 5% level, with the exact decision depending on the unrounded statistic and p-value.
Using the ordinary \(\chi^2_1\) p-value of about 0.100 would give the wrong reference distribution for this stipulated boundary problem. Conversely, applying the mixture correction to an ordinary unrestricted coefficient with null value zero would also be wrong: zero is an interior value for an unrestricted real-valued coefficient.
Checking the Analysis and Comparing Alternatives
Before interpreting an LRT, verify the comparison rather than relying only on a software p-value. A negative difference \(OFV_R-OFV_F\) for a claimed nested comparison suggests that the full model was not actually maximized successfully, the models are not nested as specified, or the values have been recorded inconsistently. A very small improvement can be meaningful with high precision, while a large improvement can be unstable if the likelihood approximation or optimization is unreliable.
Write which covariate coefficient or set of coefficients is fixed, and verify the reduced model is obtained from the full model by that restriction.
Use the same observations, outcome likelihood, and other model structures. Confirm successful and stable optimization for both fits.
Set \(q\) equal to the difference in free parameters, then determine whether the null is an interior value or a boundary.
Compute \(LR\), use the justified reference distribution, and report the p-value together with the estimated effect and its uncertainty.
Other approaches answer related but not identical questions. A Wald test uses the estimated coefficient and its standard error; it can be convenient, but may be inaccurate when the likelihood is asymmetric or the estimate is near a boundary. A score test evaluates the slope of the likelihood at the null fit and can avoid fitting the full model, but also relies on suitable regularity and information calculations. The LRT directly compares maximized likelihoods, but requires fitting both models and can be sensitive to optimization or likelihood-approximation error.
| Approach | What it uses | Practical consideration |
|---|---|---|
| Likelihood-ratio test | Difference between maximized likelihoods | Requires nested models and reliable fits of both |
| Wald test | Estimate divided by its standard error | Can be unreliable with asymmetry, weak identification, or boundary estimates |
| Score test | Likelihood slope and information under the null | May avoid fitting the full model, but still needs a valid null-based approximation |
These methods can disagree in finite samples, especially in complex nonlinear mixed-effects models. Such disagreement is a reason to investigate precision, model stability, and the shape of the likelihood—not to select whichever result gives the preferred conclusion. An LRT also does not establish that a covariate relationship is biologically plausible, that the model predicts well outside the observed covariate range, or that the estimated effect is important for decisions.
Common Pitfalls and Limitations
- Wrong reasoning: “The full model has a lower OFV, so the covariate is important.” The full model is expected to fit at least as well because it has more freedom. Correction: quantify the improvement with the LRT and a justified reference distribution, then assess effect size and uncertainty.
- Wrong degrees of freedom: using one degree of freedom for a four-level covariate because it is one covariate. Correction: count the independent coefficients jointly removed; here that is usually three.
- Invalid comparison: comparing models fitted to different observations or changing the residual-error structure at the same time as adding a covariate. Correction: make the comparison isolate the specified nested restriction.
- Ignoring a boundary: applying the ordinary chi-square test to a parameter constrained to be nonnegative at a null value of zero. Correction: establish the parameter-space geometry and use a suitable boundary reference distribution.
- Overinterpreting the p-value: treating a small p-value as evidence of a large or clinically meaningful effect. Correction: report the coefficient on its meaningful scale, with units and uncertainty, and consider the model context.
LRTs are asymptotic tools, not guarantees of valid inference in every data set. Sparse covariate ranges, weak parameter identification, influential observations, local maxima, and inaccurate likelihood approximations can undermine the comparison. A small p-value cannot repair a poorly supported model, and a nonsignificant result does not demonstrate equivalence or rule out an effect.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Two nested models have \(OFV_R=510.0\) and \(OFV_F=506.2\). The full model adds one unrestricted coefficient. Calculate the LRT statistic and its degrees of freedom.
- A five-level covariate is represented by four contrasts. What degrees of freedom are ordinarily used for a global test of no covariate effect?
- Why is the ordinary \(\chi^2_1\) reference distribution generally inappropriate for testing a single variance component equal to zero?
- What should be checked if a supposedly nested full model has a higher OFV than its reduced model?
Solutions. (1) \(LR=510.0-506.2=3.80\), with \(q=1\). The statistic alone does not determine a conclusion without selecting the reference distribution and significance level; under the regular unrestricted case, the p-value is \(P(\chi^2_1\geq3.80)\approx0.0513\). (2) The global test ordinarily has \(q=4\), because four contrast coefficients are removed jointly. (3) A variance cannot be negative, so zero is a boundary rather than an interior null value; the usual regularity conditions for \(\chi^2_1\) do not hold, and a suitable boundary distribution must be established. (4) Check that the models are genuinely nested, that the same data and likelihood were used, that the OFVs are correctly recorded, and that both optimizations converged to reliable maxima. A higher full-model OFV can indicate an estimation or specification problem.