Why a Second-Class Test Needs Special Calibration
A two-class mixture can improve a likelihood by representing systematic differences between subgroups—for example, distinct clearance patterns. But a larger model will generally fit at least as well as its one-class counterpart, even when the population contains only one class. The key question is whether the observed improvement is unusually large under a one-class model. Answering it with a standard likelihood-ratio test is not automatic: under the null, some parameters of the two-class model may lie on a boundary or may not be identifiable at all.
This matters because the familiar chi-square reference distribution for a likelihood-ratio statistic relies on regularity conditions that do not generally hold for a mixture test. In particular, the ordinary result assumes that the null parameter lies in the interior of the parameter space and that the parameters being tested are identifiable under the null. A second-class comparison can violate both conditions. A parametric bootstrap provides a practical way to estimate the null distribution for the comparison while preserving the study design and fitting procedure.
What Makes the Null Hypothesis Nonregular?
Write the two-class subject-level likelihood contribution as a mixture of two class-specific contributions. These contributions are marginal over the subject’s random effects, as required for the population likelihood. If \(\pi_i\) is the probability that subject \(i\) belongs to class 1, then
Here \(f_{i1}\) and \(f_{i2}\) are the subject’s likelihood contributions under the two class-specific models. The one-class model is contained in the closure of the two-class model in at least two important ways. First, one class can have prevalence zero, so its class-specific parameters have no effect on the likelihood. Second, the two classes can have identical distributions, so their mixture is the same distribution regardless of the class prevalence.
Both cases disrupt the regular conditions behind a standard chi-square calibration. A prevalence of zero is on the boundary of the allowed interval from zero to one. Moreover, when a class has zero prevalence, its clearance and other class-specific parameters cannot be learned from the data: changing them has no effect on the likelihood. If the two class distributions coincide instead, the likelihood does not identify their prevalence. Thus the null may leave parameters of the larger model unidentified, rather than simply setting a known, identifiable parameter to zero.
A short algebraic check shows why coincident classes cause a problem. If \(f_{i1}=f_{i2}=f_i\), then for any allowed \(\pi_i\),
The likelihood contribution is exactly the same as under a single class. Since it does not vary with \(\pi_i\), the data cannot distinguish among prevalence values at that point. The likelihood therefore has a flat direction, which is inconsistent with the usual regular setting of a uniquely identified interior parameter value. Similarly, at \(\pi_i=0\), the contribution reduces to \(f_{i2}\) and contains no information about the parameters that define \(f_{i1}\).
Worked Example: Identical Classes Leave Prevalence Unidentified
Suppose that for one hypothetical subject, the marginal likelihood contributions under two classes are both \(0.12\). The class 1 probability is \(\pi_i=0.30\). The subject’s mixture contribution is
Now change the class 1 probability to \(0.80\). The contribution becomes
The likelihood contribution remains \(0.120\). This calculation illustrates that when the class-specific distributions coincide, changing prevalence cannot improve or worsen the fit. It is not evidence for a particular prevalence; the prevalence is unidentified at this point in the model. The same reasoning applies to the full data likelihood when the class-specific distributions coincide for every subject.
Parametric-Bootstrap Calibration
A parametric bootstrap estimates how large the likelihood-ratio statistic can be when data are generated under the fitted one-class null model. Each replicate must follow the actual analysis procedure: keep the study design fixed, simulate outcomes under the null, fit both models to that simulated data, and record their likelihood-ratio statistic. Keeping the design fixed means retaining relevant features such as subjects, doses, sampling times, and covariates. The simulation should also use the fitted null model’s random-effects and residual-error assumptions.
The comparison requires comparable maximized likelihoods. In each simulated data set, the two-class fit should use the same model specification and a deliberate initialization strategy, such as multiple starts, as the fit to the observed data. If a fit fails, it is not appropriate to silently discard the replicate: failures can be related to the statistic being estimated. Investigate the cause, improve the fitting procedure, and document any remaining failures and the rule used to handle them.
Estimate the one-class NLME model and retain its fitted parameters, including the random-effects and residual-error parameters used to generate outcomes.
Fit the one- and two-class models to the observed data and calculate \(T_{\mathrm{obs}}=2(\ell_2-\ell_1)\), using the best supported maximized likelihood from the fitting procedure.
For each of \(B\) replicates, simulate a complete data set under the fitted null model at the observed design, fit both candidate models, and calculate \(T_b=2(\ell_{2b}-\ell_{1b})\).
Count the bootstrap statistics at least as large as the observed statistic. With the finite-simulation correction, calculate \(p_{\mathrm{boot}}=(1+\sum_{b=1}^{B} I[T_b\geq T_{\mathrm{obs}}])/(B+1)\).
The added one in the numerator and denominator avoids reporting a zero p-value merely because no finite set of replicates exceeded the observed statistic. This estimate is conditional on the fitted null model and the simulation procedure. It is an approximate calibration, not proof that the null model is correct.
Worked Example: Estimating a Bootstrap P-Value
Suppose a hypothetical analysis of 120 subjects gives \(\ell_1=-412.6\) for the one-class model and \(\ell_2=-407.9\) for the two-class model. The observed statistic is
The fitted one-class model is then used to generate \(B=999\) bootstrap data sets at the same subject, dose, covariate, and sampling-time design. Both models are refitted to each data set. Suppose 31 of the 999 bootstrap statistics are at least \(9.40\). Then
At a prespecified significance level of \(\alpha=0.05\), this estimated tail probability is below the threshold. Under the fitted one-class null model and this simulation procedure, statistics at least as large as \(9.40\) occurred in about 3.2% of the corrected bootstrap comparisons. This is evidence against that null model at the stated level; it is not the probability that the null is true, nor does it by itself establish that the two-class model is scientifically correct.
The estimate also has simulation error. A rough Monte Carlo standard error is
Thus a p-value near a decision threshold should not be treated as numerically exact; more replicates can reduce Monte Carlo error. Increasing \(B\) does not, however, correct a misspecified null model or a faulty fitting procedure.
When a Chi-Square Comparison Is Misleading
A common shortcut is to count the extra parameters in the two-class model and compare \(T_{\mathrm{obs}}\) with a chi-square distribution having that many degrees of freedom. For example, suppose a model adds three nominal parameters and the statistic is \(9.40\). A routine comparison with a \(\chi^2_3\) distribution would give an upper-tail probability of about \(0.024\). That number is not automatically valid for the mixture test: it relies on regularity conditions that fail when prevalence is at a boundary or class-specific parameters are unidentified under the null. The bootstrap estimate in the preceding example was \(0.0320\), illustrating that the reference distribution can matter for the conclusion.
A chi-square reference can be appropriate for likelihood-ratio tests in regular, nested models when its assumptions hold. The issue is not that likelihood-ratio tests are inherently invalid, but that a one-versus-two-class mixture comparison does not generally meet the standard conditions. Even special boundary results, which can yield mixtures of chi-square distributions in some settings, should not be imported without showing that their specific assumptions apply to the NLME mixture being tested.
Worked Example: Checking a Boundary Fit
Consider a hypothetical two-class fit with estimated class 1 prevalence \(\hat{\pi}=0.006\), and with class-specific typical clearances of \(5.0\) and \(5.1\) L/h. The small prevalence suggests that the fit is near the boundary \(\pi=0\). The clearances also suggest little separation, although their difference alone does not establish whether the class distributions are indistinguishable.
A useful check is to refit from multiple initial values and inspect whether the likelihood and estimates are stable. If different starts produce nearly the same likelihood but very different class 1 clearances or prevalences close to zero, the data may not identify those parameters reliably. The relevant null structure is then especially important: at \(\pi=0\), the class 1 clearance does not affect the likelihood; if the class distributions coincide, prevalence may be unidentified. This fit should not be interpreted as a clearly established second class just because its maximized likelihood is slightly larger.
For the formal comparison, simulate under the one-class null and refit both models with the same multistart strategy. The bootstrap distribution will reflect the tendency of the fitting procedure to find apparent separation under that null, provided the simulation and refitting are implemented adequately. A near-boundary estimate is a diagnostic prompt, not a substitute for the calibrated comparison or for scientific interpretation.
Diagnostics, Alternatives, and Limitations
Simulation calibration is only as credible as the model and computations that produce it. Check that the null model is a defensible data-generating model for the test, that simulated outcomes respect the observed design, and that both models converge reliably on observed and simulated data. Compare likelihoods across starts, monitor boundary estimates, and inspect whether the estimated bootstrap tail probability is stable as the number of replicates increases. A cluster of fitting failures, implausible parameter estimates, or highly variable results across starts can make the bootstrap calibration unreliable.
The parametric bootstrap answers a specific question: how often would this fitting procedure produce a statistic at least this large if data were generated from the fitted one-class model? It does not show that a latent class is biologically real, determine whether classes have an interpretable mechanism, or guarantee good prediction. The class assignment and parameter estimates still require careful interpretation. Results can also depend on the null model’s adequacy: if its structural or error assumptions are wrong, its simulated distribution may not represent the relevant null behavior.
| Approach | What it provides | Main caution |
|---|---|---|
| Ordinary chi-square likelihood-ratio test | A quick reference under regular nested-model conditions | Those conditions generally fail for a one- versus two-class mixture; parameter counting alone does not justify the reference distribution. |
| Parametric-bootstrap likelihood-ratio test | An empirical null distribution generated under a fitted one-class model | Depends on null-model adequacy, successful refitting, and enough replicates for useful precision. |
| Information criteria or predictive comparisons | Model-comparison information or evidence about predictive performance | They address different questions and do not provide the same calibrated null-hypothesis test. |
The earlier material on simulation checks for mixture models is useful for considering whether the fitted mixture can recover its parameters and class structure under specified scenarios. That kind of model-recovery check and the null bootstrap have different targets: recovery examines behavior under chosen generating models, whereas the test calibration estimates the statistic’s behavior specifically under the one-class null.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Why can a class-specific clearance parameter be unidentified when that class has prevalence zero?
- A two-class model has log likelihood \(-250.3\) and a one-class model has log likelihood \(-255.1\). Calculate the likelihood-ratio statistic.
- In 1,999 bootstrap replicates, 47 statistics are at least as large as the observed statistic. Calculate the corrected bootstrap p-value.
- Why does a small estimated prevalence suggest caution but not, by itself, prove that the second class is unnecessary?
- Name two important features that a null parametric-bootstrap simulation should preserve or reproduce.
Solutions. (1) At prevalence zero, the mixture likelihood contribution does not depend on that class’s likelihood or clearance, so the data provide no likelihood information about its clearance. (2) \(T=2\{(-250.3)-(-255.1)\}=2(4.8)=9.60\). (3) \(p_{\mathrm{boot}}=(1+47)/(1,999+1)=48/2,000=0.0240\). (4) A small prevalence places the fit near a boundary, where class-specific parameters can be weakly identified; whether the improvement is unusual must be assessed against a suitable null distribution. (5) Examples include the observed subjects and covariates, dosing and sampling-time design, and the fitted null model’s random-effects and residual-error assumptions.