Why specification choice needs more than a good fit
A mixture cure model represents survival as a combination of two processes: whether an individual is susceptible to the event, and, among susceptible individuals, when the event occurs. This separation gives analysts flexibility, but it also creates a model-selection problem. A covariate may help explain the cure probability, the event-time distribution among susceptible individuals, or both. Different choices can produce similar fitted survival curves while implying quite different cure fractions or latency effects.
The aim is not to find the specification with the largest likelihood at any cost. More flexible models will generally fit the observed data at least as well, but may estimate extra parameters imprecisely or predict poorly. A defensible comparison combines likelihood-based criteria, nested tests when their assumptions hold, and predictive checks. It also asks whether the available follow-up can distinguish the models in the first place.
The likelihood being compared
Let \(t_i\) be the observed follow-up time and let \(\delta_i=1\) indicate an observed event, with \(\delta_i=0\) indicating right censoring. Write \(x_i\) for incidence covariates and \(z_i\) for latency covariates. Under a model with cure probability \(\pi_i=\pi(x_i)\), the marginal survival function is \(\pi_i+(1-\pi_i)S_u(t\mid z_i)\). Thus, apart from a censoring-process factor that is common to the candidate models under independent censoring, an individual's likelihood contribution is:
An observed event must come from the susceptible group, which explains the factor \(1-\pi_i\) in the event contribution. A censored individual may be cured or may be susceptible and event-free up to \(t_i\), so both possibilities contribute to the censoring term. For independent individuals, the log likelihood is \(\ell=\sum_i\log L_i\).
Likelihood comparisons presume that candidates use the same individuals, event definition, time origin, and censoring information, and that censoring is non-informative conditional on the modeled covariates. If these conditions differ, a difference in likelihood can reflect a changed data basis or censoring model rather than a better incidence or latency specification. Models must also be fitted successfully: convergence warnings, implausible parameter estimates, or a nearly singular information matrix call for investigation before criteria are interpreted.
Likelihood-based criteria and nested tests
For a fitted model with \(k\) estimated parameters and maximized log likelihood \(\ell\), two common criteria are:
Smaller values indicate a preferred balance of fit and complexity under the respective criterion. AIC penalizes added parameters by 2 each and is commonly used as a relative measure of expected predictive information loss. BIC's penalty grows with sample size and can favor simpler models more strongly. Neither criterion establishes that a model is true, and the absolute value is not meaningful without a comparison set fitted to the same data and likelihood.
For nested models, the likelihood-ratio statistic is \(2(\ell_{\mathrm{larger}}-\ell_{\mathrm{smaller}})\). Under the null hypothesis and regular likelihood theory, it has approximately a chi-square distribution with degrees of freedom equal to the number of restrictions. This approximation requires identifiable parameters, an interior null value, and suitable smoothness and large-sample conditions. In cure models, a test that puts a cure probability on a boundary or makes latency parameters unidentified under the null may not have the usual chi-square reference distribution. Such cases need methods that account for the nonstandard null distribution, or a different comparison strategy; using the routine chi-square test without justification is not adequate.
Worked Example: Comparing specifications with AIC and BIC
Suppose three candidate models are fitted to the same hypothetical sample of \(n=240\) individuals, with the same censoring assumptions. Model A has a common Weibull latency distribution and a simple incidence model. Model B adds two incidence covariate parameters. Model C adds four parameters relative to Model A, including additional latency flexibility. The maximized log likelihoods and parameter counts are:
| Model | \(k\) | \(\ell\) |
|---|---|---|
| A | 5 | -182.4 |
| B | 7 | -178.9 |
| C | 9 | -177.8 |
For Model A, \(\mathrm{AIC}=2(5)-2(-182.4)=10+364.8=374.8\). For Model B, \(\mathrm{AIC}=2(7)+357.8=371.8\). For Model C, \(\mathrm{AIC}=2(9)+355.6=373.6\). AIC therefore ranks B first, followed by C and then A.
For BIC, \(\log(240)\approx5.4806\). The values are \(5(5.4806)+364.8=392.2\) for A, \(7(5.4806)+357.8=396.2\) for B, and \(9(5.4806)+355.6=404.9\) for C. BIC ranks A first. This disagreement is a reason to examine the scientific purpose, uncertainty, and predictions—not a reason to declare one criterion correct by default. The ranking also says nothing about whether the cure fraction is well identified.
The difference between AIC values can be written directly as a trade-off: when moving from a smaller to a larger model, the gain in log likelihood must be large enough to offset the added-parameter penalty. Specifically, if the larger model adds \(r\) parameters, then
The expression follows by substituting the AIC definition for each model and collecting terms. A negative difference favors the larger model by AIC; it does not by itself establish that every added coefficient is meaningfully different from zero.
Worked Example: A nested test for incidence covariates
In the hypothetical comparison above, suppose Model A is nested in Model B, and Model B adds two incidence coefficients. Assume the likelihood is regular for testing that both coefficients are zero: the cure probability remains identifiable under the null, the null values are interior to the parameter space, and the other parameters are identifiable. The likelihood-ratio statistic is:
The reference distribution is chi-square with \(df=2\). Its upper-tail probability is approximately \(0.0302\). At a significance level of \(\alpha=0.05\), this is evidence against the joint null that both added incidence coefficients are zero, conditional on the stated assumptions. It is not evidence that the model is true, nor does it show that the cure fraction is precisely estimated. The test asks a specific nested-model question; it does not replace comparison with a non-nested latency alternative or predictive evaluation.
Predictive comparison and the role of validation
A model-selection criterion calculated on the fitting data is not a substitute for checking predictions on data that did not determine the fitted coefficients. A predictive comparison can evaluate survival or event probabilities at clinically relevant times, preferably across covariate patterns rather than only as an overall average. For right-censored validation data, the observed-data log score can use the event or censoring likelihood contribution, provided the censoring assumptions and any censoring adjustment are handled consistently. Alternatively, a time-specific binary outcome can be scored when status through that horizon is known; censoring before the horizon must not be treated as absence of an event.
A common log score is the sum of log probabilities assigned to the observed outcomes; larger (less negative) scores are better. Calibration also matters: among people assigned a particular predicted event probability, observed event frequency should be broadly compatible with that prediction. Use the earlier material on residual and calibration diagnostics for cure models to guide diagnostic displays. For model selection, the practical point is to compare predictions at meaningful horizons and in relevant subgroups, not merely to compare a single global score.
Worked Example: Comparing predictions at a fixed horizon
Consider a hypothetical validation set of 20 individuals whose event status is completely known through a prespecified horizon of five years. Six experienced the event by five years and 14 did not. Model A predicts the same five-year event probability \(q_A=0.25\) for each person; Model B predicts \(q_B=0.32\). For these fully ascertained binary outcomes, the total log score is \(6\log(q)+14\log(1-q)\).
For Model A, the score is \(6\log(0.25)+14\log(0.75)=-8.3178-4.0275=-12.3453\). For Model B, it is \(6\log(0.32)+14\log(0.68)=-6.8366-5.3993=-12.2359\). Model B has the higher score by about \(0.1094\) log-score units over all 20 individuals, so it predicts these outcomes slightly better by this measure. The observed event proportion is \(6/20=0.30\), also closer to Model B's predicted probability than to Model A's.
This small validation set does not establish a decisive predictive advantage: the score difference is modest, and its sampling variability matters. The calculation would not be valid if individuals censored before five years were simply counted as event-free. In that situation, the censoring must be addressed, for example through a suitable censoring-adjusted score or a likelihood-based evaluation of the observed validation data.
Weak identification: when candidates tell similar stories
A cure fraction is weakly identified when the observed follow-up does not contain enough information to distinguish a persistent susceptible tail from a cured group. In that setting, increasing the estimated cure probability can often be offset by changing the latency distribution so that susceptible individuals have longer event times. This trade-off may leave the fitted survival curve and likelihood nearly unchanged while changing the estimated cure fraction substantially.
Consider a hypothetical profile-likelihood analysis that fixes the cure fraction at successive values and re-estimates the other parameters. Suppose the maximized log likelihoods are -120.00 at cure fraction 0.25 and -120.08 at cure fraction 0.40. The difference is only 0.08 log-likelihood units, even though the estimated cure fractions differ by 0.15. This flat profile indicates that the data provide little discrimination between those values under the fitted latency model. It would be misleading to report 0.25 as if it were a precisely established population cure fraction merely because it is the optimizer's single best estimate.
Useful warning signs include a broad or asymmetric profile likelihood, large standard errors, unstable estimates across starting values, high parameter correlations, and substantial changes in the cure estimate when the latency tail assumption changes. Examine uncertainty and profile behavior, not only the optimizer's point estimate or the information criterion ranking. AIC and BIC can still compare fitted models, but small differences in either should not be treated as strong evidence when the key parameters are weakly identified.
Practical comparison strategy and common pitfalls
State which covariates enter incidence, which enter latency, and what latency distributions or functional forms are being compared. Retain scientifically defensible alternatives rather than generating a large set of specifications and reporting only the winner.
Use the same observations and censoring formulation, verify convergence, and investigate implausible estimates or numerical instability before comparing criteria.
Report AIC or BIC for candidate rankings. Use a likelihood-ratio test for a nested restriction only after checking that its regularity conditions hold. Do not present that test as a general contest between non-nested models.
Evaluate calibrated predictions at relevant times and covariate patterns, and examine whether the cure fraction and latency parameters are stable enough to support the intended interpretation.
A common mistake is to choose the model with the largest likelihood, or to treat a lower AIC as proof that its cure estimate is correct. The likelihood cannot decrease when a nested model gains freely estimated parameters, so its maximum alone rewards added flexibility. AIC and BIC penalize complexity, while validation examines predictive behavior; none removes weak identification or cures a misspecified tail. The correction is to state what each comparison measures, report meaningful uncertainty and sensitivity, and avoid interpreting a small ranking difference as decisive.
The main limitation is that model choice depends on both the candidate set and the information available in follow-up. A model can rank well among the candidates considered while all candidates extrapolate poorly beyond observed times. If the clinical question concerns long-term cure, the analysis should make clear when conclusions are driven by assumptions about the latency tail rather than by observed late follow-up.
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- A model adds three parameters and increases the maximized log likelihood by 2.0. What is the change in AIC, and which model does AIC favor?
- Two models are not nested, but their likelihoods can be computed on the same data. Is the ordinary likelihood-ratio chi-square test automatically appropriate?
- At a fixed horizon, some validation subjects are censored before that horizon. Why is it incorrect to count all of them as event-free?
- A profile likelihood is nearly flat over cure fractions from 0.20 to 0.45. What does this suggest, and what should be reported or checked?
Solutions. (1) The AIC change is \(2(3)-2(2.0)=2.0\), so the larger model has AIC two points higher and AIC favors the smaller model. (2) No. The standard likelihood-ratio reference distribution is for nested models under regularity conditions; non-nested comparisons need another approach, such as information criteria or predictive validation. (3) Censoring before the horizon means the person's event status by that horizon is unknown, not known to be event-free. Use an appropriate censoring-aware evaluation. (4) The data weakly distinguish those cure fractions under the model. Report uncertainty, inspect the profile and sensitivity to latency assumptions, and avoid presenting the point estimate as precise.