Why Covariate Selection Must Respect the Transition Model
In a multi-state regression, a covariate can have an effect on one transition, several transitions, or all transitions under a shared coefficient restriction. Selecting terms therefore means deciding not only whether a variable belongs in a model, but also which transition-specific coefficients are included or constrained to be equal. A comparison is interpretable only when the model's transition structure and baseline-hazard specification are made explicit.
For example, suppose the states and allowed transitions are fixed, and each transition has its own baseline hazard. Comparing a model with an age term on one transition to a model with no age term on that transition can isolate the contribution of that term. But if the second model also changes the allowed transitions or replaces separate baseline hazards with a common baseline, its likelihood difference cannot be attributed just to age.
For a transition from state \(g\) to state \(h\), a common proportional-hazards form is
Here \(\lambda_{0,gh}(t)\) is the baseline hazard for that transition, and \(Z(t)\) is the covariate vector observed for the individual at time \(t\). Candidate models may include a covariate's coefficient on selected transitions, omit it on others, or impose a shared coefficient across a specified group of transitions. These are different restrictions on the regression parameters; they do not change the underlying transition process.
Nested Tests for Covariate Terms
A model is nested within another when it can be obtained by imposing restrictions on the larger model's parameters. For instance, omitting a transition-specific coefficient imposes the restriction that it equals zero. Requiring two transition coefficients to be equal imposes a different restriction. The likelihood-ratio test compares the maximized likelihoods under these restrictions.
The degrees of freedom count the restrictions, not the number of variables named in the model. Adding one covariate with separate coefficients on three transitions usually adds three parameters and gives \(q=3\). Adding that covariate under a single shared coefficient constraint usually adds one parameter and gives \(q=1\). The count can differ if constraints or dependencies make some coefficients redundant.
Why the Likelihood-Ratio Statistic Has This Reference Distribution
The justification is Wilks' theorem. In a regular model, the log-likelihood near the true parameter is approximately a quadratic function: a fitted parameter's log-likelihood gain is approximately one-half its score, measured in the information metric, squared. Under \(H_0\), the larger model can move in \(q\) additional parameter directions that the reduced model fixes by restrictions. The resulting gain is approximately the sum of squares of \(q\) independent standard normal quantities.
This approximation assumes that the null parameter is not on a problematic boundary, the relevant parameters are identifiable, and the likelihood is correctly specified and sufficiently informative. In transition analyses, the practical implications are important: the models must use the same subjects, event information, risk sets, and censoring treatment, and the added terms must not be unestimable because of empty or nearly empty transition-specific risk sets.
Worked Example: Testing Whether Two Transition Effects Can Be Shared
Consider a hypothetical study with a fixed transition structure and separate, flexible baseline hazards for each transition. Two transitions include the same binary exposure. The reduced model constrains its coefficients to be equal across the two transitions; the larger model estimates a separate coefficient for each. Both models use the same subjects, risk sets, and partial likelihood, and the fitted information supports both coefficients.
Suppose the reduced model has maximized log partial likelihood \(\ell_0=-248.6\), while the larger model has \(\ell_1=-245.9\). The equality restriction is one independent restriction, so \(q=1\).
Calculate the statistic.
Against a chi-squared distribution with 1 degree of freedom, the two-sided likelihood-ratio p-value is approximately \(0.020\). At a pre-specified \(\alpha=0.05\), this is evidence against the equality constraint: the data provide evidence that the exposure coefficients differ between these transitions. It is not proof that the effects truly differ, nor does it establish which transition has the larger effect without examining the estimates and their uncertainty.
The comparison is appropriate because only the coefficient constraint changed. The transition set and separate baseline-hazard specification stayed fixed. If one model had instead used a common baseline hazard, this test would not isolate coefficient sharing.
Information Criteria for Candidate Models
Likelihood-ratio tests address a specified nested null hypothesis. Information criteria instead compare relative fit and complexity across a collection of candidate models, which need not be nested. For a maximized full log-likelihood \(\ell\) and \(k\) estimated parameters, the Akaike information criterion is \(\mathrm{AIC}=-2\ell+2k\). The Bayesian information criterion is \(\mathrm{BIC}=-2\ell+k\log(n)\), where \(n\) is the number of independent observational units under the applicable likelihood formulation. In both criteria, smaller values indicate a better relative trade-off within the candidate set.
The parameter count must include estimated regression and baseline-hazard parameters when they are part of the likelihood. It must also account consistently for constrained coefficients. For a Cox model, ordinary AIC calculations based on the partial likelihood are not automatically comparable to full-likelihood AIC values; use a criterion defined for the fitted likelihood and implemented consistently across candidates. Do not mix likelihoods or parameter-count conventions.
AIC is motivated by estimating relative expected information loss, while BIC applies a stronger complexity penalty that grows with sample size. Neither criterion gives the probability that a model is true. Their numerical values are most useful as relative comparisons among candidates fitted to the same data under a coherent likelihood and compatible analysis conditions.
Worked Example: Comparing Age-Term Specifications With AIC
Suppose a hypothetical analysis compares two models with identical transition structures and the same piecewise-exponential baseline-hazard specification. Both are fitted to the same subjects. Model A has no age term, maximized log-likelihood \(\ell_A=-255.0\), and \(k_A=8\) estimated parameters. Model B adds an age coefficient for each of three transitions, has \(\ell_B=-250.0\), and \(k_B=11\).
Calculate AIC for each model.
Model B has an AIC 4 points lower than Model A. Among these two candidates, AIC therefore favors the model with transition-specific age coefficients. This comparison does not establish that age has a causal effect, that all three coefficients are nonzero, or that Model B is well calibrated. It reports a relative fit-complexity preference under the stated candidate set.
If the analysis objective is a formal test of the joint null that all three age coefficients equal zero, the nested likelihood-ratio test is a separate calculation. AIC and a hypothesis test answer different questions and can lead to different choices.
Holding the Comparison Constant
Before interpreting a test or information criterion, record exactly what changed between candidates. In particular, specify whether baseline hazards are transition-specific or shared, and whether they are Cox, parametric, or piecewise exponential. A flexible transition-specific baseline and a parametric baseline make different modeling assumptions and can yield different maximized likelihoods. Changing both the baseline and covariate terms at once confounds their contributions.
| Comparison goal | Keep fixed | What may change |
|---|---|---|
| Test whether a transition term is needed | Transition set, risk sets, data, likelihood, and baseline specification | Include versus omit the specified term |
| Test shared versus transition-specific effects | Transition set, data, likelihood, and baseline specification | Coefficient equality constraints |
| Compare candidate covariate sets by AIC or BIC | Data, likelihood definition, and transition structure | Covariate terms and their coefficient constraints |
| Compare different baseline specifications | Data and transition structure, at minimum | Baseline form; interpret as a broader model comparison, not a covariate-only comparison |
A counting-process or partial-likelihood analysis requires particular care. All candidate models must use compatible risk sets and the same event and censoring information. If different models discard different subjects because of missing covariates, a likelihood difference also reflects a change in data. Likewise, a partial-likelihood comparison cannot be treated as an ordinary full-likelihood comparison merely because software prints a likelihood-like quantity.
Worked Example: Testing a Joint Addition on Two Transitions
A hypothetical model has a fixed transition structure, separate baseline hazards, and a specified parametric baseline form. The reduced model omits a continuous biomarker from transitions \(1\to2\) and \(2\to3\); the larger model adds one coefficient on each. The same likelihood and subjects are used, and both added coefficients are estimable. Suppose \(\ell_0=-310.4\) and \(\ell_1=-307.7\). The joint null imposes two restrictions, so \(q=2\).
Likelihood-ratio test:
For a chi-squared distribution with 2 degrees of freedom, the p-value is approximately \(0.0672\). At \(\alpha=0.05\), the test does not provide sufficient evidence to reject the joint null. This is not evidence that both effects are exactly zero; it says the observed likelihood gain is not unusually large under this joint null at the chosen level.
Suppose the reduced model has \(k_0=9\) parameters and the larger model has \(k_1=11\). Their AIC values are
AIC favors the larger model by \(1.4\) points, even though the nested test does not reject at \(0.05\). There is no contradiction: the test evaluates evidence against a null at a chosen error rate, whereas AIC ranks candidates using a fit-complexity penalty.
Diagnostics, Pitfalls, and Limits
A selection procedure can produce a precise-looking result even when its comparison is invalid or its estimates are unstable. Before relying on the result, check the fitted model and the comparison itself.
- Verify nesting and degrees of freedom. Confirm that the reduced model is obtained by explicit restrictions and count independent restrictions. Do not assign degrees of freedom by counting variable names alone.
- Check transition-specific information. A coefficient on a transition with very few events or little covariate variation may be unstable or poorly identified. Large standard errors, extreme estimates, convergence warnings, or near-singular information are reasons to question ordinary chi-squared approximations.
- Check model comparability. Confirm that the same observations, risk sets, event definitions, transition structure, and likelihood conventions were used. Keep baseline-hazard choices fixed when the goal is to compare covariate terms.
- Account for repeated selection. Repeatedly testing many terms and retaining those with small p-values makes the nominal p-values overly optimistic for the resulting selected model. AIC-based search also has uncertainty not reflected by simply reporting the winning model.
- Revisit model adequacy. Selection does not validate proportional hazards, covariate functional forms, censoring assumptions, or the chosen transition structure. Earlier material on residual diagnostics and baseline-hazard choices addresses those model checks.
A common mistake is to compare two models, see a smaller AIC for one, and conclude that its extra covariate causes better outcomes or must be retained. The correction is to interpret the result narrowly: AIC ranks the specified candidates by relative fit and complexity under the chosen likelihood. It neither establishes causality nor guarantees the selected model is adequate. Similarly, failure to reject a nested null is not proof that the omitted effects are absent.
Likelihood-ratio tests also rely on regularity. When an estimate is on a boundary, parameters are not identifiable under the null, or transition-specific data are extremely sparse, the standard chi-squared reference distribution may be inaccurate. In such settings, reconsider the model formulation and the available information; a routine nested test should not be presented as automatically valid. Model selection is especially limited when several candidates are nearly tied or when the scientific question requires a prespecified contrast rather than a search over many models.
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- A covariate is added with separate coefficients on four transitions. If all coefficients are estimable and no constraints link them, how many degrees of freedom does the corresponding joint likelihood-ratio test have?
- Two models differ in both their covariate terms and whether baseline hazards are shared. Can their likelihood-ratio difference be interpreted as a test of the covariate terms alone? Why or why not?
- A reduced model has log-likelihood \(-80.0\), and a nested model has log-likelihood \(-77.5\). The larger model adds two independent coefficients. Calculate the likelihood-ratio statistic and state the reference distribution under the null.
- In a candidate set fitted to the same data and likelihood, Model A has AIC 214 and Model B has AIC 211. What can be concluded, and what cannot?
Solutions. (1) Four degrees of freedom, because the larger model adds four independent transition-specific coefficients. (2) No. The likelihood difference reflects both the covariate change and the baseline-hazard change, so it does not isolate the covariate terms. (3) \(G^2=2\{(-77.5)-(-80.0)\}=5.00\), compared approximately with a chi-squared distribution with 2 degrees of freedom if regularity conditions hold. (4) AIC relatively favors Model B within this candidate set by 3 points. This does not show that Model B is true, prove an effect, or certify its adequacy.