Tutorials › Joint Modeling of Longitudinal and Time-to-Event Data › Deriving the Competing-Risks Joint Likelihood

Competing Risks and Recurrent Events · Tutorial 402 of 697

Deriving the Competing-Risks Joint Likelihood

Learn how each subject’s longitudinal and competing-risks data combine into a conditional likelihood and how shared random effects are integrated out.

Advanced 13 min read

What You'll Learn

  • Derive the cause-specific event-density and right-censoring factors conditional on shared random effects.
  • Combine the event-history contribution with the subject’s longitudinal-data density.
  • Integrate over the random-effects distribution to obtain the observed-data subject likelihood.
  • Work through event and censoring contributions using hypothetical numerical examples.
  • Recognize why plugging in an estimated random effect or multiplying marginal likelihoods can be incorrect.
  • Check the assumptions and numerical calculations underlying a competing-risks joint likelihood.

Why the Joint Likelihood Needs an Integral

A competing-risks joint model links repeated measurements to the time and cause of an event through shared latent subject characteristics. Conditional on those characteristics, the longitudinal observations and event process can be modeled together using familiar density and hazard contributions. But the random effects are not observed. The likelihood for the data actually recorded must therefore average the conditional likelihood over their distribution.

The key bookkeeping is straightforward but consequential: at an observed event, the contribution is the hazard for the cause that occurred multiplied by survival from all causes up to that time. At right censoring, the contribution is that same all-cause survival, with no event-hazard factor. The subject’s longitudinal-data density multiplies either contribution before the random effects are integrated out.

Definition: For subject \(i\), let \(y_i\) denote the vector of observed longitudinal measurements, \(b_i\) the vector of shared random effects, and \(t_i\) the observed follow-up time. Let \(\delta_i=1\) indicate an observed event and \(\delta_i=0\) indicate right censoring. If an event is observed, \(k_i\in\{1,\ldots,K\}\) denotes its cause. Conditional on covariates, write \(f_b(b_i)\) for the random-effects density, \(f_y(y_i\mid b_i)\) for the longitudinal-data density, and \(\lambda_{ik}(t\mid b_i)\) for the cause-\(k\) hazard. The cumulative cause-specific hazard is \(\Lambda_{ik}(t\mid b_i)=\int_0^t\lambda_{ik}(u\mid b_i)\,du\).

Assumptions and Conditional Event Contributions

The derivation below uses the following assumptions. Subjects are independent conditional on model parameters and observed covariates. The longitudinal density is correctly specified conditional on \(b_i\), including any dependence among the subject’s repeated measurements. Conditional on \(b_i\) and covariates, the event process has cause-specific hazards \(\lambda_{ik}\); its total hazard is the sum of those hazards. The longitudinal measurement process and event process are conditionally independent given \(b_i\) and covariates.

For the displayed likelihood, assume follow-up and longitudinal observation times are fixed or noninformative, and censoring is independent of the event process and the shared random effects given observed covariates. We condition on the censoring scheme, or omit its nuisance contribution when it is parameter-distinct and ignorable. If censoring depends on an unobserved random effect, its mechanism generally cannot simply be omitted; it must be addressed in the model.

Given \(b_i\), the probability of remaining free of every event cause through time \(t\) is determined by the total hazard. Since the cause-specific hazards add, the total cumulative hazard is \(\sum_{k=1}^K\Lambda_{ik}(t\mid b_i)\), and the event-free survival function is its negative exponential.

$$ S_i(t\mid b_i)=\exp\left\{-\sum_{k=1}^K\Lambda_{ik}(t\mid b_i)\right\}. $$

For an event of cause \(k\) at time \(t\), the conditional density is \(\lambda_{ik}(t\mid b_i)S_i(t\mid b_i)\). To see why, over a short interval of width \(dt\), the chance of remaining event-free until \(t\) is \(S_i(t\mid b_i)\), and the conditional chance of cause \(k\) occurring in that interval is approximately \(\lambda_{ik}(t\mid b_i)dt\). Their product, divided by \(dt\), gives the density. If instead the subject is censored at \(t\), the event-history contribution is \(S_i(t\mid b_i)\): the subject is known to have had no event before censoring, but no cause-specific event was observed at that time.

Worked Example: A Cause-Specific Event Density

Suppose there are two causes, with constant hazards conditional on a fixed random effect \(b\): \(\lambda_1(t\mid b)=0.10e^b\) and \(\lambda_2(t\mid b)=0.05e^b\), per year. For a hypothetical subject with \(b=-0.5\), calculate the event-history contribution for a cause-1 event at \(t=2\) years.

The total hazard is \((0.10+0.05)e^{-0.5}=0.15(0.606531)=0.0909796\) per year. Thus the all-cause survival to 2 years is \(S(2\mid b)=\exp\{-2(0.0909796)\}=\exp(-0.181959)=0.8336\).

The cause-1 hazard is \(0.10e^{-0.5}=0.0606531\) per year. Therefore the event density is \(0.0606531(0.8336)\approx0.05056\) per year. This is a density contribution, not a probability of an event at exactly 2 years. It uses the cause-1 hazard for the event factor and both causes in the survival factor.

Combining Longitudinal Data and Integrating Random Effects

Conditional independence gives the subject-level contribution before integration: multiply the longitudinal-data density by the appropriate event-history contribution. Define the cause-specific event factor to be \(\lambda_{ik_i}(t_i\mid b_i)\) when \(\delta_i=1\), and 1 when \(\delta_i=0\). Then both event and censoring cases share the same all-cause survival factor.

$$ L_i(b_i)=f_y(y_i\mid b_i) \left\{\lambda_{ik_i}(t_i\mid b_i)\right\}^{\delta_i} \exp\left\{-\sum_{k=1}^K\Lambda_{ik}(t_i\mid b_i)\right\}. $$

When \(\delta_i=0\), the cause label is unused and the hazard factor is 1. When \(\delta_i=1\), the factor is the hazard of the observed cause. The observed-data likelihood contribution is obtained by integrating over the random-effects distribution:

$$ L_i=\int L_i(b_i)f_b(b_i)\,db_i =\int f_y(y_i\mid b_i) \left\{\lambda_{ik_i}(t_i\mid b_i)\right\}^{\delta_i} \exp\left\{-\sum_{k=1}^K\Lambda_{ik}(t_i\mid b_i)\right\} f_b(b_i)\,db_i. $$

This integral is the defining step in the marginal likelihood: all information from the longitudinal measurements and event history updates how plausible each value of \(b_i\) is, and the conditional contributions are averaged with respect to the model’s random-effects distribution. With independent subjects, the full likelihood is the product of their subject contributions, \(L=\prod_i L_i\). In general, it is a product of integrals, not one integral formed by pooling all subjects’ random effects.

Derivation: Conditional on \(b_i\), factor the joint density of the observed longitudinal data and event history as \(f_y(y_i\mid b_i)\) times the event-history contribution, using their assumed conditional independence. The event-history contribution is the cause-specific event density or censoring survival derived above. Since \(b_i\) is unobserved, the observed-data density follows by the law of total probability: integrate this conditional joint density against \(f_b(b_i)\). Independence across subjects then yields the product of the resulting subject-level integrals.

Worked Example: A Subject Likelihood With Two Measurements

Consider a hypothetical subject with two measurements \(y_{i1}=0.2\) and \(y_{i2}=0.7\), each having a normal density with mean \(b_i\) and standard deviation 1, conditionally independent given \(b_i\). The subject has a cause-1 event at 2 years. Use the two-point random-effects distribution \(P(b_i=-0.5)=P(b_i=0.5)=0.5\) and the cause-specific hazards from the previous example. Calculate the subject likelihood by exact averaging over these two possible random-effect values.

At \(b_i=-0.5\), the two measurement densities are \(\phi(0.2;-0.5,1)=0.3123\) and \(\phi(0.7;-0.5,1)=0.1942\). Their product, the conditional longitudinal density, is approximately \(0.06064\). The cause-1 event density at 2 years is \(0.10e^{-0.5}\exp\{-2(0.15e^{-0.5})\}\approx0.05056\) per year. The conditional joint density is therefore approximately \(0.06064(0.05056)=0.003066\), in measurement-density units per year.

At \(b_i=0.5\), the measurement densities are \(\phi(0.2;0.5,1)=0.3814\) and \(\phi(0.7;0.5,1)=0.3910\), giving a longitudinal density of approximately \(0.1491\). The event density is \(0.10e^{0.5}\exp\{-2(0.15e^{0.5})\}\approx0.10054\) per year. The conditional joint contribution is approximately \(0.1491389(0.1005398)=0.0149944\), or \(0.01499\) to four significant figures.

Average the two conditional contributions using their prior weights: \(L_i=0.5(0.0030659)+0.5(0.0149944)=0.0090302\). Thus the subject likelihood contribution is approximately \(0.009030\), with units determined by the two measurement densities and the event-time density. The integral is an average of the products conditional on \(b_i\); it is not generally the product of separately averaged longitudinal and event contributions.

Censoring, Checks, and Alternative Calculations

Worked Example: Censoring Versus an Observed Competing Event

Suppose a hypothetical subject has \(b_i=0\), constant cause-1 and cause-2 hazards of \(0.10\) and \(0.05\) per year, and is followed for 2 years. Find the event-history contribution if the subject is censored at 2 years; then find it if a cause-2 event is observed at 2 years.

The total hazard is \(0.10+0.05=0.15\) per year. The censoring contribution is the all-cause survival, \(S(2)=\exp\{-2(0.15)\}=\exp(-0.30)=0.7408\). There is no event-hazard multiplier for a censored observation.

For a cause-2 event at 2 years, multiply the same survival by the cause-2 hazard: \(0.05\exp(-0.30)=0.03704\) per year. The event density is smaller than survival alone because it includes the cause-2 event rate at that time. If longitudinal data were also observed, their conditional density would multiply either event-history contribution before integrating over \(b_i\).

A useful check is to temporarily ignore the cause label. Summing the densities for all possible event causes at time \(t\) gives \(S_i(t\mid b_i)\sum_k\lambda_{ik}(t\mid b_i)\), the total event density. The corresponding survival factor remains \(\exp\{-\sum_k\Lambda_{ik}(t\mid b_i)\}\). This verifies that the risk of every cause is accounted for once in survival, while an observed event contributes only its own cause-specific hazard.

The conditional likelihood can be evaluated at each \(b_i\) and then integrated using numerical methods such as quadrature; the earlier material on quadrature for random-effects integrals provides the relevant computational background. Whatever method is used, inspect the conditional log-likelihood over plausible random-effect values and check that integration is stable as the numerical accuracy is increased. In the data, verify event-cause coding, event and censoring times, units, and the at-risk endpoint. A frequent coding error is to use only the observed cause’s cumulative hazard in the survival term. That incorrectly treats subjects as surviving other causes without risk.

The integral also distinguishes the joint likelihood from two tempting alternatives. Plugging an estimated random effect into \(L_i(b_i)\) ignores uncertainty about \(b_i\) and is not the observed-data likelihood. Separately marginalizing the longitudinal and event components and multiplying them generally loses their dependence induced by the shared random effects. The correct order is to multiply conditionally and then integrate.

Common pitfall: “The subject had cause 1, so only cause 1 belongs in the likelihood.” This is wrong for the survival part: the subject had to remain free of all causes until the event time. Use the observed cause’s hazard as the event factor, but use the sum of all cause-specific cumulative hazards in the survival factor.

This likelihood relies on its assumptions. Misspecified longitudinal distributions, an incorrect cause-specific hazard model, informative visit times, dependent censoring, or an inadequate random-effects distribution can distort inference. The integral also need not have a closed form, so numerical approximation error is a practical concern. These limitations do not change the likelihood’s structure, but they do affect whether it represents the data-generating process and whether its evaluation is reliable.

Check Your Understanding

Work through these short problems; the solutions follow each question.

  1. With two constant cause-specific hazards \(0.20\) and \(0.10\) per year and fixed random effects, what is the density contribution for a cause-1 event at 1 year? Solution: The total hazard is \(0.30\), so survival is \(\exp(-0.30)\). The cause-1 density is \(0.20\exp(-0.30)=0.1482\) per year.
  2. With constant hazards \(0.05\) and \(0.03\) per year, what is the event-history contribution for censoring at 4 years? Solution: Censoring contributes survival only: \(\exp\{-4(0.05+0.03)\}=\exp(-0.32)=0.7261\).
  3. Why is \(\int f_y(y_i\mid b_i)g_i(b_i)f_b(b_i)\,db_i\) generally not equal to \(\{\int f_y(y_i\mid b_i)f_b(b_i)\,db_i\}\{\int g_i(b_i)f_b(b_i)\,db_i\}\), where \(g_i\) is the event-history factor? Solution: Both factors depend on the same \(b_i\), so their conditional product is generally associated across values of \(b_i\). Integration must average that product; multiplying the separate averages discards the dependence induced by the shared random effects.
  4. If the event time is known but its cause is not recorded, what event density factor replaces the cause-specific factor? Solution: Sum over the possible causes: \(S_i(t\mid b_i)\sum_{k=1}^K\lambda_{ik}(t\mid b_i)\). The survival term still includes \(\sum_k\Lambda_{ik}(t\mid b_i)\).
Clinical Biostats