Tutorials › Multi-Omics Data Integration › Dynamic Survival Prediction From Joint Models

Clinical Outcomes: Survival and Longitudinal Data · Tutorial 464 of 597

Dynamic Survival Prediction From Joint Models

Use a joint model to update an individual's probability of remaining event-free over a future horizon, conditioning correctly on survival and all omics data observed so far.

Advanced 12 min read

What You'll Learn

  • Define dynamic survival prediction conditional on being event-free at the prediction time.
  • Derive how observed survival and longitudinal omics measurements update the random-effects distribution.
  • Calculate future survival probabilities by averaging over a conditional posterior distribution.
  • Distinguish a baseline prediction from an updated conditional prediction.
  • Identify assumptions, diagnostics, and common errors in interpreting dynamic predictions.

Why Predictions Must Change With New Measurements

A survival prediction made at enrollment answers a different question from one made months later, after an individual has remained event-free and new omics measurements have arrived. The later prediction should use both sources of information. A trajectory that appears high-risk may alter the predicted future event probability, while remaining event-free for a period can itself change what is plausible about that individual's latent risk.

A joint model provides a coherent way to update predictions because it represents the longitudinal omics trajectories and event time through shared latent effects. The key is to condition on the individual's measurements and survival up to the prediction time, then average future survival over the resulting distribution of those latent effects. Substituting a single fitted trajectory or a single risk score generally does not perform that averaging.

Model and Conditional Prediction

Let \(T_i\) be the event time for individual \(i\), and let \(Y_i^{(s)}\) denote all omics measurements observed for that individual by time \(s\). Write \(b_i\) for the individual's shared random effects across the omics trajectories and survival process, and \(X_i\) for included baseline covariates. For omics layer \(k\), let \(m_{ik}(t;b_i)\) be its modelled underlying trajectory at time \(t\), conditional on \(b_i\). A common joint-model hazard is

$$ h_i(t\mid b_i,X_i) = h_0(t)\exp\left\{\gamma^\top X_i+ \sum_{k=1}^{K}\alpha_k m_{ik}(t;b_i)\right\}. $$

Here \(h_0(t)\) is the baseline hazard, \(\gamma\) is the vector of baseline-covariate coefficients, and \(\alpha_k\) links the current underlying trajectory in layer \(k\) to the event hazard. Conditional on \(b_i\) and the covariates, the corresponding survival function is

$$ S_i(t\mid b_i,X_i) = \exp\left\{-\int_0^t h_i(v\mid b_i,X_i)\,dv\right\}. $$

The model must specify a joint distribution for the random effects and a longitudinal measurement model for the observed assays. A product of separate assay likelihoods is appropriate only if the model assumes those measurements are conditionally independent given the random effects and included covariates. The survival submodel also specifies how the trajectory enters the hazard; other association structures are possible, but the prediction calculation must match the chosen model.

Definition: At a prediction time \(s\), the dynamic event-free survival prediction at future time \(u>s\) is \(P(T_i>u\mid T_i>s,Y_i^{(s)},X_i)\). It is the probability of remaining event-free through \(u\), conditional on being event-free at \(s\), the omics data observed by \(s\), and the included baseline covariates.

For fixed model parameters, the latent effects have a conditional posterior distribution proportional to their prior distribution, the likelihood of the observed omics data, and the probability of surviving to \(s\):

$$ p(b_i\mid T_i>s,Y_i^{(s)},X_i) \propto S_i(s\mid b_i,X_i)\, p(Y_i^{(s)}\mid b_i,X_i)\, p(b_i\mid X_i). $$

The survival factor is essential: the prediction concerns individuals who have survived event-free to \(s\), not a random individual from the enrollment population. With this posterior, the conditional prediction is

$$ P(T_i>u\mid T_i>s,Y_i^{(s)},X_i) = \int \frac{S_i(u\mid b_i,X_i)}{S_i(s\mid b_i,X_i)} \,p(b_i\mid T_i>s,Y_i^{(s)},X_i)\,db_i. $$

The survival ratio is the probability of surviving from \(s\) to \(u\) conditional on \(b_i\). Integrating that ratio over the posterior averages predictions across the latent effects still plausible given the individual's history.

Why the Updating Formula Holds

The formula follows by conditioning on the unobserved random effects. Conditional on \(b_i\), the probability of surviving past \(u\), given survival past \(s\), is \(S_i(u\mid b_i,X_i)/S_i(s\mid b_i,X_i)\). Average this conditional probability over the distribution of \(b_i\) for individuals with the observed history. That distribution is obtained by Bayes' rule.

$$ \begin{aligned} &P(T_i>u\mid T_i>s,Y_i^{(s)},X_i)\\ &=\int P(T_i>u\mid T_i>s,Y_i^{(s)},X_i,b_i) p(b_i\mid T_i>s,Y_i^{(s)},X_i)\,db_i\\ &=\int \frac{S_i(u\mid b_i,X_i)}{S_i(s\mid b_i,X_i)} p(b_i\mid T_i>s,Y_i^{(s)},X_i)\,db_i . \end{aligned} $$

This argument assumes the fitted joint model correctly describes the relevant longitudinal and event-time distributions, including their dependence through \(b_i\). It also assumes the observed history is used only up to \(s\), and that the individual is known to be event-free at \(s\). In applications, censoring and the observation process must be handled consistently with the model and prediction target; treating an event or assay recorded after \(s\) as known information would invalidate this prediction.

Worked Example: Updating Survival With the History at Six Months

Consider a hypothetical joint model with two possible latent risk states. Given the omics history and survival through \(s=6\) months, the posterior probability of the higher-risk state is \(0.70\), and that of the lower-risk state is \(0.30\). For the next six months, their hazards are constant at \(0.12\) and \(0.04\) per month, respectively. Calculate the probability of remaining event-free to month 12, conditional on being event-free at month 6 and the observed history.

The assumptions for this calculation are that the stated state probabilities are already conditional on the history and survival through month 6, and that the given hazards apply from month 6 to month 12. Under a constant hazard \(\lambda\), the six-month conditional survival is \(\exp(-6\lambda)\). Therefore,

$$ \begin{aligned} P(T_i>12\mid T_i>6,Y_i^{(6)},X_i) &=0.70\exp(-0.12\times 6)+0.30\exp(-0.04\times 6)\\ &=0.70(0.4867523)+0.30(0.7866279)\\ &=0.3407266+0.2359884\\ &=0.5767149\approx 0.5767. \end{aligned} $$

The predicted conditional probability of remaining event-free from month 6 through month 12 is about \(57.67\%\). The complementary probability of an event in that interval is \(1-0.5767149=0.4232851\), or about \(42.33\%\). These are conditional predictions for the hypothetical individual and history described, not statements about the probability that a model parameter or hypothesis is true.

New Measurements Update the Posterior Again

When another assay arrives, its likelihood changes the posterior distribution of the random effects. If the individual also remains event-free between prediction times, that survival contributes further information. The update is not simply a substitution of the new measured value into the hazard: the measurement informs the latent trajectory, and the latent trajectory informs the future hazard.

Worked Example: Incorporating a New Assay at Month Eight

Continue the hypothetical two-state example. At month 6, the conditional state probabilities were \(0.70\) for the higher-risk state and \(0.30\) for the lower-risk state. The individual is event-free at month 8, and a new assay value has likelihood \(0.20\) under the higher-risk state and \(0.60\) under the lower-risk state. The hazards remain \(0.12\) and \(0.04\) per month. Find the conditional probability of remaining event-free through month 14, given the history through month 8.

First update the state probabilities using both the assay likelihood and survival from month 6 to month 8. The unnormalized weights are

$$ \begin{aligned} w_H&=0.70(0.20)\exp(-0.12\times2)=0.1101279,\\ w_L&=0.30(0.60)\exp(-0.04\times2)=0.1661609. \end{aligned} $$

The weights sum to \(0.2762888\), so the normalized probability of the higher-risk state is \(0.1101279/0.2762888=0.3986\); the lower-risk probability is \(0.6014\). The six-month survival from month 8 to month 14 is then

$$ \begin{aligned} P(T_i>14\mid T_i>8,Y_i^{(8)},X_i) &=0.3986\exp(-0.12\times6)+0.6014\exp(-0.04\times6)\\ &\approx 0.3986(0.4867523)+0.6014(0.7866279)\\ &\approx 0.6671. \end{aligned} $$

Thus, the conditional event-free survival probability from month 8 through month 14 is approximately \(66.71\%\), and the complementary event probability is \(33.29\%\). The new assay shifts posterior probability toward the lower-risk state, but the individual’s survival through month 8 is also part of the update.

Baseline and Dynamic Predictions Answer Different Questions

A baseline prediction uses information available at enrollment. A dynamic prediction at \(s\) uses the data available by \(s\) and conditions on survival through \(s\). These should not be described as interchangeable estimates of the same quantity. Even a prediction made at \(s\) using no new omics assay must account for having survived to \(s\); a prediction that also uses the new assay incorporates further individual-specific evidence.

Worked Example: Comparing an Entry Prediction With a Conditional Prediction

Suppose the two risk states above each have probability \(0.50\) at enrollment, with constant hazards \(0.12\) and \(0.04\) per month. First calculate survival to month 12 from enrollment:

$$ \begin{aligned} P(T_i>12\mid X_i) &=0.50\exp(-0.12\times12)+0.50\exp(-0.04\times12)\\ &=0.50(0.2369278)+0.50(0.6187834)\\ &=0.4278556\approx0.4279. \end{aligned} $$

At month 6, survival alone changes the state weights. They are proportional to \(0.50\exp(-0.72)\) and \(0.50\exp(-0.24)\), giving probabilities approximately \(0.3823\) and \(0.6177\). Consequently, the prediction for survival from month 6 to month 12 using entry information and survival to month 6, but no additional omics information, is

$$ P(T_i>12\mid T_i>6,X_i) = \frac{0.4278556}{0.50\exp(-0.72)+0.50\exp(-0.24)} = \frac{0.4278556}{0.6366901} \approx0.6720. $$

This \(67.20\%\) conditional prediction differs from the entry prediction of \(42.79\%\), because the former concerns survival over months 6–12 among individuals already event-free at month 6. A further dynamic prediction can differ again if omics data collected by month 6 shift the state probabilities. The comparison is meaningful only when the conditioning information and time interval are stated.

Computation, Diagnostics, and Limitations

The integral over \(b_i\) is usually evaluated numerically, for example by quadrature, Monte Carlo draws from the fitted conditional distribution, or posterior simulation. With simulated draws \(b_i^{(1)},\ldots,b_i^{(M)}\) from \(p(b_i\mid T_i>s,Y_i^{(s)},X_i)\), the conditional survival prediction is approximated by

$$ \widehat{P}(T_i>u\mid T_i>s,Y_i^{(s)},X_i) = \frac{1}{M}\sum_{m=1}^{M} \frac{S_i(u\mid b_i^{(m)},X_i)} {S_i(s\mid b_i^{(m)},X_i)}. $$

If model parameters are treated as uncertain, prediction should also average over their uncertainty rather than report only a plug-in estimate. The formula above describes the calculation conditional on fixed parameters; parameter uncertainty can be incorporated by drawing parameters as well as random effects from their fitted uncertainty distribution.

Useful checks include confirming that predicted survival is between 0 and 1 and does not increase as \(u\) increases. Compare predictions with observed outcomes using appropriate survival-prediction assessment, and examine calibration across relevant prediction times and risk ranges. Also inspect whether the fitted longitudinal trajectories and event model adequately represent the observed data. Poor trajectory fit, implausible hazard shapes, weakly estimated association parameters, or highly influential individuals can make individual updates unstable. Estimates may be particularly uncertain when a person has few measurements or assays with substantial measurement error.

A time-dependent Cox model using observed covariate values can be a useful alternative when its covariate representation and measurement process suit the question. However, using a noisy observed assay value directly is not the same as integrating over latent trajectory uncertainty in a joint model. A baseline-only survival model is simpler but cannot update its individual prediction using subsequent omics history. No approach removes the need to define the prediction time, horizon, available data, and target population precisely.

Common pitfall: A tempting calculation is to average the two hazards first and then exponentiate, treating the result as the mixture survival. In general, \(\sum_j q_j\exp(-\lambda_j d)\) is not equal to \(\exp\{-d\sum_j q_j\lambda_j\}\). The correct prediction averages state-specific survival probabilities using the conditional posterior weights. Also, the weights must reflect survival through the prediction time as well as the observed omics history.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

  1. In one sentence, state the target of \(P(T_i>u\mid T_i>s,Y_i^{(s)},X_i)\).
  2. Why does the posterior distribution of \(b_i\) include \(S_i(s\mid b_i,X_i)\)?
  3. For two latent states with posterior probabilities \(0.25\) and \(0.75\), and six-month hazards \(0.10\) and \(0.02\) per month, write the conditional survival calculation for the next six months.
  4. Does an entry prediction of survival to month 12 answer the same question as a month-6 prediction of survival to month 12? Explain.
  5. Name one reason a dynamic prediction could be poorly calibrated even if its arithmetic is correct.

Solutions. (1) It is the probability of remaining event-free through \(u\), conditional on being event-free at \(s\), the omics history through \(s\), and included baseline covariates. (2) Because the individual is known to have survived event-free to \(s\), and that survival is informative about latent effects associated with event risk. (3) The calculation is \(0.25\exp(-0.10\times6)+0.75\exp(-0.02\times6)=0.25(0.5488)+0.75(0.8869)\approx0.8024\). (4) No. The entry prediction covers enrollment through month 12; the month-6 prediction is conditional on survival to month 6 and covers the subsequent interval. (5) For example, the joint model may misspecify the longitudinal trajectories or hazard association, or its measurement-error assumptions may be unsuitable.

Clinical Biostats