Individual and Population Survival Predictions

Two Prediction Targets

A survival model with random effects can answer two related but distinct questions. For a person whose random effect is specified, what is the probability that the event has not occurred by time \(t\)? For a population described by a random-effects distribution, what proportion is expected to remain event-free by that time? The first is a conditional prediction; the second is a population prediction, also called a marginal prediction.

These predictions differ because survival is a nonlinear function of the random effect. A prediction for a typical random effect, such as \(b=0\), generally does not equal the average prediction across the population. Keeping the target clear is important when interpreting model-based probabilities and communicating them in clinical terms.

Definition: Let \(T\) be an event time, \(x\) the covariates for a prediction target, and \(b\) a random effect. The conditional survival function is \(S(t\mid x,b)=P(T>t\mid x,b)\). If \(b\) has population distribution \(G\), the population survival function at those covariates is \(S_{\mathrm{pop}}(t\mid x)=\int S(t\mid x,b)\,dG(b)\). The conditional event probability by time \(t\) is \(1-S(t\mid x,b)\); the population event probability is \(1-S_{\mathrm{pop}}(t\mid x)\).

From a Conditional Hazard to Survival

Suppose a fitted model specifies the hazard \(h(t\mid x,b)\), the instantaneous event rate at time \(t\) for a subject with covariates \(x\) and random effect \(b\), conditional on remaining event-free until \(t\). The cumulative hazard is the integral of that hazard over time. Under the usual continuous-time survival model, the conditional survival function is the exponential of the negative cumulative hazard:

H(t\mid x,b)=\int_0^t h(u\mid x,b)\,du, \qquad S(t\mid x,b)=\exp\{-H(t\mid x,b)\}.

For an interval \((t_1,t_2]\), the conditional probability of an event in the interval is \(S(t_1\mid x,b)-S(t_2\mid x,b)\). This follows because an event in that interval means surviving past \(t_1\) but not past \(t_2\). The population interval probability is the corresponding difference of population survivals.

Assumptions and target: These calculations use the fitted conditional hazard and its implied survival distribution, with the stated covariates and random effect. A population prediction additionally requires a specified random-effects distribution representing the target population. For time-varying covariates, the hazard must be integrated along the covariate history being predicted. The calculations describe model-based probabilities; they do not by themselves account for uncertainty in estimated model parameters.

A short derivation explains the exponential survival formula. Conditional on \(x\) and \(b\), survival satisfies \(dS(t)/dt=-h(t\mid x,b)S(t)\), with \(S(0\mid x,b)=1\). Dividing by survival and integrating gives \(\log S(t\mid x,b)=-\int_0^t h(u\mid x,b)\,du\), hence \(S(t\mid x,b)=\exp\{-H(t\mid x,b)\}\).

Worked Example: Conditional Survival With a Specified Random Effect

Consider a hypothetical proportional-hazards model with a constant baseline hazard of \(0.10\) per month and a random intercept \(b\) entering as \(\exp(b)\). Set the covariate contribution to zero for this prediction, and specify \(b=0.40\). The conditional hazard is \(h(t\mid b)=0.10\exp(b)\) per month. Calculate survival through 6 months, the probability of an event by 6 months, and the probability of an event in months 6 through 12.

The hazard for this specified random effect is \(0.10\exp(0.40)=0.149182\) per month. Since it is constant, \(H(t\mid b)=0.10\exp(0.40)t\). Using the unrounded hazard in the survival calculation:

S(6\mid b=0.40)=\exp\{-6(0.10\exp(0.40))\}\approx 0.408569.

Thus, the conditional probability of an event by 6 months is \(1-0.408569=0.591431\), or approximately 0.5914. These are probabilities for a subject with the specified \(b=0.40\), not population averages.

At 12 months, \(S(12\mid b=0.40)=\exp\{-12(0.10\exp(0.40))\}\approx0.166929\). Therefore:

P(6<T\le12\mid b=0.40) =S(6\mid b=0.40)-S(12\mid b=0.40) \approx 0.408569-0.166929=0.241640.

The probability of an event during months 6 through 12 is approximately 0.2416 for this specified random effect. The calculations retain the unrounded hazard until the final reporting step.

Population Predictions Require Integration

Let \(b\) follow a distribution \(G\) in the target population. For each possible value of \(b\), the model gives a conditional survival probability. Averaging those probabilities over \(G\) gives the population survival:

S_{\mathrm{pop}}(t\mid x) =E_b\{S(t\mid x,b)\} =\int \exp\{-H(t\mid x,b)\}\,dG(b).

The expectation is taken after converting each random effect into its conditional survival probability. It is generally incorrect to average the random effects first, substitute that average into the hazard, and call the resulting survival a population prediction. For example, a mean-zero random effect does not imply that population survival equals survival calculated at \(b=0\).

Worked Example: Integrating Over a Normal Random-Effects Distribution

Continue with the hypothetical constant-baseline-hazard model \(h(t\mid b)=0.10\exp(b)\) per month, now for a population with \(b\sim N(0,0.04)\). The second argument is the variance, so the random-effect standard deviation is \(0.20\). Approximate population survival at 6 months using three-point Gauss-Hermite quadrature. For a normal random effect with standard deviation \(\omega\), this approximation uses random-effect values \(0\) and \(\pm\sqrt{3}\omega\), with weights \(2/3\) at zero and \(1/6\) at each nonzero value.

Here the nodes are \(b=0\) and \(b=\pm\sqrt{3}(0.20)=\pm0.346410\). The conditional survivals at 6 months are:

S(6\mid b=0)=\exp(-0.6)=0.548812,

S(6\mid b=0.346410)=\exp\{-0.6\exp(0.346410)\}\approx0.4281,

S(6\mid b=-0.346410)=\exp\{-0.6\exp(-0.346410)\}\approx0.6542.

Applying the quadrature weights gives:

S_{\mathrm{pop}}(6) \approx \frac{2}{3}(0.548812)+\frac{1}{6}(0.4281)+\frac{1}{6}(0.6542) \approx0.5463.

The corresponding population event probability by 6 months is approximately \(1-0.5463=0.4537\). This is a numerical approximation to the integral, not survival calculated at the mean random effect. In this example, survival at \(b=0\) is about 0.5488, distinct from the population prediction of about 0.5463.

In practice, the integral may have no closed-form expression, even when the conditional survival has a simple formula. Numerical quadrature or simulation can approximate it. Quadrature evaluates conditional survival at carefully chosen random-effect values and combines the results with weights. Simulation draws many random effects from \(G\), calculates conditional survival for each draw, and averages those values. Increasing the number of quadrature points or simulation draws can improve numerical accuracy, but does not fix an incorrect model or an inappropriate target distribution.

Worked Example: Population Event Probability Over an Interval

Using the same model and three-point quadrature, calculate the population probability of an event during months 6 through 12. The conditional survival at 12 months at the three quadrature nodes is approximately \(S(12\mid b=0)=\exp(-1.2)=0.3010\), \(S(12\mid b=0.346410)\approx0.1833\), and \(S(12\mid b=-0.346410)\approx0.4280\). Weighting these conditional survivals gives:

S_{\mathrm{pop}}(12) \approx \frac{2}{3}(0.3010)+\frac{1}{6}(0.1833)+\frac{1}{6}(0.4280) \approx0.3026.

The unrounded three-node quadrature results for survival are approximately 0.54626 at 6 months and 0.30255 at 12 months. Thus the population interval probability is:

P(6<T\le12)_{\mathrm{pop}} =S_{\mathrm{pop}}(6)-S_{\mathrm{pop}}(12) \approx0.54626-0.30255=0.24371\approx0.2437.

The population probability of an event by 12 months is approximately \(1-0.30255=0.69745\), or 0.6975. The interval probability is not obtained by evaluating the hazard at the mean random effect; it averages the interval probabilities over the random-effects distribution, equivalently subtracting the two integrated survival predictions.

Why the Two Predictions Differ

The conditional prediction answers a question about a specified \(b\). A population prediction averages across the distribution of \(b\) appropriate to the target population. They are not competing estimates of one quantity; they address different questions. In particular, \(b=0\) is the mean of a mean-zero random-effects distribution, but it does not identify the mean of the survival probabilities.

The distinction also matters when individual information is available. A conditional prediction for a particular subject may use a subject-specific random-effect estimate informed by that subject’s data. A population prediction instead integrates over the population distribution, without conditioning on that subject’s event history. These should not be described interchangeably: the first is subject-specific, while the second describes the model’s average prediction for the stated population and covariates.

Prediction	Calculation	Question answered
Conditional	\(S(t\mid x,b)\)	What is survival for the specified covariates and random effect?
Population (marginal)	\(\int S(t\mid x,b)\,dG(b)\)	What is average survival across the random-effects distribution at these covariates?
Survival at the mean random effect	\(S(t\mid x,E[b])\)	What is survival at the mean random-effect value? This is not generally a population average.

Diagnostics, Limitations, and Common Mistakes

Before reporting a population prediction, check that the random-effects distribution used in the integration matches the intended population and that the covariates are defined for the prediction target. For example, integrating random effects while fixing covariates at one value gives an average over random effects at that covariate value; it does not also average over a covariate distribution. If predictions are needed for a population with varying covariates, the target covariate distribution must also be represented.

Check numerical integration by increasing the quadrature order or simulation size and confirming that the reported probabilities are stable to the required precision. Also verify basic probability constraints: survival must be between 0 and 1 and non-increasing in time, and an interval probability must be nonnegative. A negative interval probability can indicate reversed time points, inconsistent calculations, or a numerical or coding error.

Mistake: treating survival at \(b=0\) as the population average. A mean-zero random effect only means \(E[b]=0\); it does not imply \(E\{S(t\mid b)\}=S(t\mid0)\). Integrate conditional survival over the distribution.
Mistake: averaging hazards and then calculating survival. In general, \(\exp\{-E[H(t\mid b)]\}\ne E[\exp\{-H(t\mid b)\}]\). Average the conditional probabilities for a population prediction.
Mistake: confusing conditional and marginal interpretations. State whether a probability is for a specified random effect or averaged over the population distribution, and name the time horizon and covariates.
Mistake: reporting numerical precision unsupported by the calculation. Retain unrounded values through subtraction and report a sensible number of significant figures. The interval probability is the difference of two survivals calculated on the same target and scale.

Population predictions depend on the fitted hazard, the random-effects distribution, and the specified covariates. If any of these are misspecified, integration can be performed accurately yet still produce a misleading prediction. Moreover, the numerical integration shown here treats model parameters as fixed; uncertainty intervals for predictions require additional treatment of parameter uncertainty.

Key takeaway: Calculate survival conditional on a random effect first. For a population prediction, average those conditional survival probabilities over the random-effects distribution. Do not replace that integration with survival evaluated at the mean random effect.

Check Your Understanding

Answer each question, then compare with the solutions below.

In a conditional survival model, what does \(S(8\mid x,b)\) describe?
For a population prediction, should you average \(b\) first and calculate one survival, or average the conditional survival probabilities?
If \(S_{\mathrm{pop}}(4)=0.80\) and \(S_{\mathrm{pop}}(9)=0.55\), what is the population probability of an event in \((4,9]\)?
Does a mean-zero random effect guarantee that survival at \(b=0\) equals population survival? Explain.

Solutions. (1) It is the probability of remaining event-free beyond 8 time units for the specified covariates and random effect. (2) Average the conditional survival probabilities over the random-effects distribution: \(S_{\mathrm{pop}}(t\mid x)=E_b\{S(t\mid x,b)\}\). (3) The interval probability is \(0.80-0.55=0.25\), or 25%. (4) No. In general, \(E_b\{S(t\mid b)\}\ne S(t\mid E[b])\), because survival is a nonlinear function of the random effect.