Uncertainty Intervals for Individual Risk

What an Uncertainty Interval Describes

A patient-specific prediction is often reported as a single number, such as a 20% risk of an event within the next year. That number summarizes a prediction target, but it does not show how much the estimate would vary across plausible model parameters and plausible values of the patient’s latent random effects. An uncertainty interval adds that information. It is an interval for the predicted quantity, not a guarantee that the patient’s eventual outcome lies inside it.

Let \(s\) be a prediction landmark, \(w\) a prediction horizon, and let \(H_i(s)\) denote patient \(i\)’s observed history available at \(s\). Suppose the patient is event-free at \(s\). For a given model parameter vector \(\theta\) and patient random effects \(b_i\), define the conditional survival probability and horizon risk by

These quantities are conditional on the landmark survival and the information specified in the prediction target. If the target averages over uncertain future marker paths, that integration belongs inside the calculation of \(S_i\) or \(R_i\) for each draw. An uncertainty interval can then describe variation in that integrated prediction across parameter and random-effects draws. Drawing future paths as well may add another source of variation; it should be done only when that variation belongs to the target, not merely to make the interval wider.

From Repeated Draws to an Interval

Suppose we have \(M\) draws \((\theta^{(m)},b_i^{(m)})\), for \(m=1,\ldots,M\), from a distribution appropriate to the prediction problem. For each paired draw, calculate the target risk using the same landmark history, horizon, and conditioning:

A point estimate may be the average of the draw-specific risks, \(\bar{R}_i=M^{-1}\sum_{m=1}^M R_i^{(m)}\), or their median. A percentile interval uses lower and upper quantiles of the same risk draws. For the numerical examples below, empirical quantiles use the nearest-rank convention: sort the draws as \(R_{(1)}\le\cdots\le R_{(M)}\), and set \(Q_p=R_{(\lceil Mp\rceil)}\) for \(0<p\le1\). Other software may use interpolation, so the convention should be reported when it matters.

The pairing of \(\theta^{(m)}\) and \(b_i^{(m)}\) matters. They can be dependent given the patient’s data, and their dependence can affect predicted risks. Use draws from their appropriate joint distribution; independently shuffling parameter and random-effects draws can create combinations that are not representative of the fitted model.

For Bayesian draws, quantiles of the posterior distribution of \(R_i(s,w)\) form a posterior credible interval, conditional on the model, prior, and observed data. For a frequentist confidence interval, repeated fits or another justified sampling procedure must approximate the sampling distribution of the estimator; a percentile interval has its intended coverage only when that procedure and its assumptions support it. These labels are not interchangeable. Neither interpretation means that the fixed, actual risk parameter has a frequentist probability of lying in a particular realized interval.

Survival and Risk Intervals

For every draw, survival and risk are related by \(R_i^{(m)}=1-S_i^{(m)}\). The map \(g(S)=1-S\) is decreasing. Consequently, if a particular interval for survival is \([S_L,S_U]\), its image under this map is \([1-S_U,1-S_L]\): the endpoints must be reversed. This is an endpoint transformation, not a general license to transform empirical quantiles while keeping their labels unchanged.

In particular, empirical quantiles from finite or discrete draws depend on the quantile convention. The quantile identity relating a risk quantile to a survival quantile need not hold under nearest-rank quantiles. The safest procedure is to transform every survival draw into a risk draw and calculate the desired risk quantiles directly. If an interval is already reported for survival and the question is simply its image under \(R=1-S\), transform and reorder its endpoints.

Assumptions, Diagnostics, and Failure Modes

An interval is meaningful only if its draws represent the uncertainty relevant to the stated target. In a Bayesian analysis, the draws should come from the joint posterior conditional on the observed longitudinal history and survival through the landmark. In a resampling-based frequentist analysis, the resampling and refitting procedure must represent the sampling variation for the prediction being reported. In either case, the event-free landmark condition and the patient’s observed history must be applied consistently across draws.

A further check is whether the calculation estimates the intended risk. If future marker paths are integrated over in the target, each draw should use that integration. If an inner Monte Carlo approximation is used, its numerical error should be small relative to the between-draw variation; otherwise noisy inner estimates can make the interval appear wider or less stable than the uncertainty in the target itself. Increasing the number of inner draws or checking against a more accurate integration method can reveal this problem.

• Check draw behavior. For simulation-based draws, assess convergence, mixing, autocorrelation, and effective sample size. Tail quantiles may be unstable even when the point estimate appears stable.
• Check quantile stability. Recalculate the endpoints with more draws or across independent chains or resamples. Large endpoint changes suggest inadequate Monte Carlo precision.
• Check the scale. Risks must lie between 0 and 1. Values outside that range indicate a calculation error; values near a boundary may also make symmetric intervals inappropriate.
• Check the target and conditioning. Confirm the horizon, landmark survival condition, marker-history information, and treatment of future marker paths. An interval for a different conditional or marginal quantity does not answer the same question.
• Inspect the shape. Skewed or multimodal risk draws can make a symmetric estimate-plus-or-minus-standard-error interval misleading. Quantile intervals preserve the [0,1] bounds, though they may still conceal multiple modes or other important features.

The main alternative to a percentile interval is a standard-error-based interval, such as an estimate plus or minus a normal critical value times its standard error. That approach can be compact and useful when the sampling distribution is approximately symmetric and away from the boundaries. It can be poor for a bounded, skewed risk, and the standard error must reflect the relevant uncertainty components. A point estimate alone is simplest but omits uncertainty. Whichever summary is used, the draws and interval method should match the estimand and the intended interpretation.

Interpretation and Common Pitfall

A common mistake is to write, “There is a 90% chance this patient’s event risk is between 6% and 22%,” without specifying a Bayesian probability statement or defining the random quantity. In a frequentist confidence-interval analysis, that wording is not the usual interpretation: the coverage statement concerns the long-run performance of the procedure under repeated sampling. In a Bayesian analysis, a posterior probability statement can be made about the model-defined risk conditional on the model, prior, and data, but it still concerns uncertainty in that risk, not the patient’s binary outcome.

Another mistake is to present an interval without saying whether it reflects parameter uncertainty, random-effects uncertainty, future-path uncertainty, or a combination. A narrow interval can be misleading if it conditions on a plug-in random effect while the intended target includes uncertainty about that effect. Conversely, adding randomly sampled future paths may answer a broader question than intended. The interval is only as interpretable as the uncertainty distribution and conditioning that produced it.