Bootstrap Intervals for Dynamic State-Occupation Predictions

Why Bootstrap a Landmark Prediction?

A predicted state-occupation probability is an estimate, not a known quantity. It varies because the observed sample varies, and its uncertainty can involve several fitted components: transition-specific models, estimated transition probabilities, and the distribution of landmark covariates used to form a population prediction. The earlier material on parameter uncertainty in landmark state predictions motivates quantifying this variability. A bootstrap does so by repeatedly constructing new samples from the observed data and recalculating the prediction.

Fix a landmark time \(s\), a future horizon \(u\), a state \(k\), and a target defined by the prediction procedure. Write the estimated probability of occupying state \(k\) at time \(s+u\), conditional on the specified landmark information, as \(\hat p_k(u\mid s)\). The target and the horizon are held fixed across replicates. The interval considered here is generally pointwise: it concerns one state at one selected horizon, rather than simultaneous coverage of an entire trajectory or every state and horizon.

What Must Be Resampled and Refit?

The resampling unit must match the unit that is independent under the study design. For independent subjects, resample subjects. Each resampled subject must bring the complete record needed for prediction: covariates, state history, event times, event types, and censoring information. If subjects are clustered and observations within a cluster are dependent, resample independent clusters and retain all records within each selected cluster. Resampling individual rows of a repeated-measures or multi-state record would break the dependence structure and is generally inappropriate.

In each replicate, rerun every estimation step whose uncertainty is part of the intended inference. That commonly means refitting transition-specific models or recalculating transition estimates, then recomputing the state-occupation prediction. If the target is a population-average prediction over the observed landmark cohort, reconstruct landmark eligibility and repeat the averaging according to the target definition. If the target is instead a prediction for a fixed covariate profile, keep that profile fixed while refitting the model. These choices answer different questions and should be specified before bootstrapping.

The landmark time and future horizon are typically fixed design choices for this interval. If the scientific target includes an estimated or data-selected landmark or horizon, that selection may also contribute uncertainty and needs a procedure designed for that target. Do not silently treat a data-dependent choice as fixed. The ordinary bootstrap also assumes that resampling the observed independent units is a reasonable approximation to the sampling process; it does not repair model misspecification or incorrect handling of censoring.

Intervals and Their Interpretation

With \(B\) successful replicates, sort the predictions as \(\hat p_{k,(1)}^*,\ldots,\hat p_{k,(B)}^*\). The percentile interval uses the empirical quantiles at the desired tail probabilities. For a nominal 95% interval, these are the 2.5th and 97.5th percentiles. This construction stays within \([0,1]\) when every replicate is a valid probability, although it is not guaranteed to have accurate coverage in small or irregular samples.

A normal-approximation interval instead uses the bootstrap standard deviation \(SE_{\text{boot}}\) as an estimate of the prediction's standard error:

The normal interval can extend below zero or above one, especially near a boundary or when the bootstrap distribution is skewed. The percentile interval respects the probability range but can also perform poorly when the estimator is biased, highly discrete, or irregular. Neither interval is automatically reliable merely because many replicates were run. The confidence level describes the long-run coverage of the procedure under its assumptions; it is not a probability statement about the fixed parameter after observing this particular interval.

Why Full Refitting Matters

A bootstrap should reproduce the variability of the estimator being reported. Holding a fitted transition model, a landmark covariate distribution, or another estimated component fixed omits that component’s estimation variability. It can therefore understate total uncertainty. However, the change in variability from refitting is not guaranteed to be upward: refitting can alter covariance among estimated components, and covariance can either increase or reduce the variability of their combined prediction.

The variance identity makes the point explicit. If a prediction depends on the difference of two estimates, \(\hat a-\hat b\), then \(\operatorname{Var}(\hat a-\hat b)=\operatorname{Var}(\hat a)+\operatorname{Var}(\hat b)-2\operatorname{Cov}(\hat a,\hat b)\). A strong positive covariance can make the full-refit difference much less variable than either estimate alone.

Multiple States, Diagnostics, and Alternatives

State-occupation probabilities at a common horizon form a vector whose entries sum to one. Bootstrap the vector from the same replicate, rather than resampling or analyzing each state independently. Its replicate covariance captures dependence between state probabilities. Marginal percentile intervals are still computed state by state, and their endpoints need not sum to one; they are not a confidence region for the entire probability vector.

Useful diagnostics include the fraction of replicates in which model fitting or prediction fails, the number of distinct resampled subjects, the distribution’s skewness and boundary mass, and how much the interval changes when \(B\) is increased. A large failure rate or extreme instability can indicate sparse landmark support, rare transitions, separation, or model-fitting problems. Do not simply discard failed replicates without investigating: the remaining replicates may no longer represent the intended sampling distribution. Report the resampling unit, number of attempted and successful replicates, interval method, and any failures or modifications.

The bootstrap complements, rather than replaces, other uncertainty methods. A delta-method interval uses estimated derivatives and a covariance estimate and may be efficient when the estimator is sufficiently regular and its sampling distribution is approximately normal. A parametric bootstrap simulates from a fitted model and refits it; it can be useful when the model is well specified, but its results depend more directly on that model’s distributional assumptions. The subject-level nonparametric bootstrap makes fewer parametric assumptions, but can be unstable with small samples, sparse states, or boundary probabilities. All these methods condition on the chosen model structure unless model-selection uncertainty is explicitly included.

Common Mistake and Practical Limits

• Resampling records instead of independent units. This can destroy within-person event-history dependence. Resample the complete subject record, or the independent cluster when clustering is present.
• Holding fitted components fixed without changing the target. This omits their estimation variability and can understate total uncertainty; covariance among refitted components means the direction of the net change is not guaranteed.
• Calling separate state intervals a joint 95% region. Marginal 95% intervals do not generally provide 95% simultaneous coverage across states or horizons.
• Treating a large \(B\) as a cure for weak data. More replicates reduce Monte Carlo error in estimated quantiles, but cannot create information absent from a small or sparse landmark sample.

Bootstrap intervals quantify sampling uncertainty for a specified estimator and target. They do not by themselves assess calibration, establish that the prediction model is correct, or account for unmeasured sources of uncertainty. When support is poor or replicates frequently fail, the limitation should be reported rather than concealed by a seemingly precise interval.