Why Bootstrap a Competing-Risk Prediction?
A competing-risk prediction depends on more than a regression coefficient. It also depends on the fitted cause-specific models, their estimated baseline hazards, and the calculation that combines the causes into a cumulative incidence. A standard error for a coefficient alone therefore does not describe uncertainty in the final predicted risk. Subject-level bootstrap resampling addresses this by repeating the entire estimation procedure on resampled subjects.
Let \(F_k(t\mid x)\) denote the cumulative incidence of cause \(k\) by time \(t\), for covariate profile \(x\). If \(\lambda_j(u\mid x)\) is the cause-specific hazard for cause \(j\), and \(S(u\mid x)\) is the probability of remaining free of every event just before time \(u\), then
A fitted competing-risk procedure estimates the ingredients in this expression and returns a prediction. The bootstrap asks how that prediction would vary across repeated samples of subjects from the population represented by the data, if the same fitting procedure were applied each time.
What Is Resampled and What Is Refit?
A subject’s record must travel as a unit. It includes the observed follow-up time, event indicator and cause (or censoring indicator), and the covariates or covariate history used in the model. If one subject is selected more than once, each copy contributes to that bootstrap fit. Do not resample event times separately from covariates: doing so would break the observed relationship between them.
The phrase “refit the entire procedure” is important. In each replicate, refit the cause-specific regression models, re-estimate their baseline hazards according to the chosen method, and recompute the cumulative incidence prediction. Keep fixed only choices that are genuinely prespecified parts of the analysis, such as the prediction horizon and a specified profile \(x\). If model selection, tuning, or another data-dependent step was part of the original procedure, repeat that step within each replicate too.
The standard nonparametric subject bootstrap assumes the observed subjects are independent and identically distributed draws from the target sampling population, or that the resampling scheme correctly represents the actual sampling design. The fitted prediction procedure must be defined in each resample, and the estimator should be sufficiently regular for the empirical distribution to approximate its sampling behavior. The bootstrap does not repair informative censoring, an incorrectly specified model, or a prediction horizon unsupported by the data.
From Replicates to an Interval
Write \(\widehat F_k^{*(b)}(t\mid x)\) for the prediction from bootstrap replicate \(b\), for \(b=1,\ldots,B\). A percentile interval uses the empirical quantiles of these \(B\) predictions. A nominal 95% percentile interval has endpoints at the 2.5th and 97.5th percentiles. These are confidence-interval endpoints: under repeated sampling and suitable conditions, the procedure is intended to cover the target risk about 95% of the time. They do not mean there is a 95% probability that a fixed risk lies in the particular interval.
The bootstrap standard error can also be estimated by the standard deviation of the replicates. That summary describes spread, but the percentile interval uses the replicate distribution directly and can reflect some asymmetry. Percentile intervals are not automatically best in every setting: bias-corrected and accelerated (BCa) or studentized intervals may be useful when their additional calculations are justified and stable. No interval type can compensate for a poorly supported prediction.
Worked Example: A Profile-Specific Risk Interval
Suppose a hypothetical study follows 640 independent subjects and fits the prespecified cause-specific models used for prediction. The target is the 5-year cumulative incidence of cause 1 for a fixed profile \(x\). The original fitted procedure gives \(\widehat F_1(5\mid x)=0.184\). The analyst draws \(B=2{,}000\) subject-level bootstrap samples, refits the full procedure in each, and sorts the resulting 2,000 predictions.
For illustration, suppose the empirical 2.5th and 97.5th percentiles of those predictions are 0.126 and 0.251. The percentile interval is therefore
In this hypothetical analysis, the estimated 5-year risk for the specified profile is 18.4%, with a 95% bootstrap percentile interval from 12.6% to 25.1%. This interval represents sampling uncertainty under the fitted procedure and its assumptions; it is not an interval for the proportion of future outcomes guaranteed to occur in that profile.
The profile \(x\) is held fixed in every replicate. The study subjects are resampled to refit the models, but the prediction is always evaluated at the same covariate values. This answers a different question from averaging predictions over a population whose covariate composition is itself being estimated.
Population-Average Predictions Need a Target Population
For a target population with covariate profiles \(x_1,\ldots,x_m\), a standardized average risk can be estimated as
The bootstrap must match the uncertainty represented by the target. If the target is a fixed, externally specified set of profiles, resample the model-development subjects, refit, and evaluate each fitted model on that same fixed set. If the target is the population represented by the study sample, resample subjects and average predictions over the covariate profiles in that same bootstrap sample. The latter includes variation in both model fitting and the empirical target composition. These procedures estimate uncertainty for different targets and should not be described interchangeably.
Worked Example: Average Risk in a Target Cohort
Suppose a hypothetical cohort has 400 subjects and is the target population. The estimand is the average predicted 3-year risk of cause 1 in this cohort’s source population. For each of \(B=1{,}500\) bootstrap replicates, the analyst samples 400 whole subject records with replacement, refits all cause-specific models, and averages the 3-year predictions over the 400 covariate profiles in that replicate.
Assume the original cohort’s standardized estimate is 0.072. The sorted bootstrap averages have empirical 2.5th and 97.5th percentiles of 0.051 and 0.098. The 95% percentile interval is
Thus, the estimated average 3-year risk in the target population is 7.2%, with a bootstrap interval from 5.1% to 9.8%. Each replicate averages over its own resampled profiles because the target is the population represented by the cohort, rather than a fixed list of 400 covariate records.
If instead those 400 profiles were a fixed target dataset and only model-fitting uncertainty were wanted, the analyst would retain those same profiles for every replicate and resample the development subjects. Averaging each refitted model’s predictions over this unchanged list would answer that conditional question.
Why Resampling Can Represent Prediction Uncertainty
The justification is that the empirical distribution of the observed subjects approximates the population distribution from which they were sampled. A bootstrap sample is drawn from that empirical distribution. For a sufficiently regular estimator, variation in the estimator across empirical-distribution resamples approximates its variation across repeated samples from the population.
More formally, let \(T(P)\) be the prediction produced by the complete fitting procedure when the subject-record distribution is \(P\), and let \(P_n\) be the empirical distribution of the observed records. The original estimate is \(T(P_n)\). The bootstrap uses a resampled empirical distribution \(P_n^*\) and computes \(T(P_n^*)\). Its conditional distribution of changes,
is used to approximate the sampling distribution of \(T(P_n)-T(P)\). This approximation is most reliable when the estimator behaves smoothly under small changes in the data and the relevant model fits remain well-defined. Repeating every data-dependent estimation step is what lets the resampling distribution reflect uncertainty from the procedure, rather than only one isolated component.
Worked Example: A Small Competing-Risk Bootstrap by Hand
Consider a deliberately simple hypothetical one-interval setting with 10 independent subjects, complete follow-up through the interval, and no covariates. Three subjects experience cause 1 and two experience cause 2; the remaining five experience neither event during the interval. The estimated cause-1 risk is the observed proportion, \(3/10=0.30\).
A bootstrap sample draws 10 of these subject records with replacement. One possible draw contains four copies of a cause-1 record, two copies of a cause-2 record, and four copies of a no-event record. Refitting the simple model gives a cause-1 risk of \(4/10=0.40\) for that replicate. The cause categories remain mutually exclusive; the cause-2 records are not reclassified as cause 1.
Across repeated bootstrap draws from these 10 observed records, the cause-1 count \(K^*\) has a binomial distribution with \(n=10\) and success probability \(0.30\), because each draw independently selects a cause-1 record with empirical probability \(3/10\). Thus \(K^*\sim\operatorname{Binomial}(10,0.30)\), and each risk replicate is \(K^*/10\). In this small example the distribution is discrete, and some bootstrap samples contain no cause-1 records. That is not a calculation error; it demonstrates why a small sample and a rare cause can produce unstable or uninformative intervals.
Diagnostics, Alternatives, and Limitations
Inspect more than the final two quantiles. Check the distribution of bootstrap predictions for skewness, unusual extremes, and concentrations at zero or one. Record how many replicates fail to converge, omit a cause, or cannot estimate a required quantity. A large failure fraction signals that the reported interval may describe only a selected subset of successful fits. Do not silently discard failures without investigating and reporting their cause.
Resampling must follow the sampling unit. If subjects are sampled in clusters, such as hospitals, resampling individual records as if independent will generally understate uncertainty; resample clusters when that matches the design and inferential target. For a stratified sample, resampling should respect the strata. Repeated measurements and time-varying covariate histories remain together within the selected subject record. The exact design may require a design-specific bootstrap rather than the simple independent-subject scheme.
| Approach | What it does | Useful consideration |
|---|---|---|
| Delta-method interval | Uses an estimated variance and a local approximation for the prediction. | Can be efficient when derivatives and variance estimates are reliable; may be less convenient for a complicated fitting pipeline. |
| Subject-level nonparametric bootstrap | Resamples observed subjects and refits the procedure. | Can accommodate a complex prediction calculation, but depends on representative resampling and stable refits. |
| Parametric bootstrap | Simulates records from a specified fitted data-generating model and refits or recalculates predictions. | Can be useful when the parametric model is credible, but inherits its distributional assumptions. |
A bootstrap interval is not a cure for model misspecification or inadequate follow-up. In particular, a resample cannot create information about a cause that is scarcely observed or about risk far beyond the observed follow-up. If the fitted procedure changes dramatically when a few subjects are omitted, replicate spread and failures are substantive warnings, not merely computational inconveniences. The choice between percentile, BCa, studentized, and other intervals also requires attention to bias, discreteness, and the stability of the quantities needed for the method.
Common Mistake
Check Your Understanding
Answer each question, then compare with the brief solutions.
- For a risk prediction at a fixed covariate profile, which quantities are held fixed across bootstrap replicates, and which are refit?
- A target cohort is treated as a fixed list of profiles. Should those profiles be resampled when the goal is to quantify only model-development uncertainty?
- Why must a subject’s event type, follow-up time, and covariates be resampled together?
- What does a nominal 95% bootstrap confidence interval mean, and what does it not mean?
- What should be investigated if many bootstrap replicates fail because a cause-specific model does not converge?
Solutions. (1) Hold the specified profile and prediction horizon fixed; resample subjects and refit every model and estimated component used to produce the risk. (2) No. Keep the fixed target profiles unchanged and resample the model-development subjects. (3) They are parts of one subject record; separating them would alter the observed association between covariates and outcomes. (4) Under repeated sampling and suitable conditions, the interval procedure is intended to cover the fixed target risk about 95% of the time; it does not assign a 95% probability to the parameter being in this realized interval. (5) Investigate event sparsity, model specification, and whether the resampling scheme or fitting procedure is appropriate; report the failures and do not treat a selectively filtered set of replicates as automatically valid.