Why Bootstrap the Cumulative Incidence Estimate?
At a fixed time, the Aalen–Johansen estimate gives a cumulative incidence estimate for each event cause. A pointwise confidence interval describes uncertainty in one such estimate at one selected time. The earlier material on pointwise confidence intervals for cumulative incidence develops variance-based intervals; here the emphasis is on a resampling approach that approximates the sampling distribution by repeatedly rebuilding the estimate from resampled subjects.
The key unit to resample is the subject, not an event time or an individual record fragment. Each sampled subject must carry their observed follow-up time, event or censoring status, cause if an event occurred, and any entry time used in the analysis. In every bootstrap sample, recompute the entire Aalen–Johansen estimate, then record its value at the time of interest.
How to Construct a Bootstrap Distribution
For subject \(i\), let \(Y_i\) be the observed time, the minimum of the event time and censoring time, and let \(\Delta_i\) identify whether an event was observed and, if so, its cause. A subject censored before \(\tau\) is still included in the resampling; their observed record is not converted into an event. At each distinct event time \(t_k\), the Aalen–Johansen calculation uses the risk set \(Y(t_k)\), the number \(d_{jk}\) of events from cause \(j\), and the total number \(d_k\) of events from all causes. In a replicate, these counts are recalculated, including the multiplicities created by resampling the same subject more than once.
For a simple right-censored cohort with no delayed entry, the replicate’s event-free survival estimate just before \(t_k\) is updated by the factor \(1-d_k/Y(t_k)\). The cause-\(j\) cumulative incidence estimate is the sum of the survival just before each time multiplied by the cause-\(j\) event fraction at that time:
Here the asterisks indicate that all counts and estimates come from the bootstrap replicate. With delayed entry, the replicate risk set must also respect each subject’s entry time, just as in the original analysis. Ties are handled using the same event-time convention in the original sample and every replicate.
Specify the cause \(j\) and fixed time \(\tau\) before examining the bootstrap results. The target is the pointwise estimate \(\hat{F}_{j}(\tau)\).
Draw \(n\) complete subject records with replacement. Keep each subject’s observed time, status, cause, and entry time together.
Rebuild the risk sets and cause-specific event counts, then calculate the replicate’s Aalen–Johansen estimate at \(\tau\).
Repeat for \(B\) replicates. Use the \(B\) estimates to calculate a bootstrap standard error or a pointwise interval.
Worked Calculations
Worked Example: One Bootstrap Replicate by Hand
Hypothetical data. Six subjects have these observed records: A has cause 1 at time 1; B has cause 2 at time 1; C has cause 1 at time 2; D has cause 2 at time 3; E is censored at time 2.5; and F is censored at time 3. Estimate cause 1 cumulative incidence at \(\tau=2\). There is no delayed entry.
In the original sample, six subjects are at risk at time 1, with one event from each cause. Thus \(\hat{S}(1)=1-(2/6)=4/6\), and cause 1 cumulative incidence increases by \(1/6\). At time 2, four subjects remain at risk and C has a cause 1 event. The increase is \((4/6)(1/4)=1/6\). Therefore \(\hat{F}_{1}(2)=1/6+1/6=1/3\).
Now consider one bootstrap draw of six records: A, A, B, C, E, F. A appears twice because sampling is with replacement. At time 1, the replicate risk set has six records, including two cause 1 events and one cause 2 event. The survival estimate after time 1 is \(1-3/6=3/6\), and cause 1 cumulative incidence has increased by \(2/6=1/3\). At time 2, the three remaining records C, E, and F are at risk; C has a cause 1 event. The increase is \((3/6)(1/3)=1/6\). The replicate estimate is therefore \(\hat{F}^{*}_{1}(2)=1/3+1/6=1/2\). This is one draw from the bootstrap distribution, not a new estimate to substitute for the original \(1/3\).
Worked Example: An Exact Bootstrap Distribution in a Tiny Sample
Hypothetical data. Three subjects are followed to a common time \(\tau\). By that time, one has an event from cause 1, one has an event from cause 2, and one has no observed event. All three are at risk at the common event time. The original estimate for cause 1 is \(1/3\).
In a bootstrap sample of size three, let \(K\) be the number of resampled copies of the cause 1 subject. Since each draw has probability \(1/3\) of selecting that subject, \(K\) has a binomial distribution with three draws and success probability \(1/3\). All events occur at the same time, so the replicate estimate is \(K/3\). Thus the possible estimates and their probabilities are:
| \(K\) | \(\hat{F}^{*}_{1}(\tau)=K/3\) | Probability |
|---|---|---|
| 0 | 0 | 8/27 |
| 1 | 1/3 | 12/27 |
| 2 | 2/3 | 6/27 |
| 3 | 1 | 1/27 |
For example, the probability of \(K=1\) is \(\binom{3}{1}(1/3)(2/3)^2=12/27\). The bootstrap mean is \(1/3\), and the bootstrap variance is \(2/27\), so the bootstrap standard error is \(\sqrt{2/27}=0.2722\). This calculation also follows from the binomial variance: \(\mathrm{Var}(K/3)=\{3(1/3)(2/3)\}/9=2/27\).
This exact distribution shows why a tiny sample can produce a very coarse and unstable bootstrap distribution. Using the usual percentile definition, its 2.5th percentile is 0 and its 97.5th percentile is 1, giving a 95% pointwise percentile interval from 0 to 1. The interval is extremely wide; resampling has not created information that the three-subject sample did not contain.
Summarizing Replicates and Interpreting Intervals
Write the \(B\) replicate estimates as \(\hat{F}^{*(1)}_{j}(\tau),\ldots,\hat{F}^{*(B)}_{j}(\tau)\). Their mean and sample standard deviation are:
The bootstrap standard error estimates the sampling variability of the original estimate. A common pointwise interval is the percentile interval: sort the replicate estimates and use their 2.5th and 97.5th percentiles as the endpoints for a nominal 95% interval. Its interpretation is about uncertainty in the cause-\(j\) cumulative incidence at the selected time \(\tau\), not uncertainty over an entire curve. The interval is not a statement that there is a 95% probability the fixed parameter lies within these observed endpoints.
Worked Example: Summarizing a Small Set of Replicates
Hypothetical bootstrap output. Suppose five replicate estimates at a prespecified time are 0.18, 0.20, 0.22, 0.26, and 0.34. These five draws are used only to demonstrate the arithmetic; they are far too few to estimate tail percentiles reliably.
Their mean is \((0.18+0.20+0.22+0.26+0.34)/5=0.24\). The squared deviations from 0.24 sum to \(0.0036+0.0016+0.0004+0.0004+0.0100=0.0160\). Therefore the sample standard deviation is \(\sqrt{0.0160/4}=0.06325\). This is the bootstrap standard error for this illustrative set.
Using the nearest-rank empirical quantile rule, the 2.5th and 97.5th percentiles of only five values are the minimum and maximum, 0.18 and 0.34. These endpoints demonstrate the percentile calculation, not a dependable 95% interval. In practice, use many more replicates—often at least 1,000, and more when tail quantiles are important—and check whether the reported quantiles are stable across runs.
Justification, Assumptions, and Diagnostics
The bootstrap treats the observed subjects as an empirical approximation to the population distribution of subject records. Resampling from that empirical distribution varies which subjects—and which event and censoring histories—contribute to each estimate. If the estimator behaves regularly under repeated sampling, the conditional distribution of the bootstrap estimates around the original estimate approximates the sampling distribution of that estimate. The standard deviation of the replicates then approximates its sampling standard error, and their quantiles can approximate sampling-distribution quantiles.
This justification depends on the study design and estimator, not merely on having software generate replicates. The usual subject bootstrap requires independent, identically sampled subjects from the target population, or a sampling scheme whose resampling unit matches its independent unit. The Aalen–Johansen estimator must also be appropriate for the observed data. In particular, the censoring condition needed for estimating cumulative incidence must hold: censoring should not selectively conceal event outcomes in a way the estimator fails to account for. Resampling does not correct informative censoring, poor follow-up, or a misspecified target population.
Check that the original analysis and all replicates use the same time origin, event coding, censoring rules, delayed-entry handling, and tie convention. Review the replicate distribution for impossible values, strong skewness, excessive mass at zero, or a few extreme values. Such features can reflect genuine boundary behavior or sparse events, but they can also expose coding or risk-set errors. If a cause is rare, many replicates may contain no events from that cause, producing a point mass at zero and an unreliable normal approximation. Increasing \(B\) reduces Monte Carlo noise; it does not cure a small-sample or sparse-event problem.
For clustered observations, resample independent clusters rather than individuals, while preserving all records within each selected cluster. If subjects were sampled in distinct design strata, the resampling scheme should reflect that design. When the independent sampling unit is not the subject, an ordinary subject-level bootstrap can understate or otherwise misrepresent uncertainty.
How Bootstrap Inference Differs from Other Approaches
The main alternative in this setting is the variance-based pointwise interval using the Aalen variance estimate. It is often efficient and convenient when its approximation is adequate, while the bootstrap offers an empirical view of the estimate’s sampling distribution and can reflect asymmetry. Neither approach is universally superior: sparse causes and boundary estimates can make both approximations problematic. A percentile interval is simple, but it can have poor coverage in small samples or near boundaries. Other bootstrap intervals, such as basic or normal-based intervals, transform the same resampling information in different ways; they do not remove the need to assess the data and assumptions.
A bootstrap interval calculated separately at each time is pointwise. It does not guarantee that the whole cumulative incidence curve lies within a band with a specified probability. Nor does bootstrap resampling change the estimand: the result remains a cause-specific cumulative incidence estimate in the population represented by the study, under the estimator’s assumptions.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In a subject-level bootstrap, what is sampled, and what information must stay together?
- A replicate estimate distribution has standard deviation 0.04 at time \(\tau\). What does this quantity estimate, and what does it not describe?
- Why can a rare cause lead to many bootstrap estimates equal to zero?
- Does a 95% pointwise bootstrap interval at each of ten time points automatically give 95% coverage for the entire cumulative incidence curve?
Solutions. (1) Sample \(n\) complete subject records with replacement; each subject’s observed time, event or censoring status, cause, and entry time remain together. (2) It estimates the sampling standard error of the cumulative incidence estimate at the specified time; it is not a standard error for the whole curve or a simultaneous band. (3) Some resamples omit all subjects who experienced that cause, so the recomputed cause-specific incidence can be zero. (4) No. Each interval is pointwise; simultaneous coverage of the whole curve requires a method designed for a curve-wide band.