Why G-Formula Estimates Need Uncertainty Estimates
A g-formula estimate combines fitted conditional models with predictions standardized over a target population. In longitudinal settings, those models describe a sequence of outcomes or covariates, and the final estimate depends on both their fitted parameters and the observed distribution of histories being standardized over. A single point estimate therefore does not show how much the result might vary across comparable samples.
The nonparametric bootstrap estimates sampling variability by repeatedly drawing new samples from the observed individuals, with replacement, and repeating the entire analysis. For a g-formula analysis, that means more than recalculating predictions from the original fitted models: each replicate must refit the sequential models and redo standardization. The replicate estimates then approximate the sampling distribution of the estimator, subject to the assumptions and limitations described below.
The causal interpretation of \(\psi(g)\) relies on the identification conditions for the longitudinal g-formula, including consistency, sequential exchangeability, and positivity, as well as an appropriate definition of the intervention and target population. The bootstrap does not establish these conditions. It quantifies sampling uncertainty for the estimator being applied; it does not remove bias from an unidentified causal effect, misspecified models, or a poorly supported regime.
The Subject-Level Bootstrap Algorithm
Suppose the observed data consist of \(n\) independent individuals, each with a longitudinal record of baseline covariates, time-varying covariates and treatments, and an outcome. In each bootstrap replicate, sample \(n\) individual records with replacement. A sampled record is the entire longitudinal record for one person, not a separate draw of each time point. Repeated selections of a person appear as repeated copies in that replicate.
For each replicate, refit every conditional model used by the chosen g-computation procedure. Then apply the regime or regimes to those refitted models and repeat the same standardization used for the original point estimate. For a Monte Carlo implementation, repeat its simulation or prediction procedure as well, using enough draws that simulation noise is small relative to the sampling uncertainty of interest. If exact averaging is available, repeat that instead. Use the same definition of the target population and the same estimator in the original analysis and in every replicate.
Sample \(n\) independent individuals with replacement from the observed \(n\) individuals, retaining each selected individual’s complete longitudinal record.
Fit each conditional model anew using the bootstrap sample. Do not treat estimates from the original sample as fixed.
Apply each candidate regime to the refitted models and standardize over the target population according to the original procedure.
Save each regime-specific mean and calculate their contrast within the same replicate.
Use the spread and, if desired, quantiles of the bootstrap estimates to quantify uncertainty.
If the target is the empirical distribution of the observed individuals, standardizing within a replicate uses the resampled individuals and their multiplicities. This allows the empirical covariate-history distribution to vary across replicates, as it would across samples. If instead the target distribution is a fixed external population, the analysis must specify that target and repeat the same fixed-target standardization in each replicate; the bootstrap then reflects uncertainty from fitting the models, not sampling variability in that external target distribution.
Bootstrap Standard Errors and Intervals
Let \(\widehat{\psi}^{*(b)}(g)\) be the estimate for regime \(g\) in bootstrap replicate \(b\), for \(b=1,\ldots,B\). The bootstrap standard error is the sample standard deviation of those replicate estimates. For a contrast, first calculate \(\widehat{\Delta}^{*(b)}=\widehat{\psi}^{*(b)}(g_1)-\widehat{\psi}^{*(b)}(g_0)\) within each replicate; the standard deviation of these replicate contrasts estimates the standard error of \(\widehat{\Delta}\).
The same sample-standard-deviation formula applies to the replicate contrasts. A normal-approximation interval for an estimate \(\widehat{\theta}\) is \(\widehat{\theta}\pm 1.96\widehat{SE}_{\mathrm{boot}}\), for an approximate 95% interval when the bootstrap distribution is reasonably well represented by a normal approximation. A percentile interval uses the appropriate empirical quantiles of the bootstrap estimates—for example, the 2.5th and 97.5th percentiles for a 95% interval. These are alternative interval constructions; neither should be assumed reliable without considering the shape and stability of the replicate distribution.
Worked Examples
Worked Example: Summarizing Bootstrap Contrasts
Suppose a hypothetical analysis estimates a one-year mean outcome in days under two longitudinal regimes. After refitting all models and repeating standardization in each of five illustrative bootstrap replicates, the estimates are as follows. The small number of replicates is for arithmetic practice only; a real analysis would generally use many more.
| Replicate | Mean under \(g_1\) (days) | Mean under \(g_0\) (days) | Contrast \(g_1-g_0\) (days) |
|---|---|---|---|
| 1 | 18.0 | 14.0 | 4.0 |
| 2 | 19.0 | 14.5 | 4.5 |
| 3 | 17.0 | 14.0 | 3.0 |
| 4 | 20.0 | 15.0 | 5.0 |
| 5 | 18.5 | 14.0 | 4.5 |
The replicate contrasts are \(4.0, 4.5, 3.0, 5.0,\) and \(4.5\) days. Their mean is \(4.2\) days. The squared deviations from that mean are \(0.04, 0.09, 1.44, 0.64,\) and \(0.09\), with sum \(2.30\). Therefore,
This value estimates the sampling standard error of the contrast for this hypothetical analysis. Five replicates are far too few for a dependable estimate of uncertainty or interval endpoints, but the calculation shows why the contrast must be formed within each replicate.
Worked Example: Repeating Standardization After Refitting
Consider a hypothetical two-visit analysis with binary \(L_1\), a covariate measured between treatments \(A_0\) and \(A_1\), and a continuous outcome \(Y\). A particular bootstrap resample is drawn, and the sequential models are refitted on that resample. Suppose the fitted covariate model gives \(P(L_1=1\mid A_0=0)=0.25\) and \(P(L_1=1\mid A_0=1)=0.55\). Suppose the refitted outcome model is
For this example, the target population is represented by the resampled individuals; the displayed covariate probabilities and outcome equation summarize the fitted quantities needed for the standardization. Under the sustained regime \(g_0=(A_0=0,A_1=0)\), the expected value is averaged over the fitted distribution of \(L_1\) under \(A_0=0\). Since \(L_1\) is binary, its expected value is its probability of being 1:
Under the sustained regime \(g_1=(A_0=1,A_1=1)\), use the fitted distribution of \(L_1\) under \(A_0=1\):
The contrast for this replicate is \(17.2-11.0=6.2\) outcome units. In the full analysis, the values \(0.25\), \(0.55\), and the outcome-model coefficients would be estimated anew in every bootstrap resample. Reusing the original-sample fits would omit model-fitting variation and would not implement the specified bootstrap.
Worked Example: Why the Contrast Uses Paired Replicates
Suppose a hypothetical bootstrap produces four pairs of regime means, measured in symptom-free days. The pairs are \((\widehat{\psi}^{*}(g_1),\widehat{\psi}^{*}(g_0))=(12,10),(15,12),(11,9),(14,11)\). The corresponding contrasts are \(2,3,2,\) and \(3\) days. Their mean is \(2.5\) days, and the sample standard deviation is
The two regime means tend to move together across these resamples. If their replicate standard errors are calculated separately and then treated as independent, that shared variation is ignored. In general,
The paired bootstrap contrasts retain this covariance automatically. The interval based on their distribution concerns the difference in mean symptom-free days under the two regimes, not either regime mean alone.
What the Bootstrap Captures—and What It Does Not
When individuals are the independent sampling units, resampling complete individuals preserves the within-person longitudinal structure while representing variation across sampled individuals. Refitting and re-standardizing can therefore reflect uncertainty from the estimated sequential models and from the empirical target distribution, if the latter is resampled. The bootstrap is an uncertainty procedure for the complete estimator, not merely a way to put error bars around predictions from fixed fitted models.
If observations are dependent within a higher-level unit, such as patients within clinics, the independent sampling unit may instead be the clinic. The resampling scheme must reflect the sampling design and dependence structure. Resampling individual visit records separately would generally break the longitudinal structure and would not reproduce the intended sampling process.
A bootstrap interval is not automatically valid in every setting. Its usual justification requires that the estimator behave regularly under repeated sampling and that the resampling distribution approximate its sampling distribution. Sparse data, unstable model fitting, estimates near parameter boundaries, or very few observed histories compatible with a regime can produce erratic replicate distributions. The nonparametric bootstrap cannot create information absent from the observed data. A high proportion of failed fits or extreme replicate estimates is a warning that the uncertainty summary may be unreliable; it is not a reason to silently discard those replicates.
In Monte Carlo g-computation, each estimate may also have simulation error because it approximates an expectation using a finite number of simulated draws. That is distinct from sampling uncertainty. Use enough simulation draws, or a sufficiently accurate prediction-based standardization, so that Monte Carlo variation is small relative to the bootstrap variation. If simulation is repeated inside each replicate, it contributes noise to the replicate distribution unless controlled; using sufficiently many draws or a reproducible simulation design helps prevent that noise from obscuring the sampling uncertainty.
Common Mistake
A common mistake is to bootstrap only the final predictions: fit the sequential models once, resample individuals, and average predictions from the unchanged models. The reasoning is that the predictions are the quantities being standardized, so resampling them seems to reproduce uncertainty. The correction is that the predictions depend on estimated model parameters. Holding those parameters fixed omits their sampling variation and does not reproduce the full g-formula estimator. Resample individuals, refit every required model, and redo standardization in each replicate.
Another error is to form a contrast interval by subtracting the lower endpoint for one regime from the upper endpoint for the other, or by treating the regime estimates as independent. Such calculations do not generally give the uncertainty interval for their contrast. Calculate the contrast within each shared bootstrap replicate, then use the resulting contrast distribution for its standard error or interval.
Check Your Understanding
Answer each question by referring to the estimator and sampling process, not just to the final interval.
- Why must a bootstrap replicate refit the sequential conditional models rather than reuse the original fitted models?
- In a longitudinal study with repeated visits per person, what should be resampled when people are the independent sampling units?
- Why should the regime contrast be calculated within each bootstrap replicate?
- What sources of variation may be represented when the target population is the empirical distribution of the resampled individuals?
- How does Monte Carlo error in a g-computation implementation differ from sampling uncertainty estimated by the bootstrap?
- Name one limitation of the nonparametric bootstrap that it cannot resolve.