Uncertainty Quantification for Interventional Effects

Why Quantify Uncertainty in G-Computation?

G-computation combines estimated mediator and outcome models to estimate an interventional mean or a contrast between such means. The estimate is therefore uncertain because the data determine both models, and because the final quantity is a function of their estimates. Reporting only a point estimate hides this sampling uncertainty. A confidence interval can show how much the estimated effect might vary across repeated samples from the target population.

The nonparametric bootstrap estimates this variability by repeatedly resampling observed units, refitting the models, and recomputing the target quantity. The key is to treat the whole analysis as one procedure: a bootstrap replicate must carry the mediator and outcome information from the same resampled units through to the final effect estimate.

Definition: Let $A$ denote exposure, $M$ a mediator, $C$ baseline covariates, and $Y$ an outcome. The interventional mean $\psi(a,a')$ is the mean outcome under exposure level $a$ when the mediator is drawn from its conditional distribution under exposure level $a'$, given baseline covariates, and then averaged over the covariate distribution: $$ \psi(a,a') = \int \int E[Y\mid A=a,M=m,C=c]\, dF_{M\mid A=a',C=c}(m)\,dF_C(c). $$ A g-computation estimate, $\hat{\psi}(a,a')$, replaces the population outcome regression, mediator distribution, and covariate distribution in this expression with estimates from the data.

For binary exposure, examples of contrasts include the interventional direct-effect contrast $\psi(1,0)-\psi(0,0)$ and the interventional indirect-effect contrast $\psi(1,1)-\psi(1,0)$. This tutorial concerns uncertainty in estimates of these quantities, not choosing among mediation estimands.

The identification conditions for the chosen interventional mean remain necessary. In particular, the causal interpretation requires the relevant consistency, exchangeability, and positivity conditions, as well as no interference between units under the analysis. A bootstrap does not establish these conditions. It quantifies sampling variability for the estimator and estimand actually used.

The Paired Nonparametric Bootstrap

Suppose the observed data consist of $n$ independent units, with each unit’s exposure, mediator, outcome, and baseline covariates recorded together. A nonparametric bootstrap sample draws $n$ units with replacement from the observed $n$ complete records. Some units may appear more than once and some may not appear. Within each bootstrap sample, refit both the mediator-distribution model and the outcome model, then repeat the entire g-computation calculation.

Resample complete units.
Draw $n$ observed records with replacement. Keep each record’s $A$, $M$, $Y$, and $C$ together.

Refit both models.
Estimate the conditional mediator distribution and the outcome regression using that bootstrap sample.

Recompute the target.
Apply the g-computation formula using the bootstrap-fitted models and the covariate distribution in that sample.

Repeat and summarize.
Repeat for $B$ bootstrap samples, producing $B$ estimates of each mean and contrast. Use their empirical distribution to quantify uncertainty.

The empirical covariate distribution is usually represented by the resampled units themselves: calculate predicted outcomes over those units and average. If the original analysis instead uses a specified target covariate distribution, the bootstrap calculation should preserve that target and account for any uncertainty in estimating it as appropriate.

A central 95% percentile bootstrap interval uses the 2.5th and 97.5th percentiles of the bootstrap estimates. If $\hat{\theta}^{*(1)},\ldots,\hat{\theta}^{*(B)}$ are bootstrap estimates of a target $\theta$, the interval is

\left[ Q_{0.025}\{\hat{\theta}^{*}\}, Q_{0.975}\{\hat{\theta}^{*}\} \right],

where $Q_q$ is the empirical $q$ quantile. With a sufficiently large $B$, such as 2,000 or more, these quantiles are less affected by Monte Carlo noise than with only a small number of replicates. Other bootstrap interval constructions exist; the percentile interval is direct to compute, but it is not guaranteed to have exact 95% coverage in finite samples.

Worked G-Computation and Bootstrap Replicate

Worked Example: Recomputing Interventional Means in One Resample

Hypothetical data and fitted models. Suppose a study has 100 independent participants, a binary exposure, a continuous mediator measured in hours, and an outcome measured in score points. For illustration, the fitted mediator model in the original sample gives mean mediator values of 4 hours under $A=0$ and 6 hours under $A=1$. Suppose the fitted outcome mean is

\widehat{E}[Y\mid A,M] = 10 + 2A + 3M + 0.5AM.

This regression specification has a linear mean in $M$; consequently, integrating its predicted mean over a mediator distribution requires only that distribution’s mean. The estimates are illustrative model outputs, not results from a real study. Assume for this calculation that the target covariate distribution has no varying covariates.

Under exposure $A=1$ and the mediator distribution under $A=0$, the estimated interventional mean is

\hat{\psi}(1,0) =10+2(1)+3(4)+0.5(1)(4) =10+2+12+2 =26\text{ score points}.

Under $A=0$ and the mediator distribution under $A=0$,

\hat{\psi}(0,0) =10+2(0)+3(4)+0.5(0)(4) =22\text{ score points}.

Thus the estimated interventional direct-effect contrast is $26-22=4$ score points. For comparison, $\hat{\psi}(1,1)=10+2+3(6)+0.5(6)=33$ score points, so the estimated interventional indirect-effect contrast is $33-26=7$ score points.

Now consider one paired bootstrap sample drawn from the 100 complete participant records. Suppose refitting both models in this resample gives mediator means 4.3 and 6.4 hours, and the outcome regression

\widehat{E}^{*}[Y\mid A,M] =9.8+2.1A+3.1M+0.45AM.

The bootstrap estimate of $\psi(1,0)$ is

\hat{\psi}^{*}(1,0) =9.8+2.1+3.1(4.3)+0.45(4.3) =11.9+13.33+1.935 =27.165\text{ score points}.

For $\psi^{*}(0,0)$, the exposure terms vanish:

\hat{\psi}^{*}(0,0) =9.8+3.1(4.3) =23.13\text{ score points}.

The bootstrap direct-effect estimate is therefore $27.165-23.13=4.035$ score points. This is one replicate, not a confidence interval. Repeating the paired resampling and refitting produces the distribution used to construct an interval.

Why Both Models Must Be Refit Together

An interventional mean depends jointly on the outcome response and the mediator distribution. The two fitted components come from the same observed sample, so their sampling errors can be related. For a contrast, the uncertainty in each component and their covariance both matter. Resampling the units once and using that same resample for both model fits reproduces this joint sampling variation.

The point can be seen from the variance of a difference. For bootstrap estimates of two interventional means, the variance of their difference is

\operatorname{Var}(\hat{\psi}^{*}_1-\hat{\psi}^{*}_0) = \operatorname{Var}(\hat{\psi}^{*}_1) +\operatorname{Var}(\hat{\psi}^{*}_0) -2\operatorname{Cov}(\hat{\psi}^{*}_1,\hat{\psi}^{*}_0).

The covariance term is not generally zero. It can be important because both means use the same outcome-model fit, the same sampled covariates, and related mediator-model estimates. A bootstrap that keeps one fitted model fixed omits part of the uncertainty. Resampling mediator and outcome data independently, when they are measurements on the same participants, instead breaks the dependence present in the original sampling process.

Worked Example: The Covariance in a Bootstrap Contrast

Suppose 2,000 paired bootstrap replicates of two interventional means produce standard deviations of 2.0 and 1.5 score points, with bootstrap correlation $0.60$. The covariance is

\widehat{\operatorname{Cov}}(\hat{\psi}^{*}_1,\hat{\psi}^{*}_0) =0.60(2.0)(1.5) =1.8\text{ score-points}^2.

The bootstrap standard deviation of their difference is therefore

\sqrt{2.0^2+1.5^2-2(1.8)} =\sqrt{4+2.25-3.6} =\sqrt{2.65} \approx 1.628\text{ score points}.

If the two estimates were incorrectly treated as independent, the covariance would be set to zero and the calculated standard deviation would be

\sqrt{2.0^2+1.5^2} =\sqrt{6.25} =2.5\text{ score points}.

That calculation is not the paired bootstrap uncertainty for this hypothetical example. The discrepancy shows why the contrast should be computed within each joint replicate, rather than combining marginal uncertainties as though the estimates were unrelated.

Worked Percentile Confidence Interval

Worked Example: Interpreting a Bootstrap Interval

Hypothetical analysis. In a separate hypothetical study, the estimated interventional indirect effect on an outcome measured in score points is 4.0 points. The analyst draws 2,000 paired bootstrap samples, refits both models in every sample, and calculates the indirect-effect contrast each time. Suppose the 50th ordered bootstrap estimate is 4.1 points, the 50th percentile is 4.1 points, and the empirical 2.5th and 97.5th percentiles are 0.8 and 7.1 points. (The percentile, rather than the particular order-statistic convention, determines the reported interval.)

The 95% percentile bootstrap interval is

[Q_{0.025},Q_{0.975}]=[0.8,7.1]\text{ score points}.

In repeated samples under the assumed sampling process, a procedure with valid 95% coverage would produce intervals containing the target interventional effect about 95% of the time. For this dataset, the computed interval ranges from 0.8 to 7.1 score points. It is not a statement that there is a 95% probability the fixed target effect lies in this particular interval. Since zero is outside this interval, it does not include a zero indirect-effect contrast at the nominal 95% interval level; that conclusion does not establish the identification assumptions or rule out bias.

Assumptions, Scope, and Common Pitfalls

The ordinary unit-level nonparametric bootstrap is most directly justified when the observed units are independent and identically distributed from the population relevant to the analysis, and the estimator behaves regularly enough for the bootstrap distribution to approximate its sampling distribution. Each replicate must apply the same fitting choices and g-computation steps as the original analysis. If the data arise from a cluster sample, a repeated-measures design, or a complex sampling scheme, resampling individual rows may be inappropriate; the resampling scheme must reflect the independent sampling units or design.

Bootstrap intervals describe sampling uncertainty under the analysis. They do not repair a poorly specified mediator or outcome model, lack of positivity, unmeasured confounding, or a mismatch between the estimated and scientifically intended target population. A narrow interval can coexist with substantial systematic bias. Model diagnostics and causal identification arguments are separate requirements.

Holding one model fixed. The incorrect reasoning is that uncertainty in the outcome model alone is enough. The correction is to refit both mediator and outcome models in every replicate because both contribute to the estimate.
Resampling mediator and outcome records independently. This discards their within-unit pairing and can change the joint sampling distribution. Resample complete participant records together.
Subtracting separate interval endpoints. The endpoints of intervals for two means do not generally form an interval for their difference. Compute the contrast in each replicate, then take quantiles of that contrast distribution.
Treating a bootstrap interval as a bias correction. The interval quantifies sampling uncertainty for the estimator; it does not automatically correct confounding, misspecification, or other systematic errors.

Key takeaway: To quantify uncertainty in a g-computation interventional effect, resample the independent units as complete records, refit the mediator and outcome models together, and recompute the full estimand or contrast in every replicate. The resulting bootstrap distribution reflects joint sampling uncertainty, not uncertainty about whether the causal assumptions are true.

Check Your Understanding

Answer each question using the definitions and procedures in this tutorial.

Why must the mediator-distribution model and outcome model both be refit in each bootstrap sample?
For an estimated contrast $\hat{\psi}(1,0)-\hat{\psi}(0,0)$, what quantity should be calculated in each bootstrap replicate before taking interval quantiles?
If two bootstrap interventional means have positive covariance, how does that covariance enter the variance of their difference?
What does a 95% percentile bootstrap interval describe, and what does it not mean for the particular interval computed from one dataset?
Name one source of uncertainty or bias that a nonparametric bootstrap does not resolve.