Tutorials › Causal Inference › Inference with Causal Forests

Causal Learning and Policy Decisions · Tutorial 958 of 972

Inference with Causal Forests

Understand forest-based standard errors, the conditions behind their interpretation, and how to calculate a pointwise confidence interval for a conditional treatment effect.

Advanced 10 min read

What You'll Learn

  • Distinguish a CATE estimate from its estimated standard error and confidence interval.
  • Explain the role of honest estimation, overlap, and regularity conditions in forest inference.
  • See how local forest weights and treatment-effect scores motivate a variance estimate.
  • Calculate pointwise confidence intervals on risk-difference and continuous-outcome scales.
  • Recognize why tree-to-tree variation and a confidence interval crossing zero require careful interpretation.

Why Uncertainty Matters for a Causal Forest

A causal forest can estimate how a treatment effect varies with covariates, but an estimated conditional average treatment effect (CATE) is not exact. If a forest estimates a treatment effect of 6 percentage points for patients with a particular covariate profile, the estimate alone does not say how precisely the data determine that effect. Inference adds a standard error and, commonly, a confidence interval.

The interval is about uncertainty in the estimated average effect for a covariate profile, not uncertainty about an individual patient’s personal treatment response. Its causal interpretation also depends on the assumptions that identify the CATE from the observed data. A narrow interval cannot compensate for a failure of those assumptions.

Definition: For binary treatment \(W\), potential outcomes \(Y(1)\) and \(Y(0)\), and covariates \(X\), the conditional average treatment effect is \(\tau(x)=E[Y(1)-Y(0)\mid X=x]\). A pointwise confidence interval for \(\tau(x)\) quantifies sampling uncertainty about this effect at a specified covariate value \(x\), under the model and identification conditions used.

Conditions Behind a Causal Interpretation

Statistical uncertainty and causal identification are distinct. A forest’s standard error describes uncertainty under its statistical construction; interpreting the target as a causal effect requires assumptions about treatment assignment and outcomes. For the usual observational-data interpretation, the main identification conditions are:

  • Consistency: the observed outcome equals the potential outcome under the treatment actually received.
  • No interference: one unit’s treatment does not change another unit’s outcome, as defined for the study.
  • Conditional exchangeability: \(Y(1),Y(0)\) are independent of \(W\) given the measured covariates \(X\).
  • Overlap: the treatment probability \(e(X)=P(W=1\mid X)\) is strictly between 0 and 1 in the covariate region where the CATE is estimated.

In a randomized experiment, random assignment can support exchangeability, while observational analyses must justify it using the measured covariates and study design. These assumptions identify the CATE; they do not by themselves guarantee that a particular forest estimator has a valid standard error.

Forest-based inference additionally relies on statistical conditions. Honest construction separates data used to choose splits from data used to estimate effects within leaves, limiting adaptive reuse of outcomes. The forest must have enough relevant observations near \(x\); the target effect and nuisance functions must be sufficiently regular for local estimation; outcomes or influence scores must have finite variance; and the variance estimator must reflect the forest’s sampling behavior. Implementations establish these properties under particular regularity and subsampling conditions. The exact technical conditions depend on the estimator, so a standard error should be taken from a procedure designed for that forest rather than inferred from its predictions alone.

From Local Scores to a Standard Error

One way to understand forest inference is to represent an estimate at \(x\) as a weighted average of treatment-effect scores. Forest weights \(\alpha_i(x)\) give greater influence to observations considered locally relevant, and are typically nonnegative and sum to one. A common score for a binary treatment is the doubly robust score

$$ \Gamma_i = \hat m_1(X_i)-\hat m_0(X_i) +\frac{W_i}{\hat e(X_i)}\{Y_i-\hat m_1(X_i)\} -\frac{1-W_i}{1-\hat e(X_i)}\{Y_i-\hat m_0(X_i)\}, $$

where \(\hat m_w(X_i)\) estimates the conditional mean outcome under treatment \(w\), and \(\hat e(X_i)\) estimates the treatment probability. With appropriate nuisance estimation, this score has conditional expectation equal to the CATE under the identification conditions. Cross-fitting, in which nuisance predictions for a unit are obtained without fitting the nuisance models to that unit, can help control overfitting bias.

This score representation is useful for understanding uncertainty; not every causal forest implementation literally averages these scores. Some estimate a local treatment-effect moment directly. In either case, inference needs to account for the variability of the estimator, including relevant effects of sample reuse, forest construction, and nuisance estimation.

Variance justification: If, conditional on the covariates and fixed forest weights, the scores are independent with conditional variances \(\sigma_i^2\), then the variance of their weighted average is \(\operatorname{Var}\{\sum_i\alpha_i(x)\Gamma_i\mid X,\alpha\}=\sum_i\alpha_i(x)^2\sigma_i^2\). This follows by expanding the variance of a sum: the covariance terms are zero under conditional independence, and multiplying a random variable by \(\alpha_i(x)\) multiplies its variance by \(\alpha_i(x)^2\). In an actual forest, estimated weights, nuisance functions, and adaptive sampling can add uncertainty; a suitable forest variance estimator must address those features.

Under conditions that yield asymptotic normality and make bias small relative to the standard error, a pointwise interval has the usual form:

$$ \hat\tau(x)\ \pm\ z_{1-\alpha/2}\,\widehat{SE}\{\hat\tau(x)\}, $$

Here \(\hat\tau(x)\) is the estimated CATE, \(\widehat{SE}\{\hat\tau(x)\}\) is its estimated standard error, and \(z_{1-\alpha/2}\) is the corresponding standard normal quantile. For a nominal 95% interval, \(z_{0.975}\) is approximately 1.96. The interval is pointwise: it addresses uncertainty at the specified \(x\), not simultaneous coverage over every covariate profile.

Worked Examples

Worked Example: A Risk-Difference Interval

Suppose a hypothetical causal forest estimates the CATE at a specified patient profile as \(\hat\tau(x)=0.064\), with standard error \(0.027\). The outcome is whether a complication occurs, so the effect is expressed as a risk difference. Assume the forest’s inference procedure is appropriate for the estimator and the conditions for its pointwise normal approximation are reasonable.

For a 95% interval, calculate the margin of error:

$$ 1.96(0.027)=0.05292. $$

The interval is therefore

$$ 0.064\pm0.05292 = [0.01108,\ 0.11692]. $$

Rounded to useful precision, the estimated CATE is a 6.4-percentage-point increase in complication risk, with a 95% pointwise interval from about 1.1 to 11.7 percentage points. Under the stated assumptions and approximation, the interval excludes zero, indicating evidence of a positive average effect for this profile. It does not imply that every patient with this profile experiences an increase of that size.

Worked Example: A Local-Weight Variance Calculation

For illustration, suppose a forest estimate at a target \(x\) uses four independent treatment-effect scores with weights \(0.40, 0.30, 0.20,\) and \(0.10\). The weights sum to one. Suppose the scores are \(0.10, 0.20, 0.00,\) and \(0.30\), and their estimated conditional variances are \(0.04, 0.09, 0.04,\) and \(0.16\), respectively. Treat the weights as fixed for this calculation.

The weighted effect estimate is

$$ \hat\tau(x) =0.40(0.10)+0.30(0.20)+0.20(0.00)+0.10(0.30) =0.04+0.06+0+0.03 =0.13. $$

Using the weighted-score variance formula, the estimated variance is

$$ \widehat{\operatorname{Var}}\{\hat\tau(x)\} =(0.40)^2(0.04)+(0.30)^2(0.09)+(0.20)^2(0.04)+(0.10)^2(0.16) =0.0064+0.0081+0.0016+0.0016 =0.0177. $$

Thus the standard error is \(\sqrt{0.0177}\approx0.1330\). A normal-approximation 95% interval is

$$ 0.13\pm1.96(0.1330) =0.13\pm0.2607 =[-0.1307,\ 0.3907]. $$

This wide interval reflects substantial uncertainty in this deliberately small illustration. It includes zero and both negative and positive effects. The calculation explains how local weights affect variance, but is not a substitute for a forest-specific variance estimator when weights and forest structure are estimated from data.

Worked Example: An Interval for a Continuous Outcome

Consider a hypothetical study where the outcome is a symptom score, measured in points. At a specified covariate profile, a causal forest reports an estimated treatment effect of \(2.4\) points and a standard error of \(1.1\) points. Assume the treatment effect is identified for this population and the estimator’s pointwise normal approximation is suitable.

The 95% margin of error is

$$ 1.96(1.1)=2.156\text{ points}. $$

The interval is

$$ 2.4\pm2.156=[0.244,\ 4.556]\text{ points}. $$

The estimated conditional average effect is an increase of 2.4 symptom-score points, with a 95% pointwise interval of approximately 0.24 to 4.56 points. Since the interval excludes zero, it is inconsistent with a zero effect at the conventional two-sided 5% level under the approximation and assumptions. This is an average effect at the specified profile, not a prediction of the change for an individual.

What Forest Standard Errors Do—and Do Not—Measure

A common practical error is to calculate the standard deviation of predictions across trees and call it the standard error of the CATE estimate. Trees in a forest are not independent replicates of the study: they are built from overlapping data and are combined to form one estimator. Their spread alone does not generally estimate the sampling uncertainty of that estimator. Use the variance procedure appropriate to the causal forest implementation, such as a method designed to estimate the forest prediction’s sampling variance.

A second error is to read a confidence interval as if it were a probability statement about a fixed parameter. In repeated samples, a procedure with 95% coverage produces intervals containing the fixed target effect about 95% of the time, under its assumptions. After observing one interval, it is not correct to say there is a 95% probability that the fixed CATE lies inside it.

Nor does an interval that includes zero prove that the effect is zero. It indicates that the data and procedure have not ruled out zero at that confidence level; the interval may also include effects that matter in practice. Conversely, excluding zero does not establish that the result is important for clinical decisions. Finally, a pointwise interval does not automatically cover a collection of profiles simultaneously, and neither interval type repairs confounding, poor overlap, or model bias.

1
Specify the target.
Name the covariate profile, treatment contrast, outcome scale, and target population for the CATE.
2
Check the inferential setup.
Confirm that the causal identification assumptions are defensible and that the forest’s inference method matches its estimator and sampling design.
3
Calculate and interpret the interval.
Combine the estimate and standard error using the stated confidence level, retain the outcome units, and describe the result as pointwise.
Key takeaway: A causal forest confidence interval combines an estimated conditional effect with a standard error that accounts for the estimator’s sampling variability. Its causal interpretation requires identification assumptions, and its statistical interpretation requires suitable forest-specific regularity and variance estimation.

Check Your Understanding

Use these questions to check both the calculations and the interpretation.

  1. A forest estimates a risk-difference CATE of \(0.03\) with standard error \(0.02\). Calculate a 95% pointwise interval and interpret its units.
  2. Why is the standard deviation of predictions across trees not generally the standard error of the forest estimate?
  3. In the local-weight variance formula, why does each weight appear squared?
  4. What does a 95% pointwise interval mean in repeated sampling, and what does it not mean after one interval has been observed?
  5. Name two causal identification conditions needed to interpret a forest estimate as a CATE in observational data.
Need help applying this method to your own data?

Our team supports biostatistics, statistical programming and pharmacometric analyses. Send us a request and we will reply by email.

Request an analysis
Clinical Biostats