Generalized Random Forests for Conditional Effects

From Forest Neighborhoods to Conditional Effects

A treatment effect can vary with observed characteristics: a program may have a larger average effect for one covariate profile than another. A conditional average treatment effect (CATE) describes this variation, but estimating it requires more than comparing treated and untreated outcomes in a small neighborhood. Treatment assignment may depend on covariates, and a neighborhood that is useful for one target profile may be a poor match for another.

Generalized random forests address the local-estimation problem by combining two ideas. A forest assigns greater weight to observations that repeatedly share tree leaves with the target covariate value. The resulting adaptive weights are then used to solve a local estimating equation. For treatment effects, the equation adjusts for treatment probability and expected outcome given covariates; it is not simply a weighted difference in observed means.

Definition: Let \(X_i\) be observed covariates, \(W_i\in\{0,1\}\) a binary treatment, and \(Y_i\) an observed outcome. Write \(Y_i(1)\) and \(Y_i(0)\) for the potential outcomes under treatment and control, so consistency means \(Y_i=W_iY_i(1)+(1-W_i)Y_i(0)\). The conditional average treatment effect is \(\tau(x)=E[Y(1)-Y(0)\mid X=x]\). It is an average over units with covariates \(x\), not an effect identified for each individual.

A causal interpretation of \(\tau(x)\) requires assumptions about how treatment was assigned. The standard conditions used here are: consistency; conditional unconfoundedness, \((Y(1),Y(0))\perp W\mid X\); and overlap, \(0<P(W=1\mid X=x)<1\) for covariate values of interest. These conditions identify the conditional effect from observed data. They do not say that treatment was randomized unconditionally, and overlap can be weak even when it holds mathematically.

Adaptive Weights and the Local Moment

For a target covariate value \(x\), a tree places \(x\) and some estimation observations in the same terminal leaf. Within that tree, observations in the same leaf receive equal weight and observations outside it receive zero. The forest averages those tree-specific weights. Thus, units that often share leaves with \(x\) receive larger weights. The weights are adaptive: tree partitions determine which observations count as local, rather than a fixed bandwidth or distance rule.

Definition: Let \(\alpha_i(x)\) denote the forest weight assigned to observation \(i\) when estimating at \(x\). For the usual normalized leaf weights, \(\alpha_i(x)\geq 0\) and \(\sum_i\alpha_i(x)=1\). These are neighborhood weights, not treatment probabilities and not probabilities that a particular unit has a causal effect. In an honest forest, tree construction and effect estimation use separate samples or sample roles, so the outcomes used to estimate leaf effects did not determine the corresponding leaf partitions.

Define the propensity score \(e(x)=E[W\mid X=x]\) and the observed-outcome regression \(m(x)=E[Y\mid X=x]\). For a candidate effect value \(\theta\), use the score \(\psi(O;\theta,\eta)=(W-e(X))\{Y-m(X)-\theta(W-e(X))\}\), where \(O=(Y,W,X)\) and \(\eta=(e,m)\) denotes the nuisance functions. The conditional moment equation is \(E[\psi(O;\tau(x),\eta)\mid X=x]=0\).

E\!\left[(W-e(X))\{Y-m(X)-\tau(x)(W-e(X))\}\mid X=x\right]=0.

The key identity behind this equation follows from the assumptions. Given \(X=x\), \(E[W-e(x)\mid X=x]=0\), and \(E[(W-e(x))^2\mid X=x]=e(x)\{1-e(x)\}\). Also, unconfoundedness and consistency imply \(E[(W-e(x))Y\mid X=x]=e(x)\{1-e(x)\}\tau(x)\). Since \(m(x)\) is constant conditional on \(X=x\), its contribution multiplied by \(W-e(x)\) has expectation zero. Consequently, the conditional moment equals \(e(x)\{1-e(x)\}\{\tau(x)-\theta\}\); under overlap, its unique zero is \(\theta=\tau(x)\).

E[\psi(O;\theta,\eta)\mid X=x] =e(x)\{1-e(x)\}\{\tau(x)-\theta\}.

In a sample, the forest replaces conditioning exactly on \(X=x\) with the adaptive neighborhood weights. With nuisance estimates \(\hat e_i\) and \(\hat m_i\), the estimated local moment is \(\sum_i\alpha_i(x)(W_i-\hat e_i)\{Y_i-\hat m_i-\theta(W_i-\hat e_i)\}\). Setting it to zero and solving gives a ratio, provided its denominator is positive.

\hat\tau(x)= \frac{\sum_i\alpha_i(x)(W_i-\hat e_i)(Y_i-\hat m_i)} {\sum_i\alpha_i(x)(W_i-\hat e_i)^2}.

This is a local solution to an estimating equation, not a claim that every observation in the forest neighborhood has the same effect. Its interpretation as a CATE relies on the weights concentrating on covariates near \(x\), suitable regularity of the conditional moment, adequate overlap, and sufficiently accurate nuisance estimates. Honest estimation and cross-fitting help separate fitting steps and reduce overfitting concerns; they do not by themselves guarantee good localization or overlap.

Worked Calculations

Worked Example: Constructing Weights From Two Trees

Suppose a hypothetical honest forest has two trees. At target value \(x\), the estimation observations in the same leaf are \(\{1,2,3\}\) for the first tree and \(\{1,3,4\}\) for the second. Each tree gives equal weight to observations in its target leaf. The forest averages the two sets of weights.

In the first tree, each of observations 1, 2, and 3 receives \(1/3\); in the second, each of 1, 3, and 4 receives \(1/3\). Averaging tree-specific weights gives \(\alpha_1(x)=1/3\), \(\alpha_2(x)=1/6\), \(\alpha_3(x)=1/3\), and \(\alpha_4(x)=1/6\). These sum to \(1\). Suppose, for illustration, that the nuisance-adjusted values \(W_i-\hat e_i\) are \(0.5,-0.5,0.5,-0.5\), and \(Y_i-\hat m_i\) are \(4,-1,2,-3\).

The numerator contribution before multiplying by the forest weight is \((W_i-\hat e_i)(Y_i-\hat m_i)\), and the denominator contribution is \((W_i-\hat e_i)^2\). Therefore,

\begin{aligned} \text{numerator} &=\tfrac13(0.5)(4)+\tfrac16(-0.5)(-1) +\tfrac13(0.5)(2)+\tfrac16(-0.5)(-3)\\ &=0.6667+0.0833+0.3333+0.2500=1.3333,\\ \text{denominator} &=\tfrac13(0.5)^2+\tfrac16(-0.5)^2 +\tfrac13(0.5)^2+\tfrac16(-0.5)^2=0.2500. \end{aligned}

Thus \(\hat\tau(x)=1.3333/0.2500=5.333\), in the outcome units. This calculation demonstrates the local score solution; whether the number estimates a causal CATE depends on the identification, overlap, nuisance-estimation, and localization conditions described above.

Worked Example: Allowing Propensity Scores to Vary Across Observations

Consider a second hypothetical local neighborhood with four observations. Their forest weights are \(0.4,0.3,0.2,0.1\). For each observation, the table gives the treatment indicator \(W_i\), estimated propensity \(\hat e_i\), and outcome residual \(Y_i-\hat m_i\). All residuals are in outcome units.

Observation	\(\alpha_i(x)\)	\(W_i\)	\(\hat e_i\)	\(Y_i-\hat m_i\)
1	0.4	1	0.50	3
2	0.3	0	0.25	-2
3	0.2	1	0.75	4
4	0.1	0	0.50	-1

For the numerator, the weighted score contributions are \(0.4(0.50)(3)=0.6000\), \(0.3(-0.25)(-2)=0.1500\), \(0.2(0.25)(4)=0.2000\), and \(0.1(-0.50)(-1)=0.0500\). They sum to \(1.000\). For the denominator, the weighted squared residual-treatment contributions are \(0.4(0.50^2)=0.1000\), \(0.3(0.25^2)=0.01875\), \(0.2(0.25^2)=0.01250\), and \(0.1(0.50^2)=0.02500\), summing to \(0.15625\).

\hat\tau(x)=\frac{1.000}{0.15625}=6.400.

The fitted local effect is \(6.400\) outcome units. The observations with \(\hat e_i=0.25\) or \(0.75\) contribute less to the denominator than observations with \(\hat e_i=0.50\), because their treatment residuals have smaller squared magnitude. If propensities are near zero or one throughout a neighborhood, the denominator can be small and the local estimate unstable; forest weights do not repair a lack of overlap.

Worked Example: Verifying the Population Moment

Suppose, hypothetically, that at a covariate value \(x\), \(P(W=1\mid X=x)=0.4\), \(E[Y(1)\mid X=x]=14\), and \(E[Y(0)\mid X=x]=10\). Assume consistency and conditional unconfoundedness. The CATE is \(\tau(x)=14-10=4\) outcome units, and the observed conditional mean is \(m(x)=0.4(14)+0.6(10)=11.6\).

To check the score identity, calculate the conditional expectation of \((W-e(x))(Y-m(x))\). For treated units, \(W-e(x)=0.6\), and the conditional mean outcome residual is \(14-11.6=2.4\); their probability-weighted contribution is \(0.4(0.6)(2.4)=0.576\). For controls, \(W-e(x)=-0.4\), and the mean residual is \(10-11.6=-1.6\); their contribution is \(0.6(-0.4)(-1.6)=0.384\). The total is \(0.960\).

The expected squared treatment residual is \(0.4(0.6^2)+0.6((-0.4)^2)=0.240\). Thus the moment root is \(0.960/0.240=4.000\), matching the CATE. This identity explains why the local equation identifies a conditional effect under the stated assumptions; a finite forest estimate approximates it using estimated nuisances and a weighted neighborhood.

Why the Generalized Forest Formulation Matters

The general principle is broader than one formula: define a parameter through a conditional moment, then use forest weights to solve a localized version of that moment. For treatment-effect forests, the estimating equation combines adaptive neighborhoods with residualization for treatment and outcome. Residualization is important because covariates can predict both treatment and outcome; an unadjusted local treated-minus-control difference may then reflect covariate imbalance as well as treatment effect.

The forest weights and score play distinct roles. The weights determine which observations contribute most to the estimate at \(x\). The score determines how those observations contribute after accounting for treatment assignment and outcome prediction. A useful implementation therefore needs credible nuisance predictions and a neighborhood with treatment variation. Inspecting only the number of observations in nearby leaves is not enough: the weighted denominator also indicates whether the neighborhood contains useful treatment variation.

Common Mistake

Pitfall: “The forest weight is the probability that observation \(i\) received treatment, so I can read the effect from the treated observations with the largest weights.” This confuses two different quantities. \(\alpha_i(x)\) is an adaptive neighborhood weight; \(e(X_i)\) is a treatment propensity. Neither is an individual treatment effect. The local CATE estimate is obtained by solving the weighted, nuisance-adjusted moment equation, and its causal interpretation depends on the identification assumptions.

A second practical error is to report a local estimate without considering its denominator or the quality of local support. A numerical root always exists when the denominator is positive, but a very small denominator can make the result highly sensitive to a few observations or to nuisance-estimation error. The calculation is not, by itself, evidence that the target covariate profile is well represented in both treatment groups.

Check Your Understanding

Answer each question using the definitions and local moment equation from this tutorial.

What determines whether an observation receives a large forest weight at a target covariate value?
State the three causal identification conditions used to interpret the conditional effect as causal.
Why does the conditional moment have its zero at \(\tau(x)\) when overlap holds?
What does a small denominator in the local ratio suggest about treatment variation in the weighted neighborhood?
Explain why an adaptive forest weight is not a propensity score or an individual treatment effect.