Tutorials › Causal Inference › Splitting Criteria for Heterogeneous Treatment Effects

Causal Learning and Policy Decisions · Tutorial 956 of 972

Splitting Criteria for Heterogeneous Treatment Effects

Derive and apply a causal-tree split criterion that rewards variation in treatment effects across child nodes, not variation in outcome levels alone.

Advanced 10 min read

What You'll Learn

  • Define node-specific average treatment effects for a proposed tree split
  • Derive the weighted gain from separating two child-node effects
  • Compare candidate splits using treatment-effect contrasts and child sizes
  • Explain why prognostic differences in outcome means do not establish treatment-effect heterogeneity
  • Recognize how sampling variability and treatment-arm sizes affect interpretation of a split score

Why Causal Trees Need a Different Split Target

A conventional regression tree chooses splits that improve prediction of the outcome. For a causal tree, the goal is different: find covariate-defined groups whose treatment effects differ. A variable can strongly predict outcomes while being useless for separating treatment effects. Conversely, a variable with modest predictive value for outcome levels can be valuable if it marks a meaningful change in treatment benefit.

This distinction matters for decisions based on who benefits more from treatment. A split is not evidence of effect heterogeneity merely because the treated and control groups have different outcome averages on either side. The relevant comparison is between the treatment effect in one child node and the treatment effect in the other.

Definition: Let \(Y(1)\) and \(Y(0)\) denote an individual's potential outcomes under treatment and control, and let \(X\) be a vector of pretreatment covariates. For a covariate-defined node \(j\), its conditional average treatment effect is $$ \tau_j = E[Y(1)-Y(0)\mid X\in j]. $$ A candidate split divides a parent node into two disjoint child nodes, \(L\) and \(R\). The split targets heterogeneity when \(\tau_L\) differs from \(\tau_R\).

The node effects are causal estimands, not simply differences between outcome averages in covariate groups. In a randomized trial, the treatment-control mean difference within a node identifies its average treatment effect, provided treatment assignment is randomized within that node, treatment and control outcomes are well-defined and consistently observed, and one person's assignment does not affect another person's outcome. In observational data, identification instead requires suitable assumptions such as conditional exchangeability and positivity; raw within-node outcome differences are not automatically causal.

The split score developed here uses estimated node effects. Its interpretation is clearest when those estimates are approximately unbiased for the node effects and when the tree's constant-effect approximation within each node is useful. A score is a criterion for ranking candidate partitions, not by itself a test of a causal hypothesis.

Deriving the Heterogeneity Gain

Suppose a parent node contains \(n=n_L+n_R\) observations, where \(n_L\) and \(n_R\) are the child-node sizes. Let \(\hat{\tau}_L\) and \(\hat{\tau}_R\) estimate the two child effects. The parent estimate under the same weighting is $$ \hat{\tau}_P = \frac{n_L\hat{\tau}_L+n_R\hat{\tau}_R}{n}. $$ A simple split criterion asks how much better the two child-specific constants fit than one parent-wide constant. In terms of squared deviations, the reduction is $$ G = n_L(\hat{\tau}_L-\hat{\tau}_P)^2+n_R(\hat{\tau}_R-\hat{\tau}_P)^2. $$

Substituting the weighted parent estimate gives $$ \hat{\tau}_L-\hat{\tau}_P = \frac{n_R}{n}(\hat{\tau}_L-\hat{\tau}_R), \qquad \hat{\tau}_R-\hat{\tau}_P = -\frac{n_L}{n}(\hat{\tau}_L-\hat{\tau}_R). $$ Therefore, $$ G = \left[n_L\left(\frac{n_R}{n}\right)^2+n_R\left(\frac{n_L}{n}\right)^2\right](\hat{\tau}_L-\hat{\tau}_R)^2 = \frac{n_Ln_R}{n}(\hat{\tau}_L-\hat{\tau}_R)^2. $$

This identity shows the two ingredients of the score: the squared difference in child treatment effects and a weight reflecting the sizes of both children. A large effect difference can yield a small gain if one child is tiny; a split with equally sized children receives more weight for the same effect contrast. If effects are measured in, for example, days of symptom reduction, \(G\) has units of observations times days squared. It is a fit-improvement score, not an effect estimate in days.

Key criterion: For a candidate binary split, the weighted heterogeneity gain is $$ G = \frac{n_Ln_R}{n_L+n_R}(\hat{\tau}_L-\hat{\tau}_R)^2. $$ A larger value indicates a larger improvement in the squared-error fit of child-specific effects over a single parent effect, under this criterion. It does not establish that the true effects differ.

Worked Example: Comparing Three Candidate Splits

Worked Example: Comparing Three Candidate Splits

Consider hypothetical randomized data with 200 participants. Each of four covariate profiles contains 50 participants, with 25 assigned to treatment and 25 to control. The table gives the outcome means, in points on a symptom-improvement scale, and the treatment-effect estimate in each profile.

ProfileCovariatesControl meanTreatment meanEstimated effect
1A = 0, B = 010111
2A = 0, B = 120233
3A = 1, B = 030355
4A = 1, B = 140477

Each profile's estimated effect is its treatment mean minus its control mean. For example, profile 3 has an estimated effect of \(35-30=5\) points. Compare three proposed splits, each making two children of 100 participants.

Split on A. The \(A=0\) child combines profiles 1 and 2, so its estimated effect is \((1+3)/2=2\) points. The \(A=1\) child combines profiles 3 and 4, giving \((5+7)/2=6\) points. Thus, $$ G_A=\frac{100(100)}{200}(2-6)^2=50(16)=800. $$

Split on B. The \(B=0\) child combines profiles 1 and 3, giving \((1+5)/2=3\) points. The \(B=1\) child combines profiles 2 and 4, giving \((3+7)/2=5\) points. Thus, $$ G_B=\frac{100(100)}{200}(3-5)^2=50(4)=200. $$

Split on C, defined as whether A and B have equal values. One child combines profiles 1 and 4, with effect \((1+7)/2=4\) points; the other combines profiles 2 and 3, also with effect \((3+5)/2=4\) points. Therefore, $$ G_C=\frac{100(100)}{200}(4-4)^2=0. $$

Under this criterion, split A ranks first, split B ranks second, and split C provides no improvement in the fit to treatment effects. The large outcome levels in profiles 3 and 4 do not drive this ranking directly. The score is based on the differences between child treatment-control contrasts.

Why Outcome-Level Differences Are Not Enough

Write the mean outcome under treatment in node \(j\) as \(\mu_{1j}\) and under control as \(\mu_{0j}\). The node treatment effect is \(\tau_j=\mu_{1j}-\mu_{0j}\). A node can have higher outcomes under both treatment conditions because it differs in prognosis, yet have exactly the same treatment effect as another node. The split criterion must remove that common outcome-level shift by comparing treatment with control within each child first.

Worked Example: A Prognostic Split With No Effect Heterogeneity

Suppose a hypothetical randomized sample is divided into two equally sized children. In the first child, the control mean is 5 and the treatment mean is 8 points. In the second, the control mean is 15 and the treatment mean is 18 points.

The child effects are $$ \hat{\tau}_L=8-5=3,\qquad \hat{\tau}_R=18-15=3. $$ With \(n_L=n_R=50\), the gain is $$ G=\frac{50(50)}{100}(3-3)^2=25(0)=0. $$

The second child has outcome means 10 points higher under both conditions, which may make it easier to predict outcome levels. But its estimated treatment effect is not higher. A tree whose target is effect heterogeneity should not reward this split for that prognostic difference alone.

The algebra makes the distinction explicit: $$ \mu_{1L}-\mu_{1R} = (\mu_{0L}-\mu_{0R})+(\tau_L-\tau_R). $$ A difference in treated-outcome means combines a difference in baseline prognosis with a difference in treatment effects. It cannot be interpreted as effect heterogeneity without accounting for the control means as well.

Child Size, Precision, and the Meaning of the Score

The size weight \(\frac{n_Ln_R}{n_L+n_R}\) penalizes splits with a very small child. For a fixed total size, the weight is largest when the children are balanced. Still, child size alone does not ensure precise effect estimates: treatment and control counts within each child also matter. Randomized data with unequal assignment counts, for example, can produce less precise contrasts in one child even when its total size is substantial.

Worked Example: A Large Contrast in an Unequal Split

In a second hypothetical randomized data set, a parent node has 80 participants. A proposed split produces children of sizes 20 and 60. Their estimated treatment effects are 2 and 5 days of symptom reduction, respectively. The estimated parent effect is $$ \hat{\tau}_P=\frac{20(2)+60(5)}{80}=\frac{340}{80}=4.25\text{ days}. $$ The gain is $$ G=\frac{20(60)}{80}(2-5)^2=15(9)=135 $$ participant-days squared. The same effect contrast with children of 40 and 40 would give a size multiplier of \(40(40)/80=20\), rather than 15.

For an illustration of precision, suppose each treatment and control arm has outcome standard deviation 4 days, with 10 participants per arm in the smaller child and 30 per arm in the larger child. Under independent observations, the estimated effect variances are $$ \operatorname{Var}(\hat{\tau}_L)=\frac{16}{10}+\frac{16}{10}=3.2, \qquad \operatorname{Var}(\hat{\tau}_R)=\frac{16}{30}+\frac{16}{30}=1.067. $$ If the child estimates are independent, the standard error of their difference is $$ SE(\hat{\tau}_L-\hat{\tau}_R) =\sqrt{3.2+1.067} =\sqrt{4.267} \approx 2.066\text{ days}. $$ The observed 3-day contrast is not the same thing as a precisely estimated true difference. The raw gain ranks the proposed split according to its estimated fit improvement; it does not account for all uncertainty or convert the contrast into a formal test.

Using the Criterion Carefully

For each candidate split, estimate a treatment effect in each child using a method appropriate to the design, compute the weighted squared contrast, and compare the resulting scores. In randomized data, the child estimates can be differences in treatment and control means. In observational data, they should be adjusted estimates that target the child-specific causal effects under the relevant identification assumptions. A raw difference in outcome means across covariate-defined children is not a substitute.

1
Define the target.
Specify the treatment, outcome, and node-specific average treatment effect the tree is intended to distinguish.
2
Estimate child effects.
For each candidate partition, estimate \(\hat{\tau}_L\) and \(\hat{\tau}_R\) with a method justified by the study design and identification assumptions.
3
Compute and compare gains.
Use \(G=\frac{n_Ln_R}{n_L+n_R}(\hat{\tau}_L-\hat{\tau}_R)^2\), and examine whether the candidate also leaves adequate treatment and control information in each child.

The criterion is most directly motivated by a piecewise-constant approximation: each terminal node is represented by one treatment effect. It rewards a partition when two child-specific effects fit better than one parent effect. It does not, on its own, show that the selected difference will reproduce in new data, quantify uncertainty in the final tree, or establish that assigning treatment by the split improves outcomes.

Common Mistakes

Key takeaway: A causal-tree split should be rewarded for separating child-node treatment effects. Under the weighted squared-error criterion, its gain is the child-size weight multiplied by the squared difference between estimated effects—not a measure of how different the child outcome means are.

Check Your Understanding

Use the definitions and criterion above to answer these questions.

  1. A candidate split has child sizes 30 and 70, with estimated effects of 1 and 4 points. Calculate its weighted heterogeneity gain.
  2. Two children have control means of 12 and 20 and treatment means of 16 and 24. What are the child effects, and what does the split gain indicate about estimated heterogeneity?
  3. Why can a split that substantially improves outcome prediction have zero gain for treatment-effect heterogeneity?
  4. For a fixed total parent size, why does a very small child reduce the size weight in the gain formula?
  5. In observational data, what must be justified before treating a within-child treatment-control contrast as a causal effect?
Back to Tutorials
Need help applying this method to your own data?

Our team supports biostatistics, statistical programming and pharmacometric analyses. Send us a request and we will reply by email.

Request an analysis
Clinical Biostats