Two Different Meanings of “Treatment Effect”
A treatment effect can refer to a contrast between average outcomes in a group or to the effect treatment would have on one particular unit. These are different quantities. A conditional average treatment effect describes an average for units sharing specified covariates; an individual treatment effect compares two potential outcomes for the same unit. The first can be estimated under suitable causal assumptions. The second is not directly observed, because each unit receives either treatment or control, not both.
Keeping this distinction clear matters when interpreting claims such as “treatment works for some people but not others.” Evidence that average effects differ across covariate groups is not the same as observing which individuals benefit. Nor does a positive average effect imply that every unit benefits.
The conditional average treatment effect (CATE) is therefore a feature of the conditional distribution of effects. It does not say that every unit in the covariate group has an effect equal to that conditional mean. The individual effects may vary around it, and some may have the opposite sign. Conversely, an estimated CATE that is the same across covariate groups does not establish that individual effects are constant.
Why an Individual Effect Is Not Observed
Let \(D_i\) indicate treatment receipt, with \(D_i=1\) for treated units and \(D_i=0\) for control units. Under consistency, the observed outcome is
If unit \(i\) is treated, the observed outcome is \(Y_i(1)\), while \(Y_i(0)\) remains counterfactual. If the unit is in control, \(Y_i(0)\) is observed and \(Y_i(1)\) remains counterfactual. Thus the observed data do not contain both terms needed to calculate \(\tau_i\) for that unit. Random assignment helps make treatment and control groups comparable; it does not cause both potential outcomes to be observed for the same unit.
Worked Example: A Positive Average with Different Unit Effects
Consider four hypothetical units with the following potential outcomes. These values are shown to clarify the definitions; in an actual study, both potential outcomes for each unit would not be observed.
| Unit | \(Y_i(1)\) | \(Y_i(0)\) | \(\tau_i=Y_i(1)-Y_i(0)\) |
|---|---|---|---|
| 1 | 8 | 5 | 3 |
| 2 | 7 | 6 | 1 |
| 3 | 4 | 6 | -2 |
| 4 | 9 | 8 | 1 |
The average effect across these four units is
The average is positive, but treatment lowers the outcome for Unit 3 by 2 units. The average does not describe every unit. If the covariate value \(x\) were shared by all four units, their CATE would be \(0.75\) outcome units, while their individual effects would remain \(3,1,-2,\) and \(1\). In an actual experiment, assignment would reveal only one potential outcome per unit, so this table of unit effects could not be constructed directly from the observed outcomes.
When Conditional Mean Effects Can Be Identified
To interpret an observed treatment-control comparison as a CATE, state the causal assumptions that connect observed outcomes to potential outcomes. Let \(X_i\) be measured before treatment. Suppose consistency holds, there is no interference between units, treatment assignment is conditionally independent of the potential outcomes given \(X_i\), and both treatment states have positive probability at the covariate values of interest. In notation, the key conditions are
Conditional independence is often called conditional exchangeability or unconfoundedness. In a randomized experiment, it can follow from the assignment design; in observational data it is an assumption about whether the measured covariates suffice to remove confounding. Positivity requires that comparable units can be observed in both treatment states. These conditions do not reveal each unit’s missing outcome. They identify a conditional mean contrast.
The justification follows by applying the assumptions separately to each treatment state. By consistency, among units with \(D_i=1\), the observed outcome is \(Y_i(1)\). Conditional exchangeability makes the potential outcome mean among those treated equal to the mean for all units with \(X_i=x\), and positivity ensures the treated conditional mean is defined. Therefore \(E[Y_i\mid D_i=1,X_i=x]=E[Y_i(1)\mid X_i=x]\). Similarly, \(E[Y_i\mid D_i=0,X_i=x]=E[Y_i(0)\mid X_i=x]\). Subtracting these equalities gives
The last equality uses linearity of expectation. It is an equality of conditional means, not a recovery of the two potential outcomes for any particular unit. Estimation adds sampling uncertainty and, in practice, may also depend on modeling choices; the identification result itself is about the population quantity under the stated assumptions.
Worked Example: Interpreting Estimated Conditional Effects
Suppose a hypothetical randomized study assigns treatment within two baseline covariate groups. The outcome is a score measured in points. For this illustration, compare the observed group means; assume the assignment was implemented as described, the outcome is defined consistently, and there is no interference.
| Covariate group | Treated: \(n\) | Treated: outcome total | Control: \(n\) | Control: outcome total |
|---|---|---|---|---|
| A | 20 | 140 | 20 | 100 |
| B | 20 | 160 | 20 | 140 |
For Group A, the treated mean is \(140/20=7.0\) points and the control mean is \(100/20=5.0\) points. The estimated conditional mean contrast is \(7.0-5.0=2.0\) points. For Group B, the corresponding means are \(160/20=8.0\) and \(140/20=7.0\) points, giving \(8.0-7.0=1.0\) point. The estimated contrast is 1.0 point larger in Group A than in Group B.
These are sample estimates of conditional average effects, not the effects for every person in either group. Randomization supports a causal interpretation of the group comparisons in expectation, and both treatment states occur in each group, satisfying the design-level positivity requirement here. But sampling variation means the estimates need not equal the population CATEs exactly. Even if the population CATEs really were 2.0 and 1.0 points, those facts would not tell us which individuals benefited or whether some individuals were harmed.
Same Margins, Different Individual Effects
A deeper limit is that observing or identifying the separate distributions of \(Y_i(1)\) and \(Y_i(0)\) does not in general reveal how those potential outcomes are paired within units. The individual effect depends on that pairing. The following finite-population example makes the distinction explicit and also shows why one must be careful about claims concerning the joint observed data.
Worked Example: Identical Potential-Outcome Margins, Different Effects
Consider two hypothetical worlds, each containing four units. In each world, exactly two units have \(Y_i(1)=1\) and exactly two have \(Y_i(0)=1\); the other potential outcomes are zero. In World A, the potential outcomes are aligned: units 1 and 2 have \((Y_i(1),Y_i(0))=(1,1)\), and units 3 and 4 have \((0,0)\). Thus every individual effect is zero.
In World B, the potential outcomes are opposed: units 1 and 2 have \((1,0)\), and units 3 and 4 have \((0,1)\). The individual effects are \(1,1,-1,-1\), so the average effect is also zero. In both worlds, the finite-population mean of \(Y_i(1)\) is \(2/4=0.5\), and the finite-population mean of \(Y_i(0)\) is \(2/4=0.5\). Yet the individual effects differ sharply: they are all zero in World A and vary in World B.
Now assign exactly two of the four units to treatment. Let \(S_T\) be the number of observed outcome-one units among the treated, and \(S_C\) the number among controls. In World A, the two potential-outcome vectors coincide, so the total number of observed ones across treatment and control is always \(S_T+S_C=2\). In World B, treated successes equal the number of treated units among units 1 and 2; control successes equal the number of control units among units 3 and 4. Since exactly two units are treated, those two counts are always equal: \(S_T=S_C\). For example, treating units 1 and 2 gives \((S_T,S_C)=(2,0)\) in World A and \((2,2)\) in World B.
Thus the potential-outcome margins are identical, but the finite-population joint distributions of observed treatment and control outcomes under this fixed-size assignment need not be identical. What the margins alone fail to reveal is the within-unit pairing, and hence the individual effects. For a uniformly selected unit, each arm’s potential outcome has a Bernoulli distribution with mean 0.5 in both worlds; that marginal fact does not make the full joint observed-data distributions the same.
This example separates three claims that are sometimes conflated: treatment and control potential-outcome means can be the same; the average treatment effect can therefore be zero; and individual effects can nevertheless vary. The first two do not establish the third, or rule it out. In general, the marginal distributions of the potential outcomes do not determine the distribution of their differences because the dependence between the two potential outcomes matters.
What Estimated Heterogeneity Can Establish
When estimated conditional effects differ across covariate values, and the analysis appropriately accounts for uncertainty and the design, this can support a claim of conditional mean effect heterogeneity: average effects differ across those covariate groups. It does not by itself show that the model has discovered each unit’s effect, that all members of one group respond alike, or that a specific person will benefit. A conditional estimate is a group-level prediction of a mean, not an observed personal causal contrast.
Likewise, little estimated variation in CATEs does not establish that individual effects are constant. Opposing unit-level effects can average out within every covariate group. To claim that individual effects are constant, one would need stronger assumptions or information than a set of conditional mean comparisons supplies. Such assumptions may be substantively useful, but they should be stated rather than treated as a consequence of randomization or regression adjustment.
Common Mistakes
- Calling a CATE an individual effect. The value \(\tau(x)\) is an average for units with covariates \(x\), not the known effect for each unit in that group.
- Inferring that all units benefit from a positive mean. A positive conditional average can include zero or negative individual effects.
- Equating random assignment with observing both potential outcomes. Randomization supports comparisons across groups, but each unit still reveals only one potential outcome.
- Assuming equal conditional means imply no individual heterogeneity. Effects can vary within groups and cancel in the average.
- Assuming identical potential-outcome margins imply identical joint observed-data distributions. In a finite population with fixed-size assignment, dependence between potential outcomes can affect the joint distribution of observed arm outcomes, even when the margins match.
Check Your Understanding
Answer each question by distinguishing conditional means from unit-level effects.
- If \(\tau(x)=2\) points, what does that say about the average effect among units with covariates \(x\), and what does it not say about each such unit?
- Why can random assignment identify an average effect while leaving an individual unit’s treatment effect unobserved?
- State the conditional exchangeability and positivity conditions used to identify a CATE from observed conditional means.
- Can a zero CATE imply that every unit in the group has zero effect? Explain.
- Why do the separate marginal distributions of \(Y_i(1)\) and \(Y_i(0)\) not generally determine the distribution of \(\tau_i\)?