Why a Randomized Difference in Means Has a Causal Target
In a randomized trial, the observed mean outcome among treated units is often compared with the observed mean among control units. The comparison is simple to calculate, but its causal interpretation depends on the study design. This tutorial derives the precise result: if a fixed set of units is assigned completely at random to treatment or control, and every assigned unit’s outcome is measured, then the expected treated-minus-control sample mean difference equals the average treatment effect for those units.
The expectation here is over the possible assignments in the randomization procedure. It does not say that the difference from one particular trial must equal the true effect. Rather, the estimator is centered on the target when we imagine repeating the assignment procedure over the same units with the same potential outcomes.
Assumptions and the Randomization Setup
The result concerns a finite set of \(N\) units, whose two potential outcomes are treated as fixed. The assignment is random; the potential outcomes are not being averaged over a hypothetical population sample. Under complete random assignment, exactly \(n_1\) units are selected uniformly from the \(N\) units for treatment, and the remaining \(n_0\) receive control. Both group sizes must be positive.
The study must also define a consistent treatment and outcome for each unit, and record the outcome for every unit in its assigned group. Under consistency, the observed outcome \(Y_i\) is the potential outcome corresponding to the assigned treatment \(A_i\):
This setup relies on the usual no-interference and well-defined-treatment conditions: one unit’s outcome under a treatment does not change with other units’ assignments, and the treatment state has a clear meaning. The randomization result is about the causal effect for the study units; it does not, by itself, establish that those units represent a broader population.
Deriving the Expected Difference
Let \(I_i=A_i\) indicate whether unit \(i\) is assigned to treatment. Under complete random assignment, each unit has probability \(n_1/N\) of treatment and \(n_0/N\) of control. Because the group sizes are fixed, the treated mean can be written as a sum over treatment indicators:
Likewise, the control mean is a sum over control indicators:
The potential outcomes are fixed while assignment varies. Linearity of expectation therefore lets us take the expectation term by term. Since \(E(I_i)=n_1/N\) and \(E(1-I_i)=n_0/N\), the expected treated mean is the average of all treatment potential outcomes, and the expected control mean is the average of all control potential outcomes:
Subtracting these expectations gives the result:
No assumption that every unit has the same treatment effect is needed. The derivation averages each unit’s treatment potential outcome and each unit’s control potential outcome separately; their difference is the average of the individual effects.
Worked Example: Enumerating Every Assignment
Consider four hypothetical units, with potential outcomes measured in outcome points. Suppose exactly two are assigned to treatment. The potential outcomes are shown below; in an actual trial, only one of the two potential outcomes would be observed for each unit.
| Unit | \(Y_i(1)\) | \(Y_i(0)\) | Individual effect |
|---|---|---|---|
| 1 | 12 | 10 | 2 |
| 2 | 8 | 8 | 0 |
| 3 | 9 | 5 | 4 |
| 4 | 8 | 6 | 2 |
The finite-population average treatment effect is \( \tau=(2+0+4+2)/4=2 \) outcome points. There are six equally likely assignments of two units to treatment. For each assignment, calculate the treated mean using those units’ \(Y_i(1)\) values, and the control mean using the other units’ \(Y_i(0)\) values.
| Treated units | Treated mean | Control mean | Difference |
|---|---|---|---|
| 1, 2 | \((12+8)/2=10\) | \((5+6)/2=5.5\) | 4.5 |
| 1, 3 | \((12+9)/2=10.5\) | \((8+6)/2=7\) | 3.5 |
| 1, 4 | \((12+8)/2=10\) | \((8+5)/2=6.5\) | 3.5 |
| 2, 3 | \((8+9)/2=8.5\) | \((10+6)/2=8\) | 0.5 |
| 2, 4 | \((8+8)/2=8\) | \((10+5)/2=7.5\) | 0.5 |
| 3, 4 | \((9+8)/2=8.5\) | \((10+8)/2=9\) | -0.5 |
The average difference over the six assignments is \( (4.5+3.5+3.5+0.5+0.5-0.5)/6=12/6=2 \) outcome points. It equals the finite-population average treatment effect, although several individual assignment differences are far from 2.
What the Result Means for One Trial
In practice, only one assignment is observed, so the true \(\tau\) is generally unknown: each unit reveals only its outcome under the treatment it received. The observed difference \(\hat{\tau}_{DM}\) is the available estimate. Unbiasedness means its average over repetitions of the specified assignment procedure equals \(\tau\); it does not guarantee that this one estimate equals \(\tau\), nor does it imply that every assignment is equally close to the target.
The statement also concerns the study units’ average effect. If participants were recruited in a way that limits generalization, randomizing treatment among them does not automatically make their average effect the average effect in another population. Random assignment addresses how treatment groups are formed; it is not the same as random sampling of units.
Worked Example: Calculating an Observed Trial Difference
Suppose a hypothetical randomized trial assigns three participants to treatment and three to control. Their measured outcomes, in symptom-score points, are:
| Assigned group | Observed outcomes | Mean |
|---|---|---|
| Treatment | 14, 10, 12 | 12 |
| Control | 8, 11, 9 | \(28/3=9.333\) |
The observed difference is \( \hat{\tau}_{DM}=(14+10+12)/3-(8+11+9)/3=12-9.333=2.667 \) symptom-score points. A positive value means the observed treatment-group mean exceeds the control-group mean by about 2.667 points.
If assignment was complete and random, every participant’s outcome was measured, and the treatment and outcome satisfy the conditions above, this is an unbiased estimate of the participants’ finite-population average effect. It is not proof that their true average effect is exactly 2.667 points: other valid assignments could have produced a different difference. Without the unobserved potential outcomes, the true finite-population effect cannot be calculated directly from this table alone.
Allocation Size, Checks, and Failure Modes
The derivation does not require equal group sizes. What matters is that treatment assignment is uniform among all allocations with the specified group sizes. For example, a trial assigning one unit to treatment and three to control still has the same unbiasedness property. Here is a direct check with three hypothetical units and one treated unit.
Worked Example: An Unequal Allocation
Suppose three units have potential outcomes \((Y_i(1),Y_i(0))\) equal to \((7,5)\), \((4,6)\), and \((10,8)\), respectively. Their individual effects are 2, -2, and 2 points, so \( \tau=(2-2+2)/3=2/3 \) point. With exactly one treated unit, the three assignments are equally likely.
If unit 1 is treated, the difference is \(7-(6+8)/2=0\). If unit 2 is treated, it is \(4-(5+8)/2=-2.5\). If unit 3 is treated, it is \(10-(5+6)/2=4.5\). The assignment-average difference is \( (0-2.5+4.5)/3=2/3 \) point, matching \(\tau\). The example illustrates why unequal group sizes do not invalidate the result: each unit still has treatment probability \(1/3\), and the group means average the corresponding potential outcomes correctly in expectation.
Before interpreting a difference in means causally, check the design and the data, not just the arithmetic. Useful checks include:
- Assignment: Was treatment allocated by the stated random mechanism, or were groups formed by choice, eligibility, or another process?
- Group membership: Do the observed counts match the planned allocation, and is each unit counted in its assigned group?
- Outcome completeness: Is the outcome recorded for every randomized unit? If missingness leads to dropping some units, the observed group means may no longer be averages over the randomized groups.
- Treatment and measurement: Were the treatment versions sufficiently well-defined, and was the outcome measured consistently across groups?
The main alternatives answer different needs. A covariate-adjusted analysis may be used to improve precision or to accommodate a more complex design, but it is not needed for the randomization-based unbiasedness derivation here. In an observational comparison, treatment groups are not created by random assignment, so the expectation argument above does not apply merely because two sample means can be calculated. The difference-in-means calculation remains descriptive; a causal interpretation then requires additional design or identification conditions.
Common Mistakes
- Claiming the observed difference must equal the ATE. The theorem is an expectation over assignments. A realized difference may be larger or smaller, as the enumerated example shows.
- Treating random assignment as random sampling. Assignment supports a causal comparison among the randomized units; it does not alone establish representativeness of a wider population.
- Ignoring missing outcomes. If outcomes are unavailable for some assigned units and the analysis simply omits them, the complete-measurement derivation no longer directly applies. The remaining participants need not preserve the original randomization balance.
- Using the result for a nonrandom group comparison. The proof uses each unit’s known random assignment probability. A numerical difference between groups, without that design, does not inherit the theorem’s causal justification.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions below.
- In the derivation, what is random: the potential outcomes, the assignments, or both?
- A trial’s observed difference is 1.8 points, while its finite-population ATE is 1.2 points. Is this inconsistent with the theorem? Explain.
- With \(N=10\) units and exactly \(n_1=4\) assigned to treatment, what is each unit’s treatment-assignment probability under complete random assignment?
- Does complete random assignment by itself show that the trial participants’ ATE equals the ATE in a national population?
- Why can dropping participants with missing outcomes make the stated result inapplicable to the analyzed group means?
Solutions. (1) The potential outcomes are fixed; the assignments are random. (2) No. The theorem says the estimator’s assignment expectation is 1.2 points, not that every realized difference equals 1.2. (3) Each unit has probability \(4/10=0.4\) of treatment. (4) No. Random assignment supports causal comparison for the randomized units, not representativeness of a national population. (5) The analyzed means may average outcomes over selected subsets rather than all units assigned to each group, so the expectation calculation for complete groups does not automatically apply.