Why a Difference in Means Can Estimate a Causal Effect
In a randomized experiment, we observe each unit’s outcome under the treatment it received, not under both possible assignments. A comparison of the treated and control means is therefore not a direct comparison of two observed outcomes for the same units. Its justification instead comes from the assignment mechanism: under complete randomization, every unit has the same chance of entering either group, and the group sizes are fixed. This tutorial derives what that fact implies for the difference in means.
The target here is the finite-population average treatment effect: the average of the treatment effects for the particular set of experimental units. The result is a randomization statement. It says that if we repeatedly assigned treatment according to the same design while keeping all potential outcomes fixed, the average difference in means over those assignments would equal that finite-population target.
Complete Randomization and the Estimator
Let \(Z_i=1\) if unit \(i\) is assigned to treatment and \(Z_i=0\) if it is assigned to control. Under complete randomization, exactly \(n_1\) of the \(N\) units are assigned to treatment, exactly \(n_0=N-n_1\) to control, and every set of \(n_1\) treated units is equally likely. Both group sizes are positive.
Assume the potential outcomes are well-defined and that the assignment does not change one unit’s outcome through another unit’s treatment. These are the relevant consistency and no-interference conditions associated with SUTVA. The observed outcome is then \(Y_i=Z_iY_i(1)+(1-Z_i)Y_i(0)\). The potential outcomes are held fixed in the randomization analysis; randomness comes from the assignment \(Z_1,\ldots,Z_N\).
The observed treated mean averages \(Y_i(1)\) over the treated units, and the observed control mean averages \(Y_i(0)\) over controls. Their difference is the usual difference-in-means estimator:
The expression includes all \(N\) units, but only the assigned group contributes to each sum. For example, when \(Z_i=0\), unit \(i\)'s term in the first sum is zero; when \(Z_i=1\), its term in the second sum is zero. The estimator uses the observed outcomes because the treated units reveal their \(Y_i(1)\) values and the controls reveal their \(Y_i(0)\) values.
Why the Estimator Is Unbiased
Write \(E_Z(\cdot)\) for expectation over the complete-randomization assignment distribution, conditional on the fixed potential outcomes. Each unit has probability \(n_1/N\) of being selected for treatment: there are \(\binom{N-1}{n_1-1}\) treatment sets containing that unit out of \(\binom{N}{n_1}\) equally likely sets. Thus \(P(Z_i=1)=n_1/N\), and \(P(Z_i=0)=n_0/N\).
To prove the result, take the assignment expectation of each part of the estimator. Linearity of expectation and the equal inclusion probabilities give:
For the control mean, \(E_Z(1-Z_i)=P(Z_i=0)=n_0/N\), so the corresponding calculation is:
Subtracting these two expectations yields \(N^{-1}\sum_i[Y_i(1)-Y_i(0)]=\tau\), proving the claim. The fixed group sizes matter in the estimator’s denominators, while equal treatment probabilities make each group mean unbiased for the corresponding finite-population mean.
Worked Example: Enumerating Every Allocation
Suppose a hypothetical experiment has four units and assigns two to treatment and two to control. Their potential outcomes, measured in outcome points, are:
| Unit | \(Y_i(0)\) | \(Y_i(1)\) | \(\tau_i\) |
|---|---|---|---|
| 1 | 4 | 7 | 3 |
| 2 | 6 | 7 | 1 |
| 3 | 8 | 11 | 3 |
| 4 | 10 | 12 | 2 |
The target is the average of the four unit effects: \(\tau=(3+1+3+2)/4=2.25\) outcome points. There are \(\binom{4}{2}=6\) equally likely assignments. For each assignment, calculate the mean of the treated units’ \(Y_i(1)\) values minus the mean of the controls’ \(Y_i(0)\) values.
| Treated units | Treated mean | Control mean | \(\widehat{\tau}\) |
|---|---|---|---|
| 1, 2 | \((7+7)/2=7\) | \((8+10)/2=9\) | \(-2\) |
| 1, 3 | \((7+11)/2=9\) | \((6+10)/2=8\) | 1 |
| 1, 4 | \((7+12)/2=9.5\) | \((6+8)/2=7\) | 2.5 |
| 2, 3 | \((7+11)/2=9\) | \((4+10)/2=7\) | 2 |
| 2, 4 | \((7+12)/2=9.5\) | \((4+8)/2=6\) | 3.5 |
| 3, 4 | \((11+12)/2=11.5\) | \((4+6)/2=5\) | 6.5 |
The randomization expectation is the equally weighted average over the six possible estimates: \[ E_Z(\widehat{\tau})=\frac{-2+1+2.5+2+3.5+6.5}{6} =\frac{13.5}{6}=2.25. \] It matches the finite-population average treatment effect. But a particular allocation, such as treating units 1 and 4, gives an estimate of 2.5 points rather than 2.25. Unbiasedness is an average-over-assignments property, not a claim that every assignment produces the target exactly.
What a Realized Difference Means
The difference in means is not generally the average of the treatment effects of the units who happened to be treated. To see why, write \(Y_i(1)=Y_i(0)+\tau_i\) for treated units. For any realized assignment, the estimator can be rearranged as:
The first term is the average treatment effect among the units assigned to treatment. The bracketed term is the difference in their average control potential outcome from that of the controls. This second term is not observed, because the treated units’ control potential outcomes are missing. Random assignment makes its expectation zero across assignments; it does not force it to be zero in the particular assignment observed. The theorem therefore justifies the difference in means as an unbiased estimator of the finite-population average effect, not as an exact effect for each realized group.
Worked Example: A Realized Difference Need Not Be a Group’s Average Effect
Return to the four-unit hypothetical population above and consider the assignment that treats units 1 and 4. The treated units’ average effect is \((3+2)/2=2.5\) points. Their average control potential outcome is \((4+10)/2=7\), while the control units’ average control potential outcome is \((6+8)/2=7\). Therefore the bracketed composition term is \(7-7=0\), and the observed difference is \(2.5+0=2.5\) points.
For the assignment treating units 1 and 2, the average effect among those treated is \((3+1)/2=2\) points. Their average \(Y_i(0)\) is \((4+6)/2=5\), compared with \((8+10)/2=9\) for controls. The composition term is \(5-9=-4\), so the difference in means is \(2-4=-2\) points. The estimate is negative even though every unit’s treatment effect in this population is positive. This is possible because the control potential outcomes differ substantially between the two realized groups; it does not contradict unbiasedness over all six assignments.
Worked Example: Checking the Expectation With Unequal Group Sizes
Consider five hypothetical units, with two assigned to treatment and three to control. Let their treatment potential outcomes be \(5, 9, 8, 12, 11\), and their control potential outcomes be \(4, 6, 7, 10, 9\), respectively. The finite-population mean under treatment is \((5+9+8+12+11)/5=9\); under control it is \((4+6+7+10+9)/5=7.2\). Thus \(\tau=9-7.2=1.8\) outcome units.
Each unit has probability \(2/5\) of treatment and \(3/5\) of control. Consequently, the expected treated mean is: \[ \frac{1}{2}\sum_{i=1}^{5}\frac{2}{5}Y_i(1) =\frac{1}{5}(5+9+8+12+11)=9. \] The expected control mean is: \[ \frac{1}{3}\sum_{i=1}^{5}\frac{3}{5}Y_i(0) =\frac{1}{5}(4+6+7+10+9)=7.2. \] Their difference in expectation is \(9-7.2=1.8\), exactly the finite-population average treatment effect. The argument works with unequal group sizes because each mean divides by its own fixed count, which cancels that group’s assignment probability.
Common Mistakes and Practical Interpretation
- Claiming every assignment gives the exact ATE. The derivation establishes \(E_Z(\widehat{\tau})=\tau\), not \(\widehat{\tau}=\tau\) for each realized allocation.
- Calling the estimator the treated units’ average effect. The observed difference also includes the unobserved difference in average control potential outcomes between the groups. That composition difference averages to zero over the randomization distribution, but may be nonzero in the observed assignment.
- Changing the assignment mechanism without changing the argument. The proof uses fixed group sizes and equal assignment probabilities for all units. A design with unequal probabilities or other restrictions requires its own justification.
- Interpreting the result as a population-wide claim beyond these units. The target defined here averages effects over the \(N\) experimental units. Extending conclusions to a broader population requires additional assumptions beyond this finite-population randomization result.
The result matters because it gives a direct design-based interpretation to the familiar treated-minus-control contrast, even when treatment effects vary across units. The estimate can vary from one allowable allocation to another, and this derivation alone does not quantify that variation. Its conclusion is precise: with fixed counts under complete randomization, the difference in observed group means has randomization expectation equal to the finite-population average treatment effect.
Check Your Understanding
Use the fixed-potential-outcome, complete-randomization setup to answer these questions.
- In a study with \(N=20\) units and \(n_1=8\) treated units, what is the treatment-assignment probability for any one unit?
- Why is the expected treated mean equal to the finite-population mean of \(Y_i(1)\), even though only \(n_1\) treatment outcomes are observed in one assignment?
- Does unbiasedness imply that the observed difference in means must equal the finite-population average treatment effect? Explain.
- In the decomposition of the realized difference, what does the bracketed term represent, and what is its expectation under complete randomization?
- What feature of complete randomization makes the proof inapplicable without modification to unequal assignment probabilities?