Why Four Means Form One Contrast
A two-by-two difference-in-differences design has two groups—a treated group and a comparison group—and two periods—a pre-treatment period and a post-treatment period. Its central calculation is a contrast of the four group-by-period outcome means. The first difference measures the change over time in each group; the second difference compares those two changes.
This construction is useful because a group can start at a different outcome level from the comparison group, and outcomes can change over time for reasons that affect both groups. Subtracting the groups’ changes removes these two additive components. That cancellation is an algebraic fact; whether the remaining contrast identifies a causal effect is a separate question.
The four entries are sample means for the same outcome, measured on a consistent scale and defined for the relevant groups and periods. The calculation does not require that the treated and comparison groups have the same pre-treatment mean. It does require that the cell means be meaningful and comparable: a difference in means should not be an artifact of a change in outcome definition or units.
A useful way to check the signs is to read the expression in two stages. First calculate post minus pre separately for each group. Then calculate treated change minus comparison change. Reversing the order of either subtraction reverses the sign of the final contrast.
Worked Example: Calculating the Four-Cell Contrast
Suppose a hypothetical program is introduced for one group. The outcome is monthly earnings in hundreds of dollars, and the four observed cell means are as follows.
| Group | Pre-treatment mean | Post-treatment mean |
|---|---|---|
| Treated | 50 | 62 |
| Comparison | 45 | 53 |
The treated group’s change is \(62-50=12\) hundreds of dollars per month. The comparison group’s change is \(53-45=8\) hundreds of dollars per month. Therefore,
The estimated difference-in-differences contrast is 4 hundreds of dollars per month, or $400 per month. This is the treated group’s additional increase relative to the comparison group’s increase. The arithmetic alone does not establish that the program caused that amount: a causal interpretation requires an assumption about how the two groups’ outcomes would have changed without the program.
Algebraic Cancellation in an Additive Model
The cancellation is easiest to see by writing each cell mean as the sum of a group component, a time component, and a group-by-time component. Let \(m_{g,t}\) denote the population mean outcome in group \(g\) and period \(t\). For a two-by-two table, write
where \(a_g\) is the group-specific level, \(\lambda_t\) is the time-specific component common to both groups, \(\mathbf{1}(g=T,\ t=1)\) equals 1 only for the treated group in the post-treatment period, and \(\delta\) is the group-by-time component. In this parameterization, the four means are
Substitute those four expressions into the population difference-in-differences contrast and expand every term:
The terms \(a_T\) and \(a_C\) cancel within each group’s time change. The common time change \(\lambda_1-\lambda_0\) then cancels when the two group changes are subtracted. The remaining component is the group-specific change, \(\delta\). This derivation explains precisely what the contrast removes: additive differences that do not change over time between groups, and time shocks that are common to both groups.
The additive representation is a way to describe the four means, not by itself evidence that the group-by-time component is a treatment effect. In particular, any change that would have affected the treated group differently even without treatment can also contribute to the observed group-by-time contrast.
Worked Example: Seeing Both Cancellations Numerically
Consider hypothetical outcome means, in points on a test, generated by a treated-group level of 70, a comparison-group level of 50, a common pre-treatment component of 0, and a common post-treatment component of 6. Suppose the program adds 3 points to the treated group’s post-treatment mean. The resulting cell means are:
The treated group’s change is \(79-70=9\) points. This includes the common 6-point time change and the additional 3-point treated-group change. The comparison group’s change is \(56-50=6\) points, consisting of the common time change alone. Thus,
The initial 20-point gap between groups, \(70-50\), does not enter the final contrast because it is constant over time. The common 6-point change also disappears. Under the stipulated construction of these hypothetical means, the remaining 3 points are the added treated-group component.
From the Contrast to a Causal Effect
To see why the calculation is not automatically causal, define \(Y_t(1)\) and \(Y_t(0)\) as the outcomes at time \(t\) under treatment and no treatment, respectively. Let \(G=1\) identify the treated group and \(G=0\) the comparison group. Suppose treatment is absent in period 0, begins for the treated group by period 1, and does not affect outcomes before it begins. With these conditions, the population counterpart of the observed contrast is
The average treatment effect on the treated in the post-treatment period is \(ATT=E[Y_1(1)-Y_1(0)\mid G=1]\). Add and subtract \(E[Y_1(0)\mid G=1]\) in the first group’s change. This gives the exact decomposition
Therefore, the population contrast equals the \(ATT\) when the expected change in untreated outcomes is the same for the two groups. This is the parallel-trends identification condition. The equation makes clear why cancellation of common time shocks is not enough on its own: a group-specific untreated change remains in the contrast. Establishing whether the identifying condition is credible is distinct from deriving or computing the estimator.
Worked Example: A Differential Untreated Change
Suppose a hypothetical outcome is measured in points. The treated group has a pre-treatment mean of 40 and a post-treatment mean of 47. The comparison group’s means are 35 and 38. The treated group’s outcome would have risen by 5 points without treatment, while the comparison group’s untreated outcome rises by 3 points. Suppose treatment adds 2 points to the treated group’s post-treatment outcome. These assumptions produce the stated means: the treated increase is \(5+2=7\), and the comparison increase is 3.
The estimator is
But the treatment effect in this constructed example is 2 points. The remaining 2 points come from the difference between the groups’ untreated changes, \(5-3=2\). The decomposition verifies the result:
Thus, correctly calculating the four-cell contrast does not guarantee that it equals the causal effect. Here, the time-invariant group gap is removed, but the groups do not share the same untreated change, so the contrast includes that difference as well.
How to Report and Check the Estimator
A clear report gives the two group-specific changes, their difference, the outcome units, and a cautious interpretation. For example: “The treated group’s mean increased by 12 hundreds of dollars per month, compared with an increase of 8 hundreds in the comparison group; the difference-in-differences estimate is 4 hundreds of dollars per month.” A causal claim should be added only when the design’s identifying assumptions are justified.
A useful algebraic check is to write the contrast directly in terms of its four cells:
This form confirms that the post-treatment treated mean and pre-treatment comparison mean have positive signs, while the other two means have negative signs. It also makes clear that adding the same constant to all four means leaves the estimate unchanged. More specifically, adding a fixed amount to both treated-group means or adding a common period shift to both groups’ means also leaves it unchanged.
Common Mistakes
- Comparing only post-treatment means. The difference \(\bar{Y}_{T,1}-\bar{Y}_{C,1}\) includes any pre-existing group gap. The estimator compares changes, not just post-treatment levels.
- Subtracting in inconsistent orders. Calculate post minus pre for both groups, then treated minus comparison. Mixing orders can reverse the sign.
- Calling every four-cell contrast a causal effect. The algebra removes fixed group gaps and common time changes, but the contrast can still include different untreated changes across groups.
- Ignoring units or the outcome definition. The contrast has the same units as the outcome. Comparisons are not meaningful if the outcome is measured differently across groups or periods.
Check Your Understanding
Use the four-cell contrast and its causal interpretation to answer these questions.
- If the treated group changes from 30 to 38 and the comparison group changes from 26 to 31, what is the difference-in-differences estimate, in outcome units?
- In the additive model, which terms cancel within each group’s change, and which term cancels when the group changes are subtracted?
- Why can a constant pre-treatment difference between groups coexist with a zero difference-in-differences contrast?
- If the treated group’s untreated outcome would rise by 4 points and the comparison group’s by 1 point, what component besides the treatment effect enters the population contrast?
- Why does calculating the four cell means correctly not, by itself, establish a causal effect?