Why the Contrast Needs a Causal Target
Suppose a policy is introduced for one group between two periods, while another group remains untreated. A change in the treated group after the policy is not, by itself, the policy effect: outcomes could have changed over time even without the policy. Likewise, a post-policy gap between treated and untreated groups can reflect differences that were already present before the policy. Difference-in-differences (DiD) compares changes across the two groups, but interpreting that comparison requires first being clear about the causal quantity it is meant to recover.
This tutorial defines that target in a two-period potential-outcomes framework. The focus is the distinction between the average treatment effect on the treated (ATT), an observed before-after change, an observed treated-control gap, and the DiD contrast. The contrast is a difference of observed group averages; the ATT is a difference between potential outcomes for the treated group, one of which is not observed after treatment.
Potential Outcomes in a Two-Period Design
Let \(t=0\) denote the pre-treatment period and \(t=1\) the post-treatment period. Let \(G_i=1\) indicate that unit \(i\) belongs to the group assigned treatment starting between the two periods, and let \(G_i=0\) indicate membership in the comparison group, which remains untreated. Write \(Y_{it}(1)\) for unit \(i\)'s outcome at time \(t\) under the regime in which treatment starts between periods, and \(Y_{it}(0)\) for its outcome under the regime of remaining untreated. The outcome could be, for example, annual earnings in dollars or a health measure in points.
The observed outcome is the potential outcome under the regime the unit actually experiences. The basic setup assumes a well-defined treatment and outcome, and that one unit's treatment does not change another unit's potential outcomes. It also assumes no anticipation: treatment beginning after period 0 does not affect the pre-treatment outcome. Thus \(Y_{i0}(1)=Y_{i0}(0)\). Treated units are observed untreated at \(t=0\) and treated at \(t=1\); comparison units are untreated in both periods.
The conditioning on \(G_i=1\) is essential. The ATT is not necessarily the average effect for all units in the population. Treatment effects may differ across units, and the treated group may have a different distribution of characteristics from the comparison group. The ATT answers a specific question: what was the average post-period effect for those who received the treatment?
Worked Example: Defining the ATT for a Wage Policy
Suppose a wage policy applies to a set of firms between two years. Earnings are measured in thousands of dollars per worker per year. In a hypothetical treated-group population, the mean post-period outcome under the policy is \(Y_1(1)=62\), while the mean outcome those same workers would have had without the policy is \(Y_1(0)=54\).
The treated-group effect is the difference between these two potential-outcome means: $$ ATT = E[Y_1(1)-Y_1(0)\mid G=1] = 62-54=8. $$
In this hypothetical example, the policy increases post-period earnings for the treated group by an average of 8 thousand dollars per worker per year. The mean of 54 is counterfactual: it describes the untreated outcome for treated workers, not the observed earnings of the comparison group. Substituting the comparison group's mean for that counterfactual would require an additional justification.
Three Quantities That Are Not the Same
Use \(E[Y_t\mid G=g]\) for the mean observed outcome in period \(t\) for group \(g\). In this notation, the treated group's observed pre-period mean is \(E[Y_0(0)\mid G=1]\), and its post-period mean is \(E[Y_1(1)\mid G=1]\). The comparison group's observed means are \(E[Y_0(0)\mid G=0]\) and \(E[Y_1(0)\mid G=0]\).
The treated-group before-after change is the treated group's post-period mean minus its pre-period mean. It includes both the treatment effect and whatever change would have occurred for those units without treatment. The post-period treated-control gap compares different groups at one time; it may include both the ATT and a difference in their post-period outcomes under no treatment. The pre-period treated-control gap is another cross-group comparison, measured before treatment begins.
| Quantity | Definition | What it contains |
|---|---|---|
| Treated before-after change | \(E[Y_1\mid G=1]-E[Y_0\mid G=1]\) | Treatment effect plus the treated group's untreated change over time |
| Post-period treated-control gap | \(E[Y_1\mid G=1]-E[Y_1\mid G=0]\) | ATT plus the post-period untreated-outcome gap between groups |
| Pre-period treated-control gap | \(E[Y_0\mid G=1]-E[Y_0\mid G=0]\) | Untreated-outcome gap between groups before treatment |
| Difference-in-differences contrast | \(\{E[Y_1\mid G=1]-E[Y_0\mid G=1]\}-\{E[Y_1\mid G=0]-E[Y_0\mid G=0]\}\) | ATT plus the difference between groups in their untreated changes |
The last row is the population DiD contrast. It is a well-defined contrast of observed means, but it does not equal the ATT as a matter of arithmetic alone. The relationship follows by adding and subtracting the treated group's unobserved post-period untreated mean.
Why the Decomposition Holds
For treated units, the observed post-period mean is \(E[Y_1(1)\mid G=1]\), and no anticipation means their pre-period mean is \(E[Y_0(0)\mid G=1]\). Add and subtract \(E[Y_1(0)\mid G=1]\) in their before-after change:
For comparison units, both observed outcomes are untreated potential outcomes, so their before-after change is \(E[Y_1(0)-Y_0(0)\mid G=0]\). Subtracting that change from the treated-group change gives the stated decomposition. No assumption about the equality of untreated trends was needed for the algebra. That assumption is needed if the DiD contrast is to identify the ATT.
Worked Example: Separating the Contrasts
Consider a hypothetical program with an outcome measured in points. The treated group has a pre-period mean of 50 and a post-period mean under treatment of 62. Its post-period mean had it remained untreated is 54. The comparison group has an untreated mean of 45 before the program and 49 after it.
Treated before-after change: \(62-50=12\) points. This is not the ATT alone: it includes the 4-point untreated change for the treated group, \(54-50=4\).
Comparison-group change: \(49-45=4\) points. This is its untreated change.
ATT: \(62-54=8\) points. This compares treated and untreated potential outcomes for the same treated-group population in the post period.
DiD contrast: \((62-50)-(49-45)=12-4=8\) points. Here the two groups' untreated changes are both 4 points, so the difference in those changes is zero and the DiD contrast equals the ATT.
Post-period treated-control gap: \(62-49=13\) points, while the pre-period gap is \(50-45=5\) points. The post-period gap is not the ATT because the comparison group has a different post-period untreated mean from the treated group's untreated counterfactual: \(54-49=5\) points. Subtracting the pre-period gap from the post-period gap gives \(13-5=8\), the same DiD contrast.
Worked Example: When DiD and the ATT Differ
Now consider another hypothetical outcome measured in points. The treated group has a pre-period mean of 100, a post-period treated mean of 118, and a post-period untreated counterfactual mean of 110. The comparison group has untreated means of 90 before treatment and 96 afterward.
The ATT is \(118-110=8\) points. The treated group's observed before-after change is \(118-100=18\) points, and the comparison group's change is \(96-90=6\) points. Therefore, $$ DiD=(118-100)-(96-90)=18-6=12\text{ points}. $$
The treated group's untreated change is \(110-100=10\) points, while the comparison group's untreated change is \(96-90=6\) points. Their difference is \(10-6=4\) points, so the decomposition gives \(DiD=ATT+4=8+4=12\) points. The DiD contrast exceeds the ATT by 4 points in this example because the untreated changes are not equal.
The arithmetic identifies the source of the gap; it does not tell us from observed outcomes alone which group's untreated change is the right counterfactual for the other. The untreated post-period outcome for treated units is not observed in practice.
Why the ATT Is Not the Overall Average Effect
The ATT conditions on treatment-group membership. If treatment effects vary, it can differ from the average treatment effect across all units. For example, a program might have larger effects among the people or firms selected into treatment than among those who were not selected. The DiD target described here is the treated-group effect, not automatically the effect of assigning treatment to a randomly selected unit from the full population.
Worked Example: The Treated-Group Target and the Overall Mean
Suppose a hypothetical population contains 10 units: 4 treated units and 6 comparison units. The treatment effect is 10 outcome points for each treated unit and, if treated, 2 points for each comparison unit.
The ATT averages effects only across the 4 treated units, so \(ATT=10\) points. The average treatment effect across all 10 units would instead be $$ ATE=\frac{4(10)+6(2)}{10} =\frac{40+12}{10} =5.2\text{ points}. $$
Thus the two targets differ: 10 points for the treated group versus 5.2 points for the whole population. The example illustrates why the target population must be named when describing an effect.
Interpretation and a Common Pitfall
A DiD contrast is the difference between two observed changes. Its causal interpretation as the ATT requires that the groups would have had the same mean untreated change between periods. When that condition holds, the comparison group's untreated change supplies the missing change for the treated group. Without it, the contrast still describes the observed data, but its difference from the ATT is exactly the difference in the groups' untreated trends shown above. Other design concerns, such as anticipation or spillovers, can also undermine the potential-outcomes setup.
Common mistake: “The treated group improved by 12 points, so the treatment effect was 12 points.” This confuses a before-after change with the ATT. In the first worked example, the treated group's change was 12 points, but 4 points of that change would have occurred without treatment; the ATT was 8 points. A related mistake is to call the post-period treated-control gap the treatment effect. That comparison also reflects any untreated outcome gap between the groups.
Check Your Understanding
Use the definitions and decomposition above to answer the following questions.
- In your own words, why is \(Y_1(0)\) for treated units a counterfactual?
- A treated group's outcome rises by 15 points, while the comparison group's outcome rises by 7 points. What is the DiD contrast, and what additional condition would allow it to identify the ATT?
- If the treated group's untreated outcome would have risen by 9 points and the comparison group's untreated outcome rose by 5 points, how does the DiD contrast relate to the ATT?
- Why can the post-period treated-control gap differ from the ATT even if both are measured in the same units?
- When treatment effects vary across units, why need the ATT differ from the average treatment effect for the full population?