Why Use Placebos in Difference-in-Differences?
A difference-in-differences (DiD) estimate is credible only if the comparison group helps represent how the treated group’s outcome would have changed without treatment. That identifying condition is not directly observed after treatment. Placebo analyses provide a way to look for warning signs: they apply a DiD contrast where a treatment effect should not be present, either by pretending treatment began earlier or by using an outcome the intervention should not affect.
A placebo contrast is a diagnostic, not a second estimate of the intervention’s effect. A nonzero result raises questions about the design or data. A zero result means only that this particular diagnostic did not reveal a discrepancy; it does not prove that the main DiD estimate is unbiased.
Defining a Placebo-Date Contrast
Let \(G=1\) denote the treated group and \(G=0\) the comparison group. Let \(\bar{Y}_{g,t}\) be the mean outcome in group \(g\) at time \(t\). For two periods \(a\) and \(b\), the DiD contrast is the treated group’s change minus the comparison group’s change:
For a placebo date, choose two dates that both precede the actual intervention, then treat the later date as if it were the intervention date. Because neither group has yet received treatment, the intervention cannot explain a difference in their changes over this interval. The resulting contrast can reveal differential pre-intervention movement in the outcome.
The useful comparison is between changes, not starting levels. The groups may have different outcome means at either placebo date. What matters for this diagnostic is whether their changes over the selected interval differ. The placebo dates should be chosen to match the study’s timing and outcome measurement as sensibly as possible, and neither date should include exposure to the intervention.
Worked Example: A Placebo Intervention Date Before the Program
A hypothetical city introduces a job-placement program after year 2. The table gives average monthly earnings, in hundreds of dollars, for an eligible treated neighborhood and a comparison neighborhood. All three years shown are before the program begins.
| Group | Year 0 | Year 1 | Year 2 |
|---|---|---|---|
| Treated neighborhood | 50 | 54 | 59 |
| Comparison neighborhood | 40 | 42 | 45 |
Suppose year 1 is chosen as a placebo date, so the placebo interval runs from year 0 to year 1. Earnings increased by \(54-50=4\) hundreds of dollars in the treated neighborhood and by \(42-40=2\) hundreds of dollars in the comparison neighborhood. Therefore:
The placebo contrast is 2 hundreds of dollars, or $200, over that year. Since the program had not begun, this difference cannot be attributed to its effect. It is a warning that the groups did not have identical earnings changes in this pre-intervention interval. It does not, by itself, establish why those changes differed or determine the size or direction of bias in a later estimate.
Why Should a Placebo-Date Contrast Be Zero?
The justification follows from the same parallel-trends idea that motivates DiD. Let \(\mu_{g,t}(0)\) be the mean outcome that group \(g\) would have at time \(t\) without treatment. For a placebo interval before treatment, the observed group means are untreated means. If untreated outcomes satisfy parallel trends over that interval, then their changes are equal:
Subtracting the right-hand change from the left-hand change gives zero. Since the observed means in the placebo interval equal the corresponding untreated means, the expected placebo contrast is therefore zero under this condition. This is a short implication of parallel trends, not a guarantee that every finite-sample calculation will equal exactly zero.
The diagnostic is informative only to the extent that the selected pre-intervention period is relevant to the main analysis. A contrast over an unusual short window, a window with different seasonal conditions, or a period with a change in measurement may not be a useful guide to changes after the actual intervention. A placebo date also cannot rule out a departure from parallel trends that starts only after treatment begins.
Worked Example: A Zero Placebo Contrast Despite Different Levels
In another hypothetical study, an intervention starts after year 2. Before that date, the treated group’s average outcome is 100 units in year 0 and 106 units in year 2. The comparison group’s average is 80 units in year 0 and 86 units in year 2. Calculate the placebo contrast for years 0 to 2.
The treated group changes by \(106-100=6\) units. The comparison group changes by \(86-80=6\) units. Thus:
The treated group starts 20 units above the comparison group and remains 20 units above it at year 2. That level difference does not make the placebo contrast nonzero: both groups changed by the same amount. This zero contrast is consistent with parallel trends over this particular interval, but it does not show that trends were parallel in every pre-intervention period or would remain so after treatment.
Placebo Outcomes
A second approach uses an outcome that the intervention should not affect. Calculate the DiD contrast for this outcome over the actual study’s pre- and post-intervention periods. If the placebo outcome is genuinely unaffected by treatment, a nonzero contrast is a warning that something besides the intended intervention may be producing a group-specific change.
A useful placebo outcome should be chosen for a defensible reason, not merely because it produces a convenient result. It should be measured comparably across groups and time, and it should not be affected by the intervention through a direct or indirect pathway. If it is also sensitive to the same data-collection processes or background changes as the primary outcome, its diagnostic value may be greater. Even then, its behavior cannot be assumed to mirror every source of bias in the primary outcome.
Worked Example: Adult Height as a Placebo Outcome
A hypothetical counseling program aims to improve blood-pressure management. Researchers use average adult height in centimeters as a placebo outcome: the program is not expected to change adults’ height over the study period. Consider these means before and after the program begins.
| Group | Before | After |
|---|---|---|
| Treated clinics | 168.0 cm | 168.7 cm |
| Comparison clinics | 169.1 cm | 169.2 cm |
The treated clinics’ mean height increased by \(168.7-168.0=0.7\) cm. The comparison clinics’ mean increased by \(169.2-169.1=0.1\) cm. The placebo-outcome contrast is:
The 0.6 cm contrast is not evidence that the counseling program increased height. Rather, it is an unexpected difference in a measure that should not respond to the program. The researchers should investigate possibilities such as changes in who was measured, inconsistent measurement procedures, or other group-specific changes. This finding does not identify which explanation is correct, and it does not quantify bias in the blood-pressure outcome.
Choosing and Interpreting Placebos Carefully
A placebo is only useful when its role is explicit. Before calculating a contrast, state why the chosen date is wholly pre-treatment or why the chosen outcome should be unaffected. Then verify that the groups and periods are measured on comparable definitions and scales. The arithmetic is usually simple; the design argument behind the placebo is the important part.
For a date placebo, name the pre-intervention interval. For an outcome placebo, explain why the intervention should not affect that outcome.
Confirm that the same group definitions, outcome measurement, and relevant time window are used for treated and comparison observations.
Find the treated-group change, find the comparison-group change, and subtract the latter from the former.
Describe the size and units, then explain what feature of the design or data the result calls into question. Do not label it a treatment effect.
A nonzero placebo contrast is diagnostic evidence, not proof that the main DiD estimate is invalid. It may reflect differential untreated trends, but it can also result from measurement changes, shifts in group composition, or an unsuitable placebo. Conversely, a contrast near zero does not establish the identifying assumption: the selected interval or outcome may simply fail to reveal the relevant problem. Placebos are most useful as one component of a design assessment, interpreted alongside the substantive setting and the observed outcome patterns.
Common Mistake
Wrong reasoning: “The placebo-date contrast is zero, so the groups satisfy parallel trends and the treatment effect is identified.” A single zero contrast describes only the chosen outcome and interval; it does not establish what would have happened after treatment or exclude other differences between groups.
Correction: Say that the placebo contrast did not reveal differential change for that diagnostic. Report a nonzero contrast as a reason to investigate the design and data, not as proof of a particular bias. In either case, the causal interpretation of the main DiD estimate still depends on the substantive assumptions supporting the comparison.
Check Your Understanding
For each question, distinguish the calculation from what the result can establish.
- A treatment starts after year 3. Which years could be used to form a placebo-date contrast, and why should years after treatment not be used for this purpose?
- In a placebo interval, the treated group changes by 5 outcome units and the comparison group changes by 2 units. Calculate the placebo contrast and state its diagnostic interpretation.
- Why can treated and comparison groups have different outcome levels and still have a zero placebo contrast?
- What substantive condition must hold for an outcome to serve as a placebo outcome?
- A placebo-outcome contrast is zero. Give one reason this does not prove that the main DiD estimate is unbiased.