Placebo Reassignment Inference for Synthetic Control

Why Reassign Treatment to Donor Units?

A synthetic control analysis compares an observed treated unit with a weighted combination of untreated donor units. After the intervention, the difference between the treated unit’s outcome and its synthetic control is evidence about the intervention’s effect only insofar as the synthetic control provides a credible counterfactual. A useful diagnostic is to ask: Would a gap this large also appear if a donor unit were treated as though it had received the intervention?

In a placebo reassignment, each eligible donor unit takes a turn as the “treated” unit. Its synthetic control is fitted from the other eligible donor units, using the same pre-intervention information and fitting procedure. The resulting post-intervention gaps form a reference distribution. Comparing the actual treated unit’s statistic with that distribution indicates how unusual its gap is relative to the donor-unit placebos.

This comparison is informative, but it does not by itself make treatment assignment random. In many synthetic-control applications, a policy or intervention was deliberately assigned to one region, institution, or other unit. The donor units are comparison cases, not units randomly selected to receive or avoid treatment. The distinction matters when interpreting a rank as a p-value.

Define the Placebo Statistics

Suppose unit \(1\) receives the intervention at time \(T_0\), and units \(2,\ldots,J+1\) are eligible donor units. Let \(Y_{it}\) denote the observed outcome for unit \(i\) at time \(t\). For the treated unit, let \(\widehat{Y}_{1t}^{N}\) be its synthetic-control prediction: the weighted donor outcome used to approximate what unit \(1\)'s outcome would have been without the intervention.

The statistic must match the question. If a positive gap represents an increase in the outcome and the substantive alternative is an increase, \(S_i\) is a natural one-sided statistic. If departures in either direction matter, use \(|S_i|\) for a two-sided comparison. The direction or two-sided rule should be chosen before inspecting the post-intervention placebo results; otherwise, the reported tail can be selected to make the treated result look more unusual.

The fitting procedure should also be applied consistently: use the same pre-intervention period, predictor choices, optimization rules, and post-period summary for the treated unit and placebos. The actual treated unit should not be used as a donor for post-intervention placebo predictions, because its post-intervention outcomes may themselves be affected by treatment. A donor unit that cannot be fitted credibly should not be made comparable simply by ignoring its poor pre-intervention fit; any eligibility rule should be stated and based on design or pre-intervention information, not chosen to obtain a preferred post-intervention rank.

Rank-Based Evidence

For a positive-effect alternative, compare the treated statistic with all \(J\) placebo statistics using the upper tail. Because the treated unit is itself included as one of the \(J+1\) values, the tail proportion is $$ \widehat{p}_{+}=\frac{1}{J+1}\sum_{i=1}^{J+1} I(S_i\geq S_1), $$ where \(I(\cdot)\) equals 1 when its condition is true and 0 otherwise. The numerator counts the treated unit and every placebo with a statistic at least as large as the treated statistic. Counting ties in the tail is conservative for a test based on ranks.

For a two-sided comparison based on departures in either direction, use absolute statistics: $$ \widehat{p}_{2}=\frac{1}{J+1}\sum_{i=1}^{J+1} I(|S_i|\geq |S_1|). $$ These are rank-based tail proportions. If \(S_1\) is the largest value in the positive direction and there are \(J\) donors, the smallest possible one-sided proportion is \(1/(J+1)\), not zero. With few eligible units, rank evidence is necessarily coarse.

When Does a Rank Have a Randomization-Test Interpretation?

A rank-based tail proportion has an exact randomization-test interpretation only under a suitable assignment mechanism and null hypothesis. For example, suppose treatment really was assigned uniformly at random to exactly one of \(J+1\) eligible units. Under the sharp null hypothesis that treatment changes no unit’s outcome in any period, the observed outcomes would be the same under every possible assignment. One can then calculate the same statistic for each possible assignment, treating each unit in turn as treated and fitting its synthetic control by the same rule.

If the procedure treats units symmetrically under the possible assignments, the statistic for the actually assigned unit has a uniform rank among the \(J+1\) statistics when there are no ties. With ties, counting all statistics at least as extreme as the observed one yields a conservative tail probability. The argument can be seen directly:

This justification depends on the assignment mechanism, not merely on having a list of donors. If the probabilities of assignment are unequal but known, the assignments must be weighted according to those probabilities rather than treated as equally likely. Further, the sharp null is stronger than a null about an average effect: it says there is no treatment effect for any unit. A randomization p-value for this sharp null does not automatically provide a confidence interval for an average effect.

Why Placebo Reassignment Matters—and Its Limits

Placebo reassignment helps put a treated unit’s post-intervention gap in context. If the treated statistic is more extreme than nearly all donor placebos, that is evidence that its trajectory differs from the patterns observed among these comparison units. If many placebo units show similarly large gaps, a large treated gap is less distinctive within this donor set. Either way, the comparison is most useful when placebos can be fitted credibly and the same statistic is used throughout.

The rank does not repair a weak causal design. In an observational application, the treated unit may differ systematically from the donors, and the intervention may have been assigned for reasons related to future outcomes. The donor units may not be exchangeable with the treated unit, and their outcomes need not provide a valid distribution of what would have happened to the treated unit without intervention. The causal interpretation still depends on the synthetic-control design and its identification assumptions; a favorable rank is not a substitute for them.

A rank is also not a treatment-effect estimate. The estimated gaps summarize differences in outcome units, while the rank summarizes where that statistic falls relative to the chosen placebo statistics. A small rank tail does not quantify the effect’s magnitude, and a large tail does not prove the effect is zero. With a small donor pool, the number of distinct rank values is limited, which restricts how finely the evidence can be expressed.

Common Mistakes

• Calling every placebo rank a randomization p-value. Reassigning treatment labels in an analysis does not establish that treatment was randomly assigned. Without a justified assignment mechanism and the relevant null, the rank is a comparative diagnostic, not an exact design-based test.
• Changing the statistic across units. Comparing a treated unit’s average gap with a placebo’s single-period gap does not produce a meaningful common rank. Define the outcome summary and apply it consistently.
• Choosing the tail after viewing results. A positive-direction tail and an absolute-value tail answer different questions. Specify the direction or two-sided statistic before examining the placebo ranks.
• Ignoring ties or the number of units. Ties belong in the tail count, and with \(J\) donors the smallest possible rank proportion is \(1/(J+1)\). Reporting zero or implying fine precision is incorrect.
• Treating the rank as an effect size. Report the outcome gap in its original units as well as the rank comparison. The rank alone does not say how large the estimated effect is.