Why Examine Donor Dependence?
A synthetic control represents the untreated outcome trajectory of a treated unit as a weighted combination of comparison units, called donors. A good pre-intervention fit is important, but it does not by itself show whether the comparison is broadly supported by the donor pool or relies mostly on one donor. If one unit carries most of the weight, the estimated post-intervention effect may be especially sensitive to that unit’s particular trajectory.
A direct diagnostic is to remove a donor and refit the synthetic control using the remaining eligible donors. The question is not whether the original weights can be edited to omit a unit. It is whether the estimated comparison and treatment effect remain similar when that unit is unavailable and the weights are re-estimated under the same design choices.
What Is Refit, and What Is Compared?
Let unit 1 be the treated unit, and let \(j=2,\ldots,J+1\) index the \(J\) donors. Write \(Y_{it}\) for the observed outcome for unit \(i\) at time \(t\), and let \(T_0\) be the last pre-intervention period. The synthetic-control weights \(w_j\) are nonnegative and sum to one. They are chosen to construct a weighted donor comparison that fits the treated unit on the selected pre-intervention outcomes and predictors.
For a specified set of weights, the synthetic outcome at time \(t\) is \(\sum_{j=2}^{J+1} w_jY_{jt}\). The estimated gap at a post-intervention time is the treated unit’s observed outcome minus this synthetic outcome. The average post-intervention gap over \(T_1\) post-intervention periods is:
For an exclusion analysis, let \(S\) be the set of donors retained after exclusion. The refitted weights \(w_j^{(-k)}\) for excluding donor \(k\) must satisfy \(w_j^{(-k)}\geq 0\) for \(j\in S\) and \(\sum_{j\in S}w_j^{(-k)}=1\). They are re-estimated to fit the treated unit using the same outcome and predictor specifications as the original analysis, but with donor \(k\) unavailable. The exclusion-specific effect is computed from those refitted weights, not the original weights.
Compare the original and exclusion-specific post-intervention effects in context, and also inspect pre-intervention fit. One common summary of pre-intervention fit is the root mean squared prediction error (RMSPE), the square root of the average squared difference between the treated outcome and the synthetic outcome over the pre-intervention periods. A large increase in RMSPE after exclusion means the remaining donors reproduce the pre-intervention outcome less closely. It is important to consider both diagnostics: a changed effect with much poorer pre-fit is different from a changed effect despite similarly good pre-fit.
Worked Example: Refitting After Removing the Largest-Weight Donor
Consider a hypothetical analysis with three donors, A, B, and C. The original fitted weights are 0.50, 0.30, and 0.20, respectively. The treated unit’s outcomes in two post-intervention periods are 120 and 124. Donor outcomes in those periods are A: 100 and 102; B: 110 and 112; and C: 130 and 132. The original synthetic outcomes are:
The two estimated gaps are \(120-109=11\) and \(124-111=13\) outcome units. Thus the average estimated post-intervention effect is \((11+13)/2=12\) outcome units.
Now remove donor A and refit. Suppose the hypothetical optimization gives weights 0.60 for B and 0.40 for C. These are nonnegative and sum to one. The refitted synthetic outcomes are \(0.60(110)+0.40(130)=118\) and \(0.60(112)+0.40(132)=120\). The gaps are \(120-118=2\) and \(124-120=4\), so the exclusion-specific average effect is \((2+4)/2=3\) outcome units. It is 9 outcome units lower than the original estimate of 12.
This difference signals sensitivity to donor A in this hypothetical analysis. It does not alone establish that the original estimate is invalid: the pre-intervention fit after excluding A also matters, as do the identification assumptions for synthetic control.
Measuring Weight Concentration
The weight vector describes how much each donor contributes to the original synthetic control. Several simple summaries make concentration easier to see. The largest weight identifies the donor with the greatest individual contribution. The Herfindahl concentration index, also called the sum of squared weights, accounts for all donors. Its reciprocal is often interpreted as the effective number of equally weighted donors: the number of equal contributors that would yield the observed concentration.
The effective donor count is bounded between one and \(J\). Since \(0\leq w_j\leq 1\), each \(w_j^2\leq w_j\), and hence \(H\leq\sum_j w_j=1\). Also, the Cauchy–Schwarz inequality gives \((\sum_j w_j)^2\leq J\sum_jw_j^2\). Because the weights sum to one, this implies \(H\geq 1/J\). Taking reciprocals gives \(1\leq N_{\mathrm{eff}}\leq J\). With \(K\) equally weighted donors, each weight is \(1/K\), so \(H=K(1/K)^2=1/K\), and \(N_{\mathrm{eff}}=K\), as intended.
Worked Example: Calculating Concentration and Effective Donor Count
Suppose a hypothetical synthetic control uses four donors with weights 0.70, 0.20, 0.08, and 0.02. The weights are nonnegative and sum to \(0.70+0.20+0.08+0.02=1.00\), so they form a valid weight vector.
The maximum weight is 0.70, meaning donor 1 contributes 70% of the weighted donor outcome. The concentration index is:
The effective donor count is \(N_{\mathrm{eff}}=1/0.5368\approx1.863\). Although four donors have positive weights, the concentration is comparable to that of about 1.86 equally weighted donors. This is a warning to inspect sensitivity to the leading donor; it is not proof that the estimated effect is biased or that only two donors matter substantively.
Interpreting Exclusion Results
A donor-exclusion analysis is most informative when it reports both the effect estimate and the quality of pre-intervention fit for every refit. A useful summary includes the excluded donor, the refitted weights, the average post-intervention gap, the change from the original gap, and the pre-intervention RMSPE. If the exclusion-specific effect changes substantially while pre-fit remains comparably good, the substantive conclusion depends on which donor is available. If pre-fit deteriorates sharply, the change in effect may reflect that the reduced donor pool cannot reproduce the treated unit’s prior trajectory well.
The magnitude of a change should be judged in the outcome’s units and against the scientific question. A change of 5 units can be minor for an outcome measured in thousands and consequential for an outcome measured on a narrow scale. Also examine whether the direction of the estimated gap changes, whether the main substantive interpretation changes, and whether one donor is influential because of its weight, its outcome trajectory, or both.
Worked Example: Comparing Exclusion Refits
Return to the hypothetical analysis with original average effect 12 outcome units. Suppose the optimizer produces the following refit summaries. RMSPE is measured in the same outcome units as the pre-intervention outcome.
| Donor excluded | Average post-intervention effect | Pre-intervention RMSPE |
|---|---|---|
| None (original fit) | 12.0 | 0.80 |
| A | 3.0 | 1.40 |
| B | 6.0 | 1.00 |
| C | 14.6 | 1.20 |
For excluding A, the effect change is \(3.0-12.0=-9.0\) outcome units, and the RMSPE change is \(1.40-0.80=0.60\) outcome units. For excluding B, the corresponding changes are \(6.0-12.0=-6.0\) and \(1.00-0.80=0.20\). For excluding C, they are \(14.6-12.0=2.6\) and \(1.20-0.80=0.40\).
The effect estimates range from 3.0 to 14.6 outcome units, compared with 12.0 in the original fit. In this hypothetical example, excluding A produces the largest change in the estimated effect and the largest increase in RMSPE. Excluding B also changes the estimate appreciably, while excluding C changes it less. The pattern merits careful reporting: the post-intervention estimate is not invariant to donor availability, and some exclusions also reduce pre-intervention fit. These summaries alone do not tell us which estimate is correct or quantify uncertainty around any of them.
How to Conduct and Report the Diagnostic
Record the original donor pool, fitted weights, pre-intervention fit, and post-intervention effect under the stated design choices.
Inspect donors with large weights and consider substantively important or plausibly influential donors. State the reason for each exclusion rather than selecting exclusions only because they produce a preferred result.
Remove the specified donor, then solve the synthetic-control weight problem again among the remaining eligible donors. Keep the chosen predictors, pre-intervention window, and other design choices fixed for this sensitivity comparison.
Report the refitted weights, pre-intervention RMSPE, post-intervention effects, and changes from the original results. Interpret differences in the outcome’s units and in light of pre-fit.
Common Mistake: Removing a Donor Without Refitting
A common but incorrect shortcut is to set an influential donor’s original weight to zero and use the remaining original weights as though they formed a new synthetic control. If the donor has a positive original weight, setting its weight to zero without refitting leaves the other weights summing to less than one and does not represent a valid re-estimated synthetic control. For example, changing weights \((0.50,0.30,0.20)\) to \((0,0.30,0.20)\) leaves a sum of 0.50, not 1.00. Renormalizing the remaining weights would make them sum to one, but would still not generally solve the original optimization problem for the reduced donor pool. Re-estimation is required. If the excluded donor’s original weight was already zero, setting it to zero leaves the other weights summing to one; nevertheless, an exclusion sensitivity analysis should still refit the model to check the result under the reduced pool.
A second interpretive mistake is to treat a stable estimate across exclusions as proof of causal identification. Stability is useful evidence that the result does not hinge strongly on the tested donors, but it cannot verify the assumptions that make the synthetic comparison informative. Conversely, sensitivity does not automatically disprove those assumptions; it identifies dependence that should be examined and reported.
Check Your Understanding
Answer each question using the definitions and calculations in this tutorial.
- A synthetic control has weights 0.60, 0.25, and 0.15. Calculate its maximum weight, Herfindahl concentration index, and effective donor count.
- Why must weights generally be re-estimated after removing a donor with a positive original weight?
- An exclusion refit changes the average post-intervention effect substantially but also has much larger pre-intervention RMSPE. What two features should be reported before interpreting the change?
- What does an effective donor count of about 2 mean, and what does it not mean?
- Why does stability of the estimated effect across donor exclusions not, by itself, establish causal identification?