Why the Choice Is a Design Decision
Synthetic control and comparative interrupted time series (CITS) both use outcomes observed before and after an intervention to estimate what would have happened without it. They organize the comparison differently. Synthetic control constructs a weighted combination of untreated units to resemble the treated unit before intervention. CITS compares changes in the treated series with changes in one or more control series, often allowing for trends and intervention-related changes in level or slope.
Neither method is automatically preferable, and neither becomes credible simply because it produces a precise estimate or a close-looking graph. The decision depends on the available donor units, the quality and behavior of the control trajectories, the risk of extrapolation, and the effect summary that answers the scientific question. The central design question is: which untreated comparison most plausibly represents the treated unit’s counterfactual trajectory?
What Each Design Needs
For synthetic control, the basic construction uses weights \(w_j\) on untreated donor units \(j=1,\ldots,J\), with \(w_j\geq 0\) and \(\sum_j w_j=1\). The synthetic outcome at time \(t\) is \(\sum_j w_jY_{jt}\). The weights are selected to make the synthetic unit resemble the treated unit on pre-intervention outcomes and, where used, relevant predictors. A credible interpretation requires that the donor units are not affected by the intervention, that there is no consequential spillover to them, and that a suitable weighted combination can approximate the treated unit’s untreated trajectory after intervention. Good pre-intervention fit is evidence about resemblance, not proof that the last condition holds.
CITS instead uses comparison trajectories to estimate how the treated-control contrast would have changed absent intervention. Its key identifying requirement is that the pre-intervention relationship between the treated and control series would have continued in the absence of intervention, in the form specified by the analysis. This may allow a nonzero differential trend, but it requires that no other event at the intervention time creates a change in that contrast. Comparison series should be untreated and not materially affected by spillovers. Multiple controls can help characterize the comparison trend, but their number does not by itself guarantee a valid counterfactual.
These requirements suggest five practical checks. First, assess donor support: do the donors span the treated unit’s relevant pre-intervention outcomes and predictors without requiring negative weights or extreme extrapolation? Second, assess pre-intervention fit over the full period, not just on a few selected predictors. Third, examine control-trajectory behavior: does the CITS contrast behave in a way that supports its proposed continuation? Fourth, compare extrapolation risk, including sensitivity to influential donors or modeling choices. Finally, decide which effect summary is needed: a time-specific gap or post-period average may be natural for synthetic control, while a change in level and/or slope may be central to CITS.
Worked Example: A Donor Pool Supports Synthetic Control
Worked Example: A Donor Pool Supports Synthetic Control
Suppose a city introduces a transit policy after year 3. The outcome is annual transit use in thousands of trips per 1,000 residents. Two unaffected donor cities have the following pre-intervention outcomes:
| Year | Treated city | Donor A | Donor B |
|---|---|---|---|
| 1 | 10 | 9 | 11 |
| 2 | 11 | 10 | 12 |
| 3 | 12 | 11 | 13 |
The treated outcomes lie between the two donor outcomes in every pre-intervention year. With weights \(w_A=0.5\) and \(w_B=0.5\), the synthetic outcomes are:
Thus, the pre-intervention discrepancy is zero in each year, and the weights are nonnegative and sum to one. This is strong evidence of pre-period fit and no extrapolation beyond the two donors’ convex combinations on these outcomes. It does not establish that the donors remain valid counterfactuals after year 3: the analyst must still consider spillovers, other shocks, and whether the weighted combination plausibly tracks the treated city without the policy.
Suppose the post-intervention treated outcomes are 16, 18, and 20 in years 4–6, while the synthetic outcomes are 13, 14, and 15. The estimated gaps are \(16-13=3\), \(18-14=4\), and \(20-15=5\) thousand trips per 1,000 residents. Their average is:
This summary is useful if the target is the average post-intervention gap over these three years. It should not be interpreted as an effect established by the exact pre-period match alone; the causal interpretation still depends on the design assumptions.
Worked Example: A CITS Summary Fits the Question
Worked Example: A CITS Summary Fits the Question
A health program begins after period 2. The outcome is clinic visits per 1,000 residents. A comparison series is available, and the analyst’s specified CITS model assumes that, absent the program, the treated-control difference would continue its pre-period linear trend. The series are:
| Period | Control | Treated | Treated minus control |
|---|---|---|---|
| 0 | 10 | 20 | 10 |
| 1 | 11 | 22 | 11 |
| 2 | 12 | 24 | 12 |
| 3 | 13 | 31 | 18 |
| 4 | 14 | 34 | 20 |
| 5 | 15 | 37 | 22 |
The pre-intervention differences increase by one visit per 1,000 residents per period. A segmented model for the difference \(D_t=Y_{1t}-Y_{0t}\), with intervention beginning at \(t=3\), is:
Here \(\beta_2\) is the immediate change in the difference at intervention, and \(\beta_3\) is the change in its per-period slope. In this hypothetical data, the pre-period line is \(D_t=10+t\). The post-period differences are 18, 20, and 22, so the fitted line from the listed values is \(D_t=13+2(t-3)\) for \(t\geq3\). Therefore \(\beta_0=10\), \(\beta_1=1\), \(\beta_2=5\), and \(\beta_3=1\). The model attributes an immediate increase of 5 visits per 1,000 residents and a further increase of 1 visit per 1,000 residents per period in the differential slope, relative to the specified no-intervention trend.
For example, the no-intervention difference at period 5 is the extrapolated pre-period value \(10+5=15\), whereas the observed difference is 22. The estimated period-5 effect is \(22-15=7\) visits per 1,000 residents. This CITS summary is attractive when the scientific question concerns an intervention-related level and slope change and the control series supports the assumed continuation. With only three pre-period observations, however, the linear pattern is weak evidence about long-run trend behavior; the model’s structural assumptions and the possibility of coincident events require substantive scrutiny.
Worked Example: A Good Fit Can Still Be the Wrong Basis
Worked Example: A Good Fit Can Still Be the Wrong Basis
Suppose a treated region’s pre-intervention outcome is near 50, while every eligible donor region has outcomes between 20 and 30 throughout the pre-period. A regression-based synthetic method can potentially use extrapolation to predict the treated region’s level, but a nonnegative weighted average of these donors cannot exceed 30. For any weights \(w_j\geq0\) summing to one:
The treated unit is outside the donor pool’s convex support. A standard convex synthetic control cannot match its level, and a method that extrapolates beyond the donors relies more heavily on modeling assumptions. If the treated and control series nevertheless have a stable, substantively plausible differential trend before intervention, CITS may be more defensible for the intended change-based contrast. But if that differential trend is unstable or if a separate shock coincides with the intervention, CITS is not rescued by the lack of donor support. The appropriate conclusion may be that neither design is credible with the available comparison data.
This is a design choice, not a competition to find the method with the smallest in-sample error. A visibly poor synthetic fit warns against interpreting its post-period gap. A CITS model that fits a short pre-period perfectly may still extrapolate an unsupported trend. In either case, the analyst should report the limitation rather than treat modeling flexibility as a substitute for comparison support.
A Structured Selection Process
Decide whether the main target is the post-period treated-versus-counterfactual gap, an average over selected post periods, or a change in level and slope relative to a comparison trend. Specify the outcome scale and time window.
For synthetic control, inspect the donor range, pre-period outcome fit, and weight concentration. For CITS, inspect the pre-intervention treated-control relationship and whether its trend form is plausible over the relevant period.
Ask whether the proposed controls are untreated, free from important spillovers, and exposed to comparable shocks. Examine intervention timing, other concurrent events, and pre-period behavior that could contradict the required counterfactual continuation.
Consider whether the estimate is driven by donors far from the treated unit, a few highly influential units, or an assumed trend extended beyond observed support. Check whether reasonable design choices materially change the substantive conclusion.
Prefer the design whose assumptions are most defensible for the data and target. If neither comparison construction is credible, state that the available data do not support a reliable causal estimate.
Why the Distinction Matters
Synthetic control and CITS can yield numerically similar estimates while relying on different sources of identifying information. Synthetic control’s post-period gap depends on whether the selected donor combination would have tracked the treated unit absent treatment. CITS depends on whether the specified treated-control relationship would have continued absent treatment. Thus, a set of donors that averages to a close pre-period match does not automatically validate a CITS trend assumption; likewise, a plausible CITS trend does not guarantee that a convex donor combination can reproduce the treated unit’s level and trajectory.
The intended effect summary also matters. If effects may vary over time, a sequence of synthetic-control gaps preserves that pattern, while an average compresses it into one number. A CITS level-change coefficient is not the same as an average post-period effect when the intervention also changes the slope. The interpretation must follow the model and the chosen time horizon, not merely the label of the method.
Check Your Understanding
Use the design assumptions and decision criteria above to answer the following.
- Why does a nonnegative synthetic-control weight restriction make donor support relevant to pre-period fit?
- What counterfactual relationship must a CITS analysis justify continuing after intervention?
- A synthetic control has excellent fit but places nearly all weight on one donor. What additional concerns should be examined before interpreting the post-period gap?
- In a CITS model with an intervention-related slope change, why might the level-change coefficient differ from the average effect over several post periods?
- Describe a situation in which neither synthetic control nor CITS is credible, even if one method produces a precise numerical estimate.