Summarizing Synthetic Control Treatment Effects

Why Summarize a Synthetic Control Effect?

A synthetic control analysis produces a series of gaps: at each post-intervention time, the treated unit’s observed outcome is compared with the outcome predicted by its synthetic control. A single gap answers a time-specific question. A summary across several post-intervention periods answers a different question, and its value depends on which periods are included. The average describes the mean gap over a stated horizon; the cumulative summary adds the gaps across that horizon.

These summaries are useful because intervention effects may change over time. An effect could grow, fade, reverse, or fluctuate. Reporting only the final-period gap can miss that pattern, while reporting only an average can conceal it. A clear analysis therefore states the time-specific gaps, the summary used, and the post-intervention horizon.

From a Synthetic Outcome to a Time-Specific Gap

Let \(T_0\) denote the final pre-intervention period. For the treated unit, let \(Y_{1t}\) be its observed outcome at time \(t\), and let \(\widehat{Y}_{1t}^{N}\) be the synthetic control estimate of the treated unit’s outcome at that time in the absence of the intervention. The synthetic outcome is formed using the synthetic control constructed for the analysis; its construction is not repeated here.

The gap is an estimate of the causal effect only under the design’s identifying conditions. In particular, the treated unit’s observed outcome after the intervention represents its outcome under treatment, while the synthetic control must provide a credible estimate of its untreated outcome over the post-intervention period. This requires, among other things, that the intervention does not affect the donor units used to form the synthetic control, that there is no relevant anticipation before the intervention, and that the synthetic control tracks the treated unit’s untreated trajectory well enough after the intervention. Without these conditions, the gap remains a treated-versus-synthetic comparison, but its causal interpretation is not justified.

The summaries below combine these period-specific gaps; they do not repair a failure of the identifying conditions. Nor does a calculated average or cumulative gap, by itself, quantify statistical uncertainty.

Average and Cumulative Effects Over a Stated Horizon

Index the first \(H\) post-intervention periods as \(j=1,\ldots,H\), with \(j=1\) corresponding to \(t=T_0+1\). For equally spaced periods, define the average effect estimate as the arithmetic mean of the \(H\) gaps and the cumulative effect estimate as their sum.

The average has the same units as one gap. For example, if the outcome is an annual employment rate measured in percent, each gap is in percentage points and the average is also in percentage points. It is the mean gap across the specified annual periods, not a change per year. The cumulative sum has outcome-units multiplied by periods. For that same outcome, summing annual rate gaps gives percentage-point-years. This is an additive summary over time, not the final-period effect and not the average effect.

For equally spaced periods, the two summaries are linked by a simple identity:

This identity follows directly by multiplying the average, which is the sum of the \(H\) gaps divided by \(H\), by \(H\). It also helps check calculations: the cumulative effect must equal the average multiplied by the number of periods, subject to rounding.

Why the Chosen Horizon Matters

A horizon is part of the definition of an average or cumulative effect, not a cosmetic reporting choice. Adding a post-intervention period changes the average according to how its gap compares with the previous average. If the average over \(H\) periods is \(\hat{\tau}_{\mathrm{avg}}(H)\), then including the next gap gives

Subtracting the earlier average yields

Thus, the new average rises if the added gap is above the previous average, falls if it is below, and stays the same if it equals the previous average. The cumulative sum, by contrast, always changes by exactly the added gap. This is why two studies, or two summaries of one study, can report different average effects simply because they use different post-intervention windows.

Periods of Unequal Length

The ordinary formulas treat each period equally. That is appropriate when periods are equally spaced and the target is the average across periods. If periods have unequal durations, an unweighted average still describes the mean of the period-specific gaps, but it does not generally describe the average gap per unit of calendar time. A time-weighted summary uses each period’s duration.

Let \(d_j\) be the duration represented by period \(j\). If each gap is treated as constant over its period, the duration-weighted cumulative effect is the sum of each gap multiplied by its duration. The time-weighted average is that sum divided by the total duration.

For equally long periods, the duration weights cancel in the average, recovering the ordinary arithmetic mean. The duration-weighted cumulative quantity has outcome-units multiplied by time. The assumption that each period’s gap represents its full duration should be stated; if gaps are observed only at isolated dates, multiplying by durations is an additional approximation rather than an automatic consequence of the data.

Reporting and Common Pitfalls

A useful report makes the outcome, time window, and units explicit. For instance: “Across the four annual post-intervention periods, the mean estimated gap in the employment rate was 2.5 percentage points; the sum of the four annual gaps was 10 percentage-point-years.” If effects vary considerably, show the time-specific gaps alongside the summary so the reader can see whether the average conceals a changing pattern.

• Calling an average a rate of change. The mean of annual gaps in an employment rate is measured in percentage points, not percentage points per year. A rate of change concerns how an outcome changes over time and is a different quantity.
• Giving a cumulative sum the units of a single-period gap. Adding annual rate gaps gives rate units multiplied by years, such as percentage-point-years. Adding annual counts gives the sum across the annual periods, such as trips over the stated horizon.
• Omitting the horizon. An average over two years and an average over five years answer different questions. State which post-intervention periods are included.
• Treating a summary as a causal guarantee. Arithmetic does not establish that the synthetic series is a valid untreated counterfactual. Causal interpretation still depends on the identifying conditions.
• Assuming a cumulative effect is the final-period effect. The cumulative value adds gaps across time; report the last period’s gap separately if that is also of interest.