Control-Series Diagnostics for Comparative Interrupted Time Series

Why Diagnose the Control Series?

A comparative interrupted time series (CITS) uses the trajectory of one or more comparison units to help represent how the treated unit’s outcome might have changed without the intervention. A control series is useful only to the extent that it tracks changes relevant to the treated series. Having several available controls does not guarantee that they are suitable: a control may have a different pre-intervention trajectory, respond differently to shared shocks, or dominate a combined comparison.

Control-series diagnostics examine these concerns from several angles. Pre-intervention trajectory plots make differences over time visible. Placebo intervention dates ask whether similar-looking changes appear before the real intervention. Outcome relationships assess whether controls move with the treated outcome, and leave-one-control-out checks show whether conclusions depend heavily on one comparison unit. These checks are complementary; none alone establishes that the comparison identifies a causal effect.

The diagnostic target is often the gap \(D_t\), not just the separate paths. If treated and control outcomes rise together while their difference remains reasonably stable before intervention, that is consistent with a useful comparison. If the gap drifts substantially or changes abruptly well before intervention, a post-intervention gap change is harder to attribute to the intervention.

This interpretation depends on the substantive design as well as the plots. The controls should not themselves receive the intervention or be affected by spillovers from it; outcome definitions and measurement should be comparable over time; and no other event should cause a differential change in the treated unit at the intervention. In particular, pre-intervention similarity is diagnostic evidence, not proof that the untreated treated-unit trajectory would have followed the control series after intervention.

Pre-Intervention Trajectories and the Gap

Start by plotting the treated outcome and each candidate control over time, marking the intervention date. Inspect the pre-intervention period before focusing on post-intervention differences. Look for persistent divergence, abrupt pre-intervention changes, different responses to known events, and controls whose scales or patterns make them unsuitable for a meaningful comparison. Plotting the gap \(D_t\) alongside the separate series can make gradual differential trends easier to see.

A simple way to describe a linear pre-intervention trajectory is with a slope. If the treated series has slope \(b_1\), and the controls have slopes \(b_j\), then the combined control slope is \(\sum_j a_jb_j\). The slope of the gap is therefore the treated slope minus the weighted control slope. This follows by subtracting the two linear trends:

Thus, a gap slope near zero describes little linear differential trend over the observed pre-intervention period. It does not show that every possible post-intervention shock would affect treated and control units equally. A slope summary can also miss nonlinear patterns, short-lived deviations, or a change in trend near the end of the pre-period.

Visual assessment matters because similar endpoint slopes do not guarantee similar trajectories at every time. For example, a control could match the treated outcome early in the pre-period and diverge near the intervention date. Conversely, a small estimated slope difference in a short or noisy series may be uncertain. Interpret plots in light of the number and spacing of observations, the outcome’s scale, and substantive knowledge about events that occurred during the period.

Placebo Intervention Dates

A placebo-date diagnostic assigns a false intervention date within the actual pre-intervention period and checks whether the treated-control contrast appears to change there. Since the real intervention has not yet occurred, a large apparent break at a placebo date can reveal instability in the gap, an unmodeled event, or a procedure that tends to find breaks even in the absence of treatment.

For a chosen date \(\tau\), a segmented description of the gap is

where \(I(t\geq\tau)\) equals 1 at and after \(\tau\), and 0 before it. In this description, \(\beta_2\) represents a level change at \(\tau\), while \(\beta_3\) represents a change in slope. For a placebo check, fit the model using observations that all precede the real intervention. The date must leave enough observations on both sides to make the comparison meaningful. A placebo is a diagnostic, not another estimate of the intervention’s effect.

It can be informative to examine more than one plausible placebo date, but searching many dates and highlighting only the most striking one exaggerates the evidence of instability. Choose dates using a prespecified rule where possible, report the range examined, and interpret the results in light of serial correlation and the available pre-period. A small placebo contrast does not guarantee that the actual post-intervention comparison is unbiased.

Outcome Relationships and Common Movement

A relevant control often moves with the treated outcome during the pre-intervention period, particularly in response to common seasonal patterns or broader shocks. Scatterplots, correlations, and comparisons of changes over time can describe this relationship. These summaries should be treated as evidence about tracking, not as a selection rule that turns the most highly correlated series into a valid control automatically.

Correlation is sensitive to trends and can be high even when short-term movements do not align. It also does not establish that a control is unaffected by the intervention or that treated and control units would have had comparable post-intervention changes absent treatment. Examine the plot and substantive relationship alongside any numerical summary, and consider whether the association persists after accounting descriptively for time patterns.

Excluding Controls One at a Time

A combined control can conceal dependence on one unit. A leave-one-control-out diagnostic removes each control in turn and recalculates the comparison using the same general design and modeling choices. Compare the resulting pre-intervention fit, gap behavior, and estimated contrast. If excluding one unit substantially changes the result, that unit has considerable influence, and the analysis should explain why it remains included or report the sensitivity clearly.

For weighted controls, specify how weights are handled. One transparent check holds the original weights for the remaining controls and renormalizes them to sum to 1; another refits the weights under the same prespecified rule. These answer different questions. A refit can change both control composition and fit, so report which procedure was used. Do not choose the version that yields the preferred estimate.

Putting the Diagnostics Together

A useful diagnostic sequence is to assess each control before relying on a combined series. Keep the checks tied to the design question: whether the controls track the treated unit in ways that make the chosen comparison credible. Document deviations rather than treating a single summary measure as a pass-or-fail test.

Taken together, these diagnostics can expose obvious weaknesses and clarify how strongly conclusions depend on design choices. They cannot observe the treated unit’s untreated post-intervention outcome. A credible causal interpretation still requires the identifying assumptions of the design, including that the comparison captures the relevant untreated trajectory and that no coincident differential event explains the observed change.

Common Mistakes

• Treating a good pre-period plot as proof. Similar observed histories support the comparison but cannot guarantee similar untreated trajectories after intervention. State the evidence and the remaining assumption separately.
• Choosing controls by correlation alone. Correlation can reflect common trends, and it says nothing by itself about spillovers or future differential shocks. Use plots and substantive knowledge as well.
• Calling every placebo a formal test. A false-date break is a warning about instability, not an estimate of the treatment effect. Avoid scanning many dates and reporting only the largest break.
• Ignoring influential controls. A combined result may look stable until a single unit is removed. Report leave-one-out changes and explain the control’s role.
• Changing several design choices at once. If exclusions, weights, and model specifications all change together, it becomes difficult to identify the source of sensitivity. Vary one feature at a time where possible.