A Worked Comparative Interrupted Time-Series Analysis

Why Compare Interrupted Trajectories?

An intervention may coincide with a change in the treated population’s outcome even when the intervention had no effect. A secular trend, seasonal fluctuation, or wider shock could also change outcomes. A comparison series helps separate changes shared by both series from changes that occur differentially in the treated series. Comparative interrupted time-series (CITS) analysis formalizes this comparison by modeling the treated-control outcome gap before and after intervention.

The analysis is only as credible as its comparison and assumptions. A segmented regression can summarize a trajectory, but its coefficients do not by themselves establish that an intervention caused the observed change. In particular, the pre-intervention gap should be examined to judge whether a linear continuation is a plausible counterfactual.

The Segmented Regression Model

Set \(t=0\) at the intervention period, with negative \(t\) values for pre-intervention periods and nonnegative values for post-intervention periods. Define \(P_t=1\) for \(t\geq 0\) and \(P_t=0\) otherwise, and define \(R_t=\max(0,t)\), the number of periods since intervention. A standard linear segmented model for the gap is:

Before intervention, the model is \(D_t=\beta_0+\beta_1t+\varepsilon_t\): \(\beta_0\) is the extrapolated gap at \(t=0\), and \(\beta_1\) is the pre-intervention linear trend in the gap per period. After intervention, the fitted gap is \((\beta_0+\beta_2)+(\beta_1+\beta_3)t\). Thus \(\beta_2\) is the change in the gap’s level at the intervention, relative to the pre-intervention line, and \(\beta_3\) is the change in its slope, measured in outcome units per period.

At \(k\) periods after intervention, the modeled departure from the extrapolated pre-intervention gap is \(\beta_2+k\beta_3\). This is the estimated differential change at that horizon. It is not necessarily the treated series’ raw change: the comparison series and the pre-existing gap trend both matter.

Worked Example: Fitting the Differential Level and Slope

Why the Coefficients Are Differences Between Lines

The interpretation of the level and slope coefficients follows directly by writing the model in its pre- and post-intervention forms. Before intervention, \(P_t=R_t=0\), so the fitted line is \(\beta_0+\beta_1t\). After intervention, \(P_t=1\) and \(R_t=t\), so it is \((\beta_0+\beta_2)+(\beta_1+\beta_3)t\). Subtracting the pre-intervention line from the post-intervention line at time \(t=k\) gives:

This subtraction explains why the coefficients describe differential changes in the gap, not changes in the treated series alone. It also shows why reporting only \(\beta_2\) can be incomplete: if the slope changes, the estimated departure from the extrapolated pre-intervention gap evolves with time.

The segmented model can also be viewed as fitting one line to the pre-intervention gaps and another to the post-intervention gaps, then expressing the latter line’s intercept and slope as differences from the former. To see this, the pre-period fitted line has intercept \(\beta_0\) and slope \(\beta_1\); the post-period line has intercept \(\beta_0+\beta_2\) and slope \(\beta_1+\beta_3\). With separate, unrestricted lines for the two periods, least squares minimizes the sum of the pre-period squared residuals plus the sum of the post-period squared residuals. The resulting two line fits can therefore be translated into the four coefficients of the segmented model.

Checking Pre-Intervention Fit

A crucial diagnostic is whether a linear trend describes the pre-intervention gap well enough to make its continuation a credible counterfactual. Examine the pre-period observations, the fitted line, and residuals \(e_t=D_t-\hat{D}_t\). Residuals that systematically curve, trend, or cluster can reveal a mismatch that a single summary statistic conceals. The number of pre-intervention periods also matters: a short series may provide little information about trend shape.

A common summary is the pre-intervention root mean squared error (RMSE), the square root of the average squared difference between observed and fitted pre-period gaps. It is expressed in the outcome’s units. RMSE describes the typical magnitude of these in-sample discrepancies; it is not a test of the causal assumptions and does not, by itself, determine whether fit is adequate. Its size must be considered alongside the outcome scale, residual patterns, and the purpose of the analysis.

Assumptions, Inference, and Common Mistakes

A causal interpretation of \(\beta_2+k\beta_3\) requires more than a good-looking regression output. The comparison must represent how the treated series would have evolved relative to its control without the intervention. The pre-intervention gap provides evidence about that relationship but cannot verify that it would persist after intervention. A new shock affecting only the treated series, spillover of the intervention to the comparison, or a change in the way outcomes are measured could invalidate the interpretation.

Time-series errors may also be serially correlated: adjacent periods can have related unexplained deviations. Standard errors based on independent errors may then be misleading. Use an inference procedure appropriate to the series and its dependence structure; do not infer precision from a large number of time points as though they were independent observations. In this tutorial’s examples, the calculations describe fitted contrasts, not statistical significance or uncertainty intervals.

Common mistake: “The treated outcome rose by 7 units at month three, so the intervention effect was 7.” This confuses a modeled change in the treated outcome with a differential change relative to the comparison and extrapolated pre-intervention gap. The corrected interpretation in the fitted example is a 7-unit-per-1,000-person departure of the treated-control gap from its pre-intervention trajectory at month three, conditional on the design and model assumptions.

Also avoid treating an insignificant pre-intervention trend test as proof that the two series would have remained comparable. A failure to detect a trend difference is not evidence that all relevant post-intervention counterfactual assumptions hold. Inspect the actual gap and residuals, state the assumptions, and interpret the estimate in the context of plausible competing explanations.