Worked Analysis of a Staggered Policy Rollout

From Rollout Dates to an Estimand

A staggered rollout creates a sequence of treatment cohorts: some units begin a policy earlier than others, while some may remain untreated. The central analytical task is not merely to fit a regression. It is to define when treatment begins for each unit, identify which units can provide a credible comparison at each date, and state exactly which cohort-time effects the analysis summarizes.

Consider a hypothetical policy introduced across local jurisdictions. Let \(G_i\) be the first period in which jurisdiction \(i\) is treated; set \(G_i=\infty\) for jurisdictions never treated during the observation window. Treatment is absorbing: once treated, a jurisdiction remains treated. Event time is \(k=t-G_i\), so \(k=0\) is the first treated period, \(k<0\) is a pre-treatment period, and \(k>0\) is a later period after adoption.

The target is not automatically one universal “policy effect.” Effects may differ across adoption cohorts and across time since adoption. A group-time estimate keeps that structure visible before any summary is constructed.

Assumptions and Eligible Comparisons

Before computing a difference-in-differences contrast, specify the assumptions that give it a causal interpretation. The following are the core conditions for the simple, unconditional comparison used here.

• Consistent treatment timing and outcomes: treatment status is correctly recorded, and the observed outcome equals the potential outcome under the treatment actually received.
• No anticipation: before its adoption period, a cohort’s outcomes are unaffected by its future treatment. In particular, \(Y_{it}=Y_{it}(0)\) for \(t<G_i\).
• Parallel untreated trends: for cohort \(g\) and an eligible untreated comparison group \(C_{g,t}\), the cohort and comparison group would, absent treatment, have the same average outcome change from the baseline \(g-1\) to period \(t\): \(E[Y_{it}(0)-Y_{i,g-1}(0)\mid G_i=g]=E[Y_{it}(0)-Y_{i,g-1}(0)\mid C_{g,t}=1]\).
• Valid comparison support: an untreated comparison group exists at the relevant dates, with outcomes observed for the periods needed to form the contrast.
• No interference: one jurisdiction’s treatment does not change another jurisdiction’s outcome in a way that invalidates its potential outcomes.

For a comparison at period \(t\), eligible controls are units not yet treated by \(t\) and units never treated. Already-treated units are not untreated controls: their outcomes may include their own treatment effects. For each cohort-time contrast, use the same baseline \(g-1\) for the treated cohort and its comparison group. If treatment is anticipated, the treatment-date definition and baseline must be revised so that the baseline is genuinely unaffected.

Worked Analysis of Cohort-Time Effects

The table gives hypothetical mean outcomes, measured in policy-outcome points, for three equally sized cohorts. The early cohort adopts in period 2, the late cohort adopts in period 3, and the never-treated cohort remains untreated. Periods \(-1\), 0, and 1 are untreated for all three groups.

Why the Difference Identifies an Effect

The identifying logic can be seen by decomposing the treated cohort’s observed change. No anticipation means its period-\(g-1\) outcome is untreated. In period \(t\ge g\), its observed change combines the change it would have experienced without treatment and its treatment effect at \(t\). Parallel untreated trends say the comparison group’s observed change estimates that first component.

Subtract the second line from the first. The parallel-trends assumption makes the untreated-change terms equal, so they cancel. The remaining quantity is \(E[Y_{it}(1)-Y_{it}(0)\mid G_i=g]=ATT(g,t)\). This derivation explains both the identifying assumption and why already-treated units are inappropriate controls: their observed changes need not equal untreated changes.

Event-Time Aggregation and Pretrend Diagnostics

To describe dynamics, map group-time estimates to event time \(k=t-g\), then combine estimates at the same \(k\) using stated weights. The weights determine whose effects the summary represents. For example, if the two observed adoption cohorts are weighted equally, the event-time-zero estimate is the average of their first-period effects. In the hypothetical table, those estimates are 4.5 points for the early cohort and 3 points for the late cohort:

At event time \(k=1\), only the early cohort has an observed post-adoption estimate in this five-period window, so the corresponding estimate is \(\widehat{\theta}_{1}=\widehat{ATT}(2,3)=7\) points. This difference between event-time estimates could reflect changing effects, cohort composition, or both. An event-time summary should make clear which cohorts contribute at each \(k\); changing support can make adjacent estimates represent different populations.

Pre-treatment contrasts can reveal departures from the parallel-trends pattern in observed outcomes. They are diagnostics, not direct observations of the treated group’s counterfactual trend after adoption. In the table, from period \(-1\) to period 0, the early cohort changes by \(50-48=2\) points, and the late and never-treated cohorts each change by 2 points. The early cohort’s placebo difference-in-differences contrast is therefore \(2-(2+2)/2=0\) points. From period 0 to period 1, the changes are again 2 points for each group, yielding another zero contrast. These values show no discrepancy in these particular pre-period changes; they do not prove that post-treatment untreated trends would have remained parallel.

A nonzero pretrend estimate can raise concern about the design, but a noisy estimate may have little power to reveal meaningful differences. Conversely, failure to reject a pretrend test does not validate parallel trends. Use the pretrend pattern alongside institutional knowledge, the treatment-date definition, the eligible comparison groups, and sensitivity analysis for departures from parallel trends.

Clustered Inference and Interpretation

When policy assignment occurs at a jurisdiction or other group level, outcomes within an assignment cluster can share shocks and be serially correlated over time. Treating every unit-period observation as independent can understate uncertainty. Inference should account for dependence at the level at which treatment is assigned, using a cluster-robust variance estimator or an appropriate method for the number and structure of clusters. The standard error quantifies sampling uncertainty under the chosen estimator and inference procedure; it does not measure uncertainty about whether the identifying assumptions are true.

Common Pitfalls in a Staggered Rollout

• Using already-treated units as untreated controls. Their outcomes may contain treatment effects. At each \(g,t\), restrict comparisons to units untreated through \(t\) or never treated.
• Changing the baseline without changing the estimand. The baseline for cohort \(g\) is \(g-1\) in the contrasts defined here. Use the same baseline for treated and comparison changes.
• Calling a clean pretrend test proof of identification. Pre-period evidence can diagnose visible discrepancies, but cannot reveal the untreated post-period outcomes of treated units.
• Reporting an event-time average without its weights or support. State which cohorts contribute and how their effects are weighted at each event time.
• Ignoring assignment-level dependence. Many observations do not necessarily mean many independent treatment assignments. Cluster uncertainty at an appropriate level.

The disciplined workflow is to define adoption cohorts and anticipation timing, specify the cohort-time estimand, select only valid untreated comparisons, calculate and aggregate effects with explicit weights, examine pre-treatment evidence without overstating it, and report inference that reflects the assignment structure. Each step makes a different part of the causal claim inspectable.