Reporting Staggered Event-Study Results Responsibly

Why Reporting Choices Are Part of the Result

An event-study graph is not a complete description of a staggered-adoption analysis. Its coefficients depend on which treated cohorts contribute at each event time, how their effects are combined, and which units provide untreated comparisons. Those details determine what population an estimate describes. Omitting them can make a changing sequence of estimates look like a single population’s treatment-effect trajectory.

Responsible reporting therefore does more than name an estimator or display a graph. It states the causal quantity being targeted, the comparison group and aggregation rule, the event-time support, the uncertainty procedure, and the assumptions required for a causal interpretation. It also distinguishes those assumptions from evidence that can only probe their plausibility.

Definition: Let $G_i=g$ indicate that unit $i$ first adopts treatment in period $g$. Let $Y_{it}(1)$ and $Y_{it}(0)$ denote its outcome in period $t$ under treatment and no treatment, respectively. For $t\ge g$, the cohort-time average treatment effect on the treated is $$ ATT(g,t)=E\!\left[Y_{it}(1)-Y_{it}(0)\mid G_i=g\right]. $$ The event time is $e=t-g$. An event-time effect combines $ATT(g,g+e)$ across the cohorts for which that effect can be estimated.

The notation describes a target, not a guarantee that the data identify it. A causal interpretation requires assumptions that connect observed outcomes to the unobserved no-treatment outcomes for adopting units. Reporting should keep the target and those assumptions visible.

State the Estimand, Comparison Group, and Weights

A report should say whether it presents cohort-time effects, an event-time aggregation, or both. For an event time $e$, a general weighted profile is

\theta_e=\sum_{g\in C_e}w_{g,e}ATT(g,g+e), \qquad \sum_{g\in C_e}w_{g,e}=1, \qquad w_{g,e}\ge 0.

Here $C_e$ is the set of cohorts that contribute a valid estimate at event time $e$, and $w_{g,e}$ is the weight assigned to cohort $g$. The weights might represent cohort shares in a specified target population, or they might give equal weight to every contributing cohort. Those rules answer different questions. A report should name the rule and explain what population it represents; saying only “the average effect” is not enough.

It should also identify who supplies the untreated comparison outcomes. A never-treated comparison uses units that do not adopt during the study period. A not-yet-treated comparison uses units that have not adopted by the relevant period. The latter comparison group can change over calendar time as additional units adopt. The choice must be stated because it defines the comparison behind the estimates and the corresponding identifying conditions.

Worked Example: Reporting a Weighted Event-Time Effect

Suppose two hypothetical adoption cohorts contribute estimates for the first year after adoption. Cohort A has 100 eligible treated units and an estimated effect of 4.0 additional outcome units, with an estimated standard error of 1.20. Cohort B has 50 eligible treated units and an estimated effect of 1.0 additional outcome unit. Suppose the reported aggregation uses treated-unit shares as weights, and a cluster bootstrap for the aggregate gives a standard error of 0.94 outcome units.

The cohort shares are $100/(100+50)=0.6667$ and $50/(100+50)=0.3333$. The event-time estimate is therefore

\hat{\theta}_1 =0.6667(4.0)+0.3333(1.0) =2.667+0.333 =3.000.

Using a normal-approximation 95% interval and the stipulated aggregate standard error gives

3.000\ \pm\ 1.96(0.94) =3.000\ \pm\ 1.842 =[1.158,\ 4.842].

A responsible report could say: “For the supported cohorts, the treated-unit-share-weighted estimated effect one year after adoption was 3.00 outcome units (95% confidence interval: 1.16 to 4.84), using the stated cluster-bootstrap standard error.” This describes an average for the cohorts and weights specified; it does not imply that every cohort has an effect of 3.00. The interval calculation uses the given aggregate standard error, not an assumption that the cohort estimates are independent.

Report Which Cohorts Support Each Event Time

Event-time support is the set of cohorts with data and a valid comparison at a given value of $e$. It often shrinks at longer horizons: later adopters cannot contribute many post-adoption periods if the data end soon after their adoption. As a result, the weights $w_{g,e}$, and sometimes the represented population, can change across the event-study profile.

A graph should therefore be accompanied by a cohort-by-event-time support table, a clear statement of the included cohorts and weights at selected horizons, or another explicit account of support. If the goal is to compare effects over time for a fixed cohort population, one option is to restrict the profile to cohorts observed at all reported event times. That produces a different target from using every available cohort at each horizon, and may discard substantial information. Whichever target is used, disclose it.

Worked Example: Detecting a Change in Cohort Composition

Consider three hypothetical cohorts with adoption years 2018, 2020, and 2021 and outcome data through 2023. Their treated-unit counts are 40, 30, and 30. Suppose all three cohorts have valid comparisons at event time $e=1$, with estimated effects of 2.0, 4.0, and 1.0 outcome units, respectively. Using cohort-size weights, the first-year profile is

\hat{\theta}_1 =\frac{40}{100}(2.0)+\frac{30}{100}(4.0)+\frac{30}{100}(1.0) =0.8+1.2+0.3 =2.3.

At event time $e=3$, only the 2018 cohort has three post-adoption years within the data period. If its estimated effect is 6.0 outcome units, then the supported estimate is $6.0$, based on that cohort alone. It is not a like-for-like comparison with the first-year estimate of 2.3: the estimates use different cohort compositions and can also differ in event time.

A suitable description is: “The cohort-size-weighted estimate at year one was 2.3 units across all three cohorts. At year three, only the 2018 cohort was supported; its estimate was 6.0 units.” The second estimate alone does not show that the average effect for all three cohorts rose from 2.3 to 6.0.

Make the Comparison and Identifying Assumptions Explicit

The causal claim depends on how the unobserved no-treatment path for the treated cohort is learned from comparison units. For example, a difference-in-differences interpretation requires that, absent adoption, the treated cohort and its specified comparison group would have followed the required parallel-trends condition over the periods used. Conditional versions require that condition within the stated covariate adjustment structure. The report should name the version used rather than say only “parallel trends.”

Other conditions commonly required include no anticipation (future adoption does not affect outcomes before adoption), correct treatment timing and outcome measurement, and the existence of eligible comparison units with adequate overlap for the design. If units can affect one another’s outcomes, the analysis also needs an appropriate no-interference condition or a clearly defined alternative. The precise conditions depend on the estimator and comparison group; the report should not imply that one checklist automatically establishes identification.

These assumptions are not established by a small or statistically insignificant pre-adoption estimate. Pre-period estimates and sensitivity analyses can reveal patterns that challenge a design or show how conclusions respond to departures from assumptions, but they cannot prove the unobserved post-adoption counterfactual trends. State the assumptions as assumptions, and describe diagnostics as evidence about their plausibility.

Worked Example: Writing a Difference-in-Differences Interpretation

In a hypothetical analysis, a treated cohort’s mean outcome rises from 10 to 14 units between the pre-adoption and post-adoption periods. A not-yet-treated comparison group’s mean rises from 8 to 10 units over the same periods. The difference-in-differences estimate is

(14-10)-(10-8)=4-2=2\text{ outcome units}.

This arithmetic says that the treated cohort’s increase exceeded the comparison group’s increase by 2 units. It does not by itself establish a causal effect. If the specified parallel-trends condition holds for the treated cohort and this not-yet-treated comparison group, together with the design’s other requirements such as no anticipation, the estimate can be interpreted as an estimated average treatment effect for the treated cohort in the post period.

A report should give the comparison group and the assumption alongside the estimate: “Relative to units not yet treated in the comparison period, the estimated post-adoption difference-in-differences was 2 outcome units; its causal interpretation requires parallel untreated trends for these groups and no anticipation.” This wording reports the computed contrast without turning the assumption into a fact.

Report Uncertainty and Interpretation in Context

For each reported estimate, give its units, the uncertainty measure, and how that uncertainty was calculated. In staggered designs, repeated observations within units and common shocks within adoption or assignment clusters can make independent-observation standard errors inappropriate. State the clustering level or other inference procedure and ensure it matches the design and estimator. If the method uses a bootstrap, describe the resampling unit; resampling individual rows when treatment is assigned at a higher level may not preserve the relevant dependence.

Label confidence intervals as pointwise if each interval is constructed for one event time separately. A set of pointwise intervals does not generally provide the same coverage guarantee for the entire event-study profile as a simultaneous confidence band. If discussing whether the whole profile differs from zero or whether effects change across horizons, use inference designed for that joint claim, and identify it. A p-value is not the probability that the null hypothesis is true, and statistical uncertainty does not capture bias from violated identifying assumptions.

A concise results table can make the essential distinctions visible:

Report item	What to specify
Estimand	Cohort-time effect or event-time target, including the treated population and outcome units
Comparison	Never-treated or not-yet-treated units, and the periods in which they are eligible
Aggregation	Which cohorts contribute and whether weights are equal, population-based, or otherwise defined
Support	Available cohorts at each event time and any restriction to common support
Uncertainty	Standard errors or intervals, clustering/resampling procedure, and pointwise or simultaneous scope
Identification	Parallel-trends condition, no anticipation, and other estimator-specific assumptions

Common Reporting Mistakes

Calling an estimate “the effect” without naming its target. A weighted profile is an average for specified cohorts and weights, not necessarily a universal effect for all treated units.
Comparing horizons as if the same cohorts contribute. If support changes, say so; do not describe a difference in profile values as a change over time for a fixed population unless the contributing population is held fixed.
Leaving the control group implicit. Identify whether comparison units are never treated or not yet treated, and clarify that eligibility can vary over time.
Treating a pretrend diagnostic as proof. A diagnostic can be informative, but it cannot verify the unobserved post-adoption counterfactual trend.
Reporting a confidence interval without its scope. State whether it is pointwise or simultaneous and how standard errors account for the dependence in the design.

Key takeaway: A staggered event-study result is interpretable only when readers can identify the estimand, comparison group, cohort weights, event-time support, uncertainty procedure, and identifying assumptions. A graph without these details can conceal changes in what is being averaged.

Check Your Understanding

For each question, focus on what a reader would need in order to interpret the reported finding.

An event-time estimate uses equal cohort weights. What population-level distinction should the report clarify?
Why might the estimate at year four after adoption use a different cohort population from the estimate at year one?
What must be stated about the comparison group when estimates use not-yet-treated units?
What does a 95% pointwise interval for one event time not automatically guarantee about the whole profile?
Why does a near-zero pre-adoption estimate not establish parallel trends after adoption?