End-to-End Repeated-Period DiD Analysis Workshop

From a Repeated-Period Dataset to a DiD Estimate

A difference-in-differences (DiD) analysis is not just a regression command. It is a chain of decisions: define which treated and control units and periods contribute to the contrast, specify the outcome scale, inspect whether the groups’ pre-intervention trajectories are plausible, and calculate uncertainty at the level where treatment was assigned. Each decision affects what the final number means.

This workshop considers a common intervention date: one group is exposed from a particular period onward, while a comparison group remains untreated throughout. There may be several periods before and after that date. The goal is to estimate the treated group’s average change relative to the control group’s change, then judge how far the data and design support a causal interpretation.

Definition: Let $Y_{it}$ be the outcome for unit $i$ in period $t$. Let $T_i=1$ for units in the treated group and $T_i=0$ for control units; let $Post_t=1$ for post-intervention periods and $Post_t=0$ for pre-intervention periods. The pooled repeated-period DiD contrast is $$ \widehat{\tau}_{DiD} = (\bar{Y}_{T,post}-\bar{Y}_{T,pre}) - (\bar{Y}_{C,post}-\bar{Y}_{C,pre}), $$ where each group-period mean averages the included observations in that cell. It is the treated group’s pre-to-post change minus the control group’s pre-to-post change.

The contrast is a descriptive comparison unless the design supports a causal interpretation. For that interpretation, the key identifying condition is parallel trends: in the absence of treatment, the treated and control groups would have had the same average outcome change over the relevant pre-to-post periods. The intervention must not affect outcomes before its onset, and the comparison must not be contaminated by spillovers from treatment. The outcome and group definitions should also be comparable over time. These conditions are substantive claims about the design; a regression cannot establish them by itself.

Worked Example: Defining the Contrast Before Fitting a Model

Worked Example: Calculating a Pooled DiD Contrast

Suppose a hypothetical policy begins between period 2 and period 3. The outcome is emergency visits per 1,000 residents. The table gives the mean outcome for each group in each period.

Period	Treated group	Control group	Status
1	50	48	Pre
2	52	49	Pre
3	57	51	Post
4	60	53	Post

First average each group across its pre-periods and post-periods. The treated-group pre mean is $(50+52)/2=51$ visits per 1,000 residents, and its post mean is $(57+60)/2=58.5$. The control-group pre mean is $(48+49)/2=48.5$, and its post mean is $(51+53)/2=52$.

The treated group’s change is $58.5-51=7.5$ visits per 1,000 residents. The control group’s change is $52-48.5=3.5$. Therefore,

\widehat{\tau}_{DiD} =(58.5-51)-(52-48.5) =7.5-3.5 =4.0

The estimated relative increase is 4.0 emergency visits per 1,000 residents over the pooled post-versus-pre comparison. This is the observed contrast, not yet a justified causal effect. In particular, the treated-control gap is 2 visits per 1,000 in period 1 and 3 in period 2. That widening pre-intervention gap is a reason to scrutinize the parallel-trends assumption rather than to assume it holds.

Representing the Contrast With Fixed Effects

For a balanced panel with units observed in each period, a common repeated-period model is

Y_{it} = \alpha_i+\lambda_t+\beta(T_i \times Post_t)+\varepsilon_{it}.

Here $\alpha_i$ is a unit fixed effect, absorbing stable differences between units; $\lambda_t$ is a period fixed effect, absorbing shocks common to both groups in each period; and $\beta$ is the pooled treated-group post-intervention contrast. The separate terms $T_i$ and $Post_t$ are absorbed by unit and period fixed effects, respectively. In an unbalanced or differently weighted analysis, the regression coefficient need not equal the simple cell-mean contrast, so the weighting and sample should be made explicit.

The model summarizes the post-intervention treated-control gap relative to the pre-intervention gap. Its causal interpretation still requires the design assumptions. In particular, a period effect controls shocks common to groups; it does not control a shock that affects only treated units at the intervention date. If effects vary substantially across post periods, a single $\beta$ is a pooled summary under the model’s weighting, not a claim that the effect was identical in every period.

Worked Example: Connecting Cell Means and the Regression

Use the four-period group means in the previous example, and suppose every unit is observed in every period with equal group-period weights. The average treated-control gap in the pre-periods is $[(50-48)+(52-49)]/2=(2+3)/2=2.5$ visits per 1,000 residents. The average gap in the post-periods is $[(57-51)+(60-53)]/2=(6+7)/2=6.5$.

The difference between those average gaps is $6.5-2.5=4.0$, matching the pooled contrast. In this balanced, equally weighted case, the coefficient on $T_i \times Post_t$ in the fixed-effects model is 4.0 visits per 1,000 residents. The period fixed effects allow the overall outcome level to differ in each period, while the interaction captures the additional post-period change in the treated group relative to the control group.

This numerical match does not repair the pre-period pattern: the treated-control gap increased from 2 to 3 before treatment. Nor does it show that the model’s assumptions are true. It shows how the chosen contrast is represented by the regression in this balanced example.

Check Trends and Composition Before Interpreting

Plotting group means by period is a useful first diagnostic. The pre-intervention portions of the trajectories should be examined for direction, magnitude, and uncertainty, not merely for whether they look visually parallel. A pre-trend test can be imprecise, and failure to reject a difference is not proof that trends are sufficiently alike. Conversely, an apparent difference should prompt investigation of its size, uncertainty, plausibility, and consequences for the design.

For the hypothetical data above, treated outcomes increase by 2 between the two pre-periods, while control outcomes increase by 1. The observed differential pre-period change is therefore $2-1=1$ visit per 1,000 residents. This is a diagnostic discrepancy, not a direct measurement of what the treated group’s untreated post-intervention trend would have been. The observed pre-periods cannot prove parallel trends after the intervention.

Also check whether the population represented by each group-period mean changes. In a panel, inspect whether the same units remain observed and whether attrition differs by group or period. In repeated cross-sections, compare the composition of sampled individuals over time. Changes in age, risk, location, or other outcome-related characteristics can alter group means even without a change in the outcome process for comparable units. Unit fixed effects do not automatically solve changing membership or differential attrition.

Worked Example: Detecting a Composition Warning

Suppose the hypothetical treated and control groups are districts, and the outcome is the same emergency-visit rate. A covariate summary reports the following average age among residents represented in each period.

Group	Pre-intervention average age	Post-intervention average age	Change
Treated	40 years	44 years	4 years
Control	41 years	42 years	1 year

The differential compositional change is $(44-40)-(42-41)=4-1=3$ years. Thus, the groups’ observed populations shifted differently. If age is related to emergency visits, the raw DiD contrast may partly reflect this change rather than a change attributable to the policy.

This check does not establish the direction or size of bias: that would require information about the relationship between age and the outcome and about the sampling or panel process. It does identify a concrete threat to comparability. The analyst should report the composition change, investigate its source, and consider a prespecified adjusted or comparable-population analysis as a sensitivity check. Adjustment itself relies on assumptions and does not guarantee removal of bias.

Cluster the Uncertainty at the Assignment Level

Repeated observations from the same unit, and outcomes from units sharing a treatment-assignment or policy environment, may be dependent. Treating all unit-period records as independent can make the standard error too small. Cluster-robust inference allows general dependence within a cluster while relying on independence across clusters. The cluster level should reflect the assignment or dependence structure, not simply the number of rows in the data. With few clusters, conventional cluster-robust approximations can be unreliable, so the number of clusters and the method used for inference need to be stated.

For a regression with coefficient estimate $\hat{\beta}$, a cluster-robust standard error $SE(\hat{\beta})$ is obtained from a sandwich variance estimator that aggregates score contributions within clusters. In a simple large-cluster approximation, the test statistic for $H_0:\beta=0$ is $\hat{\beta}/SE(\hat{\beta})$. With finite cluster degrees of freedom, a $t$ reference distribution is commonly used; the chosen degrees of freedom and inference procedure should be reported. Clustering changes the uncertainty calculation, not the point estimate or the identifying assumptions.

Worked Example: Reporting Clustered Uncertainty

Suppose the fixed-effects regression for a hypothetical study assigns treatment at the district level and includes 24 districts. Its estimate is $\hat{\beta}=4.0$ emergency visits per 1,000 residents, with district-clustered $SE(\hat{\beta})=1.6$. For illustration, use a $t$ reference distribution with $df=23$, and test $H_0:\beta=0$ against a two-sided alternative.

The test statistic is

t=\frac{\hat{\beta}-0}{SE(\hat{\beta})} =\frac{4.0}{1.6} =2.50.

With 23 degrees of freedom, the two-sided p-value is approximately 0.020. At the 5% significance level, this statistic is inconsistent with a zero coefficient under the model and reference distribution. It does not give the probability that the null hypothesis is true, and it does not establish that the estimate is causal.

Using the 95% critical value $t_{0.975,23}\approx2.069$, the confidence interval is

4.0 \mathbin{\pm} 2.069(1.6) = 4.0 \mathbin{\pm} 3.310 = [0.690,\ 7.31].

Under the regression and inference assumptions, the interval procedure would cover the corresponding model parameter in 95% of repeated samples. In this example, the interval describes effects from about 0.69 to 7.31 visits per 1,000 residents. It reflects sampling uncertainty as calculated with district clustering; it does not account automatically for bias from a failed parallel-trends assumption or changing composition.

A Practical Analysis Sequence

Define the comparison.
Specify treated and control units, the common intervention date, included periods, outcome scale, and whether the data are a panel or repeated cross-sections.

Calculate and label the contrast.
Show the treated and control pre-to-post changes and their difference. State its units and the population represented by the cell means.

Fit the stated model.
Use unit and period fixed effects for a common-date repeated-period design, describe the interaction coefficient and sample, and make any weighting choices transparent.

Assess design diagnostics.
Plot and quantify pre-period changes; examine membership, attrition, and covariate composition. Treat these as evidence about plausibility, not proof of identification.

Report inference and limits.
Cluster at the appropriate level, state the number of clusters and inference method, and distinguish the estimated association from its conditional causal interpretation.

Common Mistakes and the Bottom Line

Calling the interaction causal by itself. A regression coefficient has a causal interpretation only under assumptions such as parallel untreated trends, no anticipation, and no relevant spillovers.
Using a pre-trend test as a pass/fail certificate. A nonsignificant test can reflect low power; it does not establish parallel trends. A significant or visibly important discrepancy deserves substantive attention.
Ignoring changing composition. Fixed effects do not make changing group membership harmless. Report who is represented in each period and how the analysis addresses meaningful shifts.
Clustering by observation rather than design. If treatment is assigned to districts, hundreds of individual records do not create hundreds of independent treatment assignments.
Overstating the confidence interval. The interval quantifies uncertainty under the fitted model and inference procedure; it does not include every source of design bias.

A defensible repeated-period DiD report connects the arithmetic of the contrast to the regression specification, the diagnostics to the identifying assumptions, and the standard error to the assignment structure. For the hypothetical example, the estimate is 4.0 additional emergency visits per 1,000 residents, with a district-clustered 95% interval of 0.690 to 7.31. The widening pre-period gap and differential age composition make a causal reading cautious: they are reasons to investigate and qualify the result, not details to hide behind the reported p-value.

Check Your Understanding

Use the definitions and reasoning in this tutorial to answer the following questions.

A treated group changes from 30 to 38 units of an outcome, while a control group changes from 27 to 32. Calculate the pooled DiD contrast and state its units if the outcome is measured in points.
In a unit- and period-fixed-effects DiD model, what does the coefficient on the treated-group-by-post-period interaction summarize?
Why does a nonsignificant pre-trend test not demonstrate that parallel trends holds?
Suppose the same units are not observed in every period. Name two composition or observation-process features worth checking.
If treatment is assigned to regions, why might a standard error that treats every person-period record as independent be misleading?
What does a cluster-robust confidence interval address, and what important source of uncertainty or bias does it not automatically address?