Difference-in-Differences With Panels and Repeated Cross-Sections

Two Ways to Observe Outcomes Over Time

Difference-in-differences (DiD) compares the change in outcomes for a treated group with the change for a comparison group. The comparison removes changes common to both groups only under an appropriate parallel-trends assumption. The arithmetic is familiar from the two-period design; the data structure matters because it determines which people contribute to each change and which population the estimate describes.

A panel follows the same units in both periods, so each unit has a before-and-after outcome. A repeated cross-section samples units separately in each period: the people observed before need not be the people observed after. Neither structure is inherently preferable. The key questions are whether the samples represent the intended populations over time and whether the identifying assumptions apply to those populations.

Definition: Let \(D_i=1\) mark membership in the group exposed to treatment after the intervention, and \(D_i=0\) mark membership in the untreated comparison group. Let \(t=0\) denote the pre-treatment period and \(t=1\) the post-treatment period. The two-period DiD contrast is the treated group's outcome change minus the comparison group's outcome change. In a panel, changes are calculated for the same units; in repeated cross-sections, they are differences between group-period means.

Panel DiD: Compare Changes for the Same Units

For a panel, write \(Y_{i0}\) and \(Y_{i1}\) for unit \(i\)'s observed outcomes before and after the intervention. If there are \(n_1\) treated units and \(n_0\) comparison units, the sample estimator is

\widehat{\delta}_{\text{panel}} = \frac{1}{n_1}\sum_{i:D_i=1}(Y_{i1}-Y_{i0}) - \frac{1}{n_0}\sum_{i:D_i=0}(Y_{i1}-Y_{i0}).

Because each difference uses two outcomes from the same unit, the panel estimator uses paired changes. It does not require every treated unit to have the same baseline outcome as a comparison unit. Instead, it compares average changes between the groups.

Worked Example: A Balanced Panel of Workers

Suppose a hypothetical wage program begins between two measurement periods. The same three treated workers and three comparison workers are observed in both periods. Wages are measured in dollars per hour, and assume for this calculation that every listed worker remains in the panel.

Group	Worker	Before	After	Change
Treated	A	50	58	8
Treated	B	60	64	4
Treated	C	55	62	7
Comparison	D	48	51	3
Comparison	E	52	54	2
Comparison	F	57	59	2

The treated group's average change is \((8+4+7)/3=19/3=6.333\) dollars per hour. The comparison group's average change is \((3+2+2)/3=7/3=2.333\) dollars per hour. Therefore,

\widehat{\delta}_{\text{panel}} = \frac{19}{3}-\frac{7}{3} = 4.000\ \text{dollars per hour}.

The estimate says that treated workers' mean wage increased by 4.000 dollars per hour more than the comparison workers' mean wage did. It is not, by itself, proof that the program caused that difference. A causal interpretation also requires the assumptions described below.

Repeated Cross-Sections: Compare Four Group-Period Means

With repeated cross-sections, denote the sampled outcome for unit \(i\) in period \(t\) by \(Y_{it}\), but do not assume that the same \(i\)'s appear in both periods. Let \(\bar{Y}_{d,t}\) be the sample mean for group \(d\) in period \(t\). The estimator is

\widehat{\delta}_{\text{cross}} = (\bar{Y}_{1,1}-\bar{Y}_{1,0}) - (\bar{Y}_{0,1}-\bar{Y}_{0,0}).

This calculation needs a sample mean in each of the four group-period cells. It does not require matched records across periods. For the estimates to describe changes in stable group populations, however, the samples in each cell must represent those intended populations, rather than different mixtures of people whose outcomes differ for reasons unrelated to the intervention.

Worked Example: Repeated Cross-Sections of Households

A hypothetical service is introduced in one region. Different households are sampled before and after the introduction. The outcome is monthly spending in dollars. Assume each sample represents the intended region and comparison population for its period, and that there are no treatment spillovers into the comparison region.

Suppose the treated-region mean is 20 dollars before (\(n_{1,0}=100\)) and 28 dollars after (\(n_{1,1}=120\)). The comparison-region mean is 18 dollars before (\(n_{0,0}=80\)) and 22 dollars after (\(n_{0,1}=90\)). The repeated-cross-section calculation is

\widehat{\delta}_{\text{cross}} = (28-20)-(22-18) = 8-4 = 4.000\ \text{dollars per month}.

The treated-region mean rose by 8 dollars per month, while the comparison-region mean rose by 4 dollars per month. Their difference in changes is 4.000 dollars per month. The sample sizes affect the precision of the cell means, but not this arithmetic. As with the panel estimate, a causal interpretation requires a parallel-trends condition and suitable population representation.

Identification and the Parallel-Trends Condition

To state what DiD identifies, let \(Y_{it}(0)\) be the outcome unit \(i\) would have in period \(t\) without treatment. For a treated unit, let \(Y_{i1}(1)\) be its post-period outcome under treatment. Assume treatment is absent in period 0, is received by the treated group in period 1, and does not affect comparison units. The average treatment effect on the treated (ATT) is

\operatorname{ATT} = E[Y_{i1}(1)-Y_{i1}(0)\mid D_i=1].

The central identifying condition is that, without treatment, the treated and comparison groups would have had the same average outcome change. This is a condition about counterfactual trends, not a claim that the groups have equal outcome levels.

E[Y_{i1}(0)-Y_{i0}(0)\mid D_i=1] = E[Y_{i1}(0)-Y_{i0}(0)\mid D_i=0].

The panel identification argument follows by writing the treated group's observed change as its untreated change plus its treatment effect. The comparison group's observed change is its untreated change, because it is untreated in both periods. Subtracting the comparison change from the treated change cancels the common untreated trend:

\begin{aligned} &E[Y_{i1}-Y_{i0}\mid D_i=1] - E[Y_{i1}-Y_{i0}\mid D_i=0] \\ &= E[Y_{i1}(0)-Y_{i0}(0)\mid D_i=1] + \operatorname{ATT} - E[Y_{i1}(0)-Y_{i0}(0)\mid D_i=0] \\ &=\operatorname{ATT}. \end{aligned}

This argument requires consistent treatment and outcome definitions, no anticipation of treatment in the pre-period, and no spillovers that change comparison-group outcomes. In addition, panel observations must represent the target groups. If units drop out, the complete-case panel directly describes those retained; extending its interpretation to the original population requires that retention not select units in a way that invalidates the relevant trends or target.

For repeated cross-sections, the same subtraction identifies the ATT when the four cell means represent the same well-defined group populations over time, and the corresponding population means satisfy parallel trends in the absence of treatment. In this setting, representativeness is not just a matter of drawing a large sample: changes in who belongs to, or is sampled from, a group can change its mean even when no individual's outcome changes in the way attributed to treatment.

When Do the Two Designs Have the Same Estimand?

A population estimand is the quantity a design targets; an estimate is the quantity calculated from a particular sample. Panel and repeated-cross-section DiD have the same population estimand when both refer to the same treated and comparison populations over time, define outcomes and treatment consistently, and apply the same relevant parallel-trends condition. In practice, this often means that each period's cross-section represents the same fixed group populations that the panel follows.

Even in that case, finite-sample estimates need not be numerically equal. The panel estimate averages changes among its observed units; the repeated-cross-section estimate combines four sample means, which may be based on different people. Sampling variation and the particular units observed can make the estimates differ. Equality of estimands is not a claim of identical samples or identical results.

A panel's paired observations also affect sampling variability. Under independent sampling of treated and comparison units, a simple standard-error estimate for a panel DiD uses the sample variance of unit-level changes in each group:

SE(\widehat{\delta}_{\text{panel}}) = \sqrt{ \frac{s^2_{\Delta Y,1}}{n_1} + \frac{s^2_{\Delta Y,0}}{n_0} }.

For repeated cross-sections sampled independently in all four cells, the corresponding basic variance estimate sums the four cell-mean variance estimates:

SE(\widehat{\delta}_{\text{cross}}) = \sqrt{ \frac{s^2_{1,1}}{n_{1,1}} + \frac{s^2_{1,0}}{n_{1,0}} + \frac{s^2_{0,1}}{n_{0,1}} + \frac{s^2_{0,0}}{n_{0,0}} }.

Here \(s^2_{\Delta Y,d}\) is the sample variance of individual changes in group \(d\), and \(s^2_{d,t}\) is the sample variance within group-period cell \((d,t)\). These expressions rely on the stated independent-sampling conditions; designs with repeated sampling, clustering, or more complex sampling require variance calculations that reflect that structure. The panel formula uses the variability of paired changes, not the separate before- and-after variances as if the same units were unrelated samples.

Worked Example: How Changing Composition Can Create a False Contrast

Consider a hypothetical repeated-cross-section study with two types of people, low-outcome and high-outcome. There is no treatment effect. In the treated population, the low type's outcome rises from 10 to 12 and the high type's outcome rises from 20 to 22. Thus each type's untreated outcome increases by exactly 2. Before the intervention, the treated-region sample is 80% low type and 20% high type; afterward, it is 20% low type and 80% high type.

The treated-region pre-period mean is \(0.8(10)+0.2(20)=12\). Its post-period mean is \(0.2(12)+0.8(22)=20\), so the observed change is \(20-12=8\). Suppose the comparison population has an unchanged 50%/50% composition, with its two types following the same respective outcome changes. Its mean rises from \(0.5(10)+0.5(20)=15\) to \(0.5(12)+0.5(22)=17\), a change of 2. The resulting contrast is \(8-2=6\), despite the absence of any treatment effect.

The extra 6 comes from the treated-region composition shift, not from a treatment-induced change in either type's outcome. This example shows why the repeated-cross-section means must refer to stable target populations, or why composition changes must otherwise be addressed before interpreting a raw contrast. A panel that followed a fixed set of people would not show this particular change in mixture, though panel attrition could create an analogous problem if the people retained changed selectively over time.

Common Pitfall: Treating the Data Structure as a Minor Detail

Wrong reasoning: “Both estimates use a before mean and an after mean, so panel and repeated-cross-section DiD are automatically the same.” Correction: They coincide as estimands only when the target populations and assumptions align. Different samples or changing composition can alter the repeated-cross-section contrast.
Wrong reasoning: “A panel controls for every possible difference between groups because it observes the same people twice.” Correction: Pairing removes fixed individual outcome levels from the change, but it does not remove group-specific changes in untreated outcomes. Parallel trends is still needed.
Wrong reasoning: “A larger repeated cross-section solves composition bias.” Correction: A larger sample can estimate each changing-composition mean more precisely; it does not make those means represent a stable population.
Wrong reasoning: “The computed DiD is the treatment effect.” Correction: It is an observed difference in changes. It identifies an ATT only under the design, sampling, and counterfactual-trend assumptions.

Key takeaway: Panel DiD compares paired unit-level changes; repeated-cross-section DiD compares changes in four group-period means. Either can identify an ATT under parallel trends, but their target populations align only when the group composition and population representation are appropriate.

Check Your Understanding

Use the distinction between paired changes, cell means, and target populations to answer these questions.

In a panel, the treated group's mean change is 5 units and the comparison group's mean change is 2 units. What is the DiD estimate, and what does it describe?
Why does repeated-cross-section DiD not require observing the same people in both periods? What population-representation condition remains important?
State the parallel-trends condition in terms of untreated potential outcomes. Does it require equal pre-period outcome levels?
Give one reason a panel estimate based only on complete cases might not describe the original study population.
Can panel and repeated-cross-section estimates differ in a finite sample even when they target the same population ATT? Explain briefly.