Imputation Estimation With Untreated Outcomes

Why Impute Untreated Outcomes?

In a staggered-adoption design, a unit’s outcome after treatment combines its untreated potential outcome with the effect of treatment. The untreated outcome is not observed for that unit at that time, so estimating a treatment effect requires constructing a counterfactual. Imputation estimation does this in two stages: fit a model for untreated outcomes using observations that are untreated, then compare treated units’ observed outcomes with their model-predicted untreated outcomes.

The approach makes the counterfactual construction explicit. It also makes its main vulnerability visible: if the model fitted to untreated outcomes does not adequately describe treated units’ untreated outcomes, prediction residuals need not be causal effects.

Definition: Let \(G_i\) denote unit \(i\)’s first treatment period, with \(G_i=\infty\) for a never-treated unit. Let \(Y_{it}(0)\) be its untreated potential outcome in period \(t\), and \(Y_{it}\) its observed outcome. For a treated cohort \(g\) and a period \(t\geq g\), the cohort-time average treatment effect is \(ATT(g,t)=E[Y_{it}(1)-Y_{it}(0)\mid G_i=g]\). An imputation estimator predicts \(Y_{it}(0)\) for treated observations from a model fitted using untreated observations, then averages the differences between observed and predicted outcomes within each cohort-time cell.

Fitting the Untreated-Outcome Model

A common imputation estimator specifies an additive model for untreated potential outcomes:

Y_{it}(0)=\alpha_i+\lambda_t+\varepsilon_{it}.

Here \(\alpha_i\) is a unit effect, \(\lambda_t\) is a calendar-time effect, and \(\varepsilon_{it}\) is an untreated-outcome disturbance. The unit effect allows units to have different baseline levels. The time effect captures shocks common across units in a period. The model does not require treatment effects to be constant: the effect may vary by cohort and calendar time.

The model is estimated using only observations that are untreated. Under no anticipation, a unit is untreated in periods before its adoption date; never-treated units are untreated in all observed periods. Define the untreated-observation set as \(\Omega_0=\{(i,t):t<G_i\}\), including all observed periods for never-treated units. Estimate the fixed effects by least squares over this set:

(\hat{\alpha},\hat{\lambda}) \in \arg\min_{\alpha,\lambda} \sum_{(i,t)\in\Omega_0} \bigl(Y_{it}-\alpha_i-\lambda_t\bigr)^2.

The individual fixed effects and time fixed effects are not separately unique until a normalization is chosen, such as setting one \(\lambda_t\) to zero. That normalization does not affect fitted values \(\hat{\alpha}_i+\hat{\lambda}_t\), which are what the estimator uses. A treated unit’s pre-treatment observations can contribute to estimating its \(\alpha_i\); its treated outcomes must not contribute to the fit.

Assumptions for the basic imputation interpretation: (1) No anticipation: before \(G_i\), the observed outcome equals \(Y_{it}(0)\). (2) Correct untreated-outcome model: the additive unit-and-time model describes the conditional mean of untreated outcomes for the units and periods being analyzed, with untreated disturbances having mean zero for the relevant cohort-time comparisons. (3) Support: the untreated observations provide enough information to estimate the relevant unit and time effects, and to connect them in the fixed-effects model. These conditions make treated units’ predicted untreated outcomes informative; they are not consequences of fitting the regression.

The additive model implies a form of parallel trends. If the disturbance has mean zero conditional on cohort, then for a unit in cohort \(g\), the expected change in its untreated outcome from \(s\) to \(t\) is \(\lambda_t-\lambda_s\). The unit-specific baseline \(\alpha_i\) cancels:

E[Y_{it}(0)-Y_{is}(0)\mid G_i=g] = \lambda_t-\lambda_s.

Thus, this specification imposes a structured restriction on untreated outcomes across units and time. It can accommodate different fixed baseline levels, but not arbitrary cohort-specific untreated trends unless those trends are included in the model.

From Predictions to Cohort-Time Effects

For every treated observation in cohort \(g\) and period \(t\geq g\), the fitted untreated outcome is \(\hat{Y}_{it}(0)=\hat{\alpha}_i+\hat{\lambda}_t\). With \(N_g\) sampled units in cohort \(g\), the imputation estimator is:

\widehat{ATT}(g,t) = \frac{1}{N_g} \sum_{i:G_i=g} \left(Y_{it}-\hat{\alpha}_i-\hat{\lambda}_t\right), \qquad t\geq g.

Each term is a prediction residual for a treated unit. The cohort-time estimate is their average, in the outcome’s units. This is a sample average; its causal interpretation as an estimate of \(ATT(g,t)\) depends on the assumptions above and on the sampling or asymptotic framework used for the data.

Worked Example: Fitting Fixed Effects and Imputing a Cohort Effect

Consider a hypothetical two-period panel. Units A, B, and C adopt treatment in period 1; units N1 and N2 are never treated. The observed outcomes are:

Unit	Adoption	Outcome in period 0	Outcome in period 1
A	1	10	15
B	1	12	15
C	1	14	20
N1	Never	8	10
N2	Never	10	12

Assume no anticipation and the additive untreated-outcome model. Normalize \(\lambda_0=0\). Every unit is untreated in period 0, so its period-0 outcome gives its fitted unit effect: \(\hat{\alpha}_A=10\), \(\hat{\alpha}_B=12\), \(\hat{\alpha}_C=14\), \(\hat{\alpha}_{N1}=8\), and \(\hat{\alpha}_{N2}=10\). In period 1, only N1 and N2 are untreated. Their changes are both 2, so the fitted common time effect is \(\hat{\lambda}_1=2\). The untreated observations used in the fit are consistent with these fixed effects.

The imputed period-1 outcomes for A, B, and C are \(10+2=12\), \(12+2=14\), and \(14+2=16\). Their residuals are \(15-12=3\), \(15-14=1\), and \(20-16=4\). Therefore,

\widehat{ATT}(1,1) = \frac{3+1+4}{3} = \frac{8}{3} \approx 2.667.

The estimated period-1 effect for this cohort is 2.667 outcome units per unit. It is not an estimate of the effect for every individual: it is the average of the three observed-minus-imputed differences.

Worked Example: Multiple Cohorts and Calendar-Time Effects

Suppose a hypothetical panel has three units: A adopts in period 1, B adopts in period 2, and N is never treated. Their outcomes are shown below. For A, outcomes after adoption are observed treated outcomes; they are excluded from the untreated-model fit.

Unit	Adoption	Period 0	Period 1	Period 2
A	1	10	16	19
B	2	12	14	18
N	Never	8	10	12

Normalize \(\lambda_0=0\). The period-0 observations give \(\hat{\alpha}_A=10\), \(\hat{\alpha}_B=12\), and \(\hat{\alpha}_N=8\). In period 1, B and N are untreated; both have an increase of 2, so \(\hat{\lambda}_1=2\). In period 2, N is untreated and has increased by 4 from period 0, so \(\hat{\lambda}_2=4\). The untreated outcomes used to fit the model follow these fitted effects.

For cohort 1 in period 1, A’s imputed outcome is \(10+2=12\), giving a residual of \(16-12=4\). For cohort 1 in period 2, A’s imputed outcome is \(10+4=14\), giving \(19-14=5\). For cohort 2 in period 2, B’s imputed outcome is \(12+4=16\), giving \(18-16=2\). Thus the three cohort-time estimates in this small example are \(\widehat{ATT}(1,1)=4\), \(\widehat{ATT}(1,2)=5\), and \(\widehat{ATT}(2,2)=2\), all in outcome units. The model’s calendar-time effects are estimated using untreated observations, while the treatment effects are calculated from treated units’ residuals.

Why the Residual Average Targets an Effect

The estimator’s logic follows by substituting the untreated-outcome model into the observed outcome for a treated unit. For \(G_i=g\) and \(t\geq g\), write the observed outcome as the untreated potential outcome plus the individual treatment effect, \(Y_{it}=Y_{it}(0)+\tau_{it}\), where \(\tau_{it}=Y_{it}(1)-Y_{it}(0)\). Then:

Y_{it}-\hat{\alpha}_i-\hat{\lambda}_t = \tau_{it} +\varepsilon_{it} +(\alpha_i-\hat{\alpha}_i) +(\lambda_t-\hat{\lambda}_t).

Averaging this equality over units with \(G_i=g\) shows what the cell residual average contains: the average treatment effect, plus untreated disturbances and fixed-effect prediction errors. If the untreated model is correctly specified, the disturbances average to zero for the target cohort-time comparison, and the fitted counterfactuals consistently estimate the relevant untreated conditional means, the latter terms vanish in expectation or asymptotically. The remaining target is \(E[\tau_{it}\mid G_i=g]=ATT(g,t)\). This justification also shows why a good fit to the training data alone is not sufficient: the model must predict treated units’ untreated outcomes correctly.

The fit must also have support. Each treated unit needs untreated observations that identify its unit effect, and the untreated unit-period observations must connect the relevant units and periods well enough to identify their relative fixed effects. If no unit is untreated in a late period, for example, that period’s time effect cannot be learned from the untreated sample without additional structure. A numerical prediction produced despite such a gap may rely on an unstated extrapolation rather than on the imputation design.

Worked Example: Aggregating Cohort-Time Residuals

Suppose a hypothetical analysis estimates effects for two cohorts in the same calendar period. In cohort 1, \(N_1=40\) treated units have imputation residuals summing to 100 outcome units. In cohort 2, \(N_2=60\) treated units have residuals summing to 60 outcome units. The cohort-time estimates are:

\widehat{ATT}(1,t)=\frac{100}{40}=2.5, \qquad \widehat{ATT}(2,t)=\frac{60}{60}=1.0.

If the target is the average residual among all 100 treated units in these two cohorts in period \(t\), the correct pooled estimate weights each cohort effect by its number of treated units:

\widehat{ATT}_{\mathrm{pooled}}(t) = \frac{40(2.5)+60(1.0)}{40+60} = \frac{100+60}{100} = 1.6.

The pooled estimate is 1.6 outcome units. It differs from the unweighted average \((2.5+1.0)/2=1.75\), which gives equal weight to the two cohorts rather than to the treated units. The appropriate aggregation depends on the target population; the estimator’s residuals do not choose that target automatically.

Common Mistakes and Practical Checks

Including treated outcomes in the untreated fit. The incorrect reasoning is that more observations must improve the fixed-effects estimates. Treated outcomes contain treatment effects; including them can shift the fitted untreated-outcome model and contaminate the counterfactual predictions. Fit the model on untreated observations only.
Calling every treated-minus-predicted residual a causal effect. A residual is a causal effect only under the untreated-outcome model, no anticipation, and adequate support. A fitted regression does not establish those conditions.
Assuming fixed effects allow arbitrary trends. Unit fixed effects allow different levels, not different untreated slopes over time. If untreated trends differ systematically by cohort, the basic additive model may impute the wrong counterfactual.
Ignoring the aggregation target. A simple average of cohort estimates weights small and large cohorts equally. If the target is an average across treated units, use cohort-size weights, as in the preceding example.

Before interpreting an imputation estimate, check which observations entered the fit, whether the fixed effects are supported by untreated observations, and whether the additive model is substantively plausible for the untreated outcomes. These checks concern identification and counterfactual prediction; they are separate from the uncertainty calculation for the estimated effects.

Check Your Understanding

Answer each question using the definitions and estimator developed above.

Under no anticipation, which observations from a unit that adopts in period \(g\) can enter the untreated-outcome model?
In the additive model, what feature of untreated outcomes is represented by \(\alpha_i\), and what feature is represented by \(\lambda_t\)?
A treated unit has an observed outcome of 24 and an imputed untreated outcome of 19. What is its imputation residual, and what does it measure under the identifying assumptions?
Why can the basic unit-and-time fixed-effects model fail when cohorts have different untreated trends?
Two cohorts have estimated effects of 3 and 1, with 20 and 80 treated units respectively. What is the treated-unit-weighted pooled estimate?