Tutorials › Econometrics (Graduate) › Common Time Effects in Unbalanced Panels

Panel Designs and Unbalanced Data · Tutorial 291 of 952

Common Time Effects in Unbalanced Panels

Estimate common period effects with unit effects in an unbalanced panel, and diagnose whether the observed overlap identifies the comparisons you want.

Intermediate 10 min read

What You'll Learn

  • Specify an additive unit-and-time-effects model for an unbalanced panel
  • Derive the within-unit least-squares equations for common period effects
  • Use overlap among observed periods to assess identification
  • Interpret estimates when particular unit-period cells are absent
  • Distinguish estimable within-component contrasts from unidentified cross-component comparisons
  • Recognize why fixed effects alone do not correct selective panel observation

Why Common Time Effects Need Overlap

A panel may have an outcome recorded for some, but not all, units in every period. Unit indicators account for persistent differences across units, while period indicators represent time shifts shared by the units. The central question is whether the observed unit-period pattern contains enough overlap to distinguish those shared shifts from the unit effects.

An absent cell does not automatically prevent estimation. For example, a unit observed in periods 1 and 2 can link those periods even if another unit is missing in period 2. But if one group of units is observed only in early periods and another group only in later periods, there may be no observed path linking the two sets of periods. In that case, a common set of period effects cannot be identified across the full panel without additional restrictions.

Definition: A common period effect is an additive shift in the conditional outcome shared across units in a given period. In an unbalanced panel, the unit-period observation pattern is connected if every observed unit and period can be linked to every other through a sequence of observed unit-period cells. Connectedness allows unit and period effects to be identified relative to a normalization, subject to any further regressors or restrictions in the model.

The Additive Model and Its Assumptions

Let \(y_{it}\) be the outcome for unit \(i\) in period \(t\), and let \(\mathcal{T}_i\) be the set of periods in which that unit is observed. Consider the model

$$ y_{it}=\alpha_i+\lambda_t+u_{it}, \qquad t\in\mathcal{T}_i. $$

Here \(\alpha_i\) is a unit-specific intercept, \(\lambda_t\) is the common effect for period \(t\), and \(u_{it}\) is an unobserved disturbance. To set the origin of the period effects, use period 1 as the reference and impose \(\lambda_1=0\). Then \(\lambda_t\) for \(t>1\) is the estimated difference between period \(t\) and period 1, conditional on the unit effects. This is a normalization, not a claim that period 1 has no substantive effect.

The additive form assumes that each period shift is the same across units; it does not include unit-specific time effects. Least squares can be calculated whenever the design has the required rank. For an interpretation of the estimates as conditionally unbiased, a standard sufficient condition is that the disturbances have mean zero conditional on the observation pattern and the included unit and period indicators. If observation or attrition is related to unobserved outcomes, that condition may fail. Unit and period indicators do not, by themselves, correct selective panel observation.

Within-Unit Estimation

For each unit, define \(\bar{y}_i\) as its mean outcome across its observed periods and \(\bar{\lambda}_i\) as the mean of the corresponding period effects. Subtracting the unit-specific mean removes \(\alpha_i\):

$$ y_{it}-\bar{y}_i = (\lambda_t-\bar{\lambda}_i)+(u_{it}-\bar{u}_i). $$

This transformation explains why a unit observed in only one period contributes no within-unit comparison: its outcome equals its own unit mean, so its demeaned outcome is zero. It can still be part of the fitted model, but it cannot, by itself, reveal a period contrast separately from its unit effect.

With period 1 as the reference, form a vector \(z_{it}\) of period indicators for periods 2 through \(T\). Its entry for period \(k\) is 1 when \(t=k\) and 0 otherwise. Write \(\bar{z}_i\) for the mean of these indicators over unit \(i\)'s observed periods. The within-unit least-squares equations are

$$ \left[ \sum_i\sum_{t\in\mathcal{T}_i} (z_{it}-\bar{z}_i)(z_{it}-\bar{z}_i)' \right]\hat{\lambda} = \sum_i\sum_{t\in\mathcal{T}_i} (z_{it}-\bar{z}_i)(y_{it}-\bar{y}_i). $$

The matrix on the left is the within-transformed cross-product matrix for the period indicators. If it has full rank, the least-squares period-effect contrasts are uniquely determined under the reference-period normalization. If it is rank deficient, some contrasts cannot be separated from unit effects. For the additive model without additional regressors, connectedness of the observed unit-period graph is the corresponding overlap condition: treat units and periods as two types of nodes, with an edge for every observed cell. A path through observed cells links the effects across that graph.

Worked Example: Three Units Linking Three Periods

Suppose the following hypothetical observations are available. Blank cells are unobserved, not outcomes equal to zero.

UnitPeriod 1Period 2Period 3
A\(-17/3\)\(1/3\)\(16/3\)
B0—0
C—03

The cells are unbalanced, but the observation graph is connected: A is observed in all three periods, B links periods 1 and 3, and C links periods 2 and 3. Fit unit effects and period effects, with \(\lambda_1=0\). After removing each unit mean, the contributions to the period-indicator cross-product matrix are as follows. A, observed in all periods, contributes \(2/3\) to each diagonal and \(-1/3\) to each off-diagonal. B, observed in periods 1 and 3, contributes \(1/2\) to the period-3 diagonal. C, observed in periods 2 and 3, contributes \(1/2\) to each diagonal and \(-1/2\) to the off-diagonal. Thus

$$ \sum_{i,t}(z_{it}-\bar{z}_i)(z_{it}-\bar{z}_i)' = \begin{pmatrix} 7/6 & -5/6\\ -5/6 & 5/3 \end{pmatrix}. $$

For the right-hand side, A's mean outcome is zero, so its contributions for periods 2 and 3 are \(1/3\) and \(16/3\). B's two outcomes are both zero, so it contributes zero. C's mean is \(3/2\); its demeaned outcomes in periods 2 and 3 are \(-3/2\) and \(3/2\). Its centered period-indicator vectors are \((1/2,-1/2)\) and \((-1/2,1/2)\), respectively. Its right-hand-side contributions are therefore

$$ \begin{aligned} \text{period 2: } &(1/2)(-3/2)+(-1/2)(3/2)=-3/2,\\ \text{period 3: } &(-1/2)(-3/2)+(1/2)(3/2)=3/2. \end{aligned} $$

Adding across units gives \(1/3-3/2=-7/6\) for period 2 and \(16/3+3/2=41/6\) for period 3. Hence the normal equations are

$$ \begin{pmatrix} 7/6 & -5/6\\ -5/6 & 5/3 \end{pmatrix} \begin{pmatrix} \hat{\lambda}_2\\ \hat{\lambda}_3 \end{pmatrix} = \begin{pmatrix} -7/6\\ 41/6 \end{pmatrix}. $$

Multiplying both equations by 6 gives \(7\hat{\lambda}_2-5\hat{\lambda}_3=-7\) and \(-5\hat{\lambda}_2+10\hat{\lambda}_3=41\). Solving yields

$$ \hat{\lambda}_2=3,\qquad \hat{\lambda}_3=\frac{28}{5}=5.600. $$

The fitted common period effect for period 2 is 3 outcome units above the period-1 reference, and the fitted effect for period 3 is 5.600 outcome units above it. These are least-squares contrasts under the additive model; they are not automatically causal effects. In particular, the result depends on the common-effect specification and does not show what would have happened in an unobserved cell for a particular unit.

What Missing Cells Allow

An unobserved cell means that the corresponding unit-period outcome was not used in estimation. It does not mean that its value is zero, and the model does not directly observe that unit's outcome in that period. If the overlap graph remains connected, the common period contrasts can still be identified by combining comparisons across observed units. The estimates then rely on the assumption that the period shifts are common, even when no single unit is observed in every period.

A unit observed in just one period provides no within-unit contrast after its intercept is removed. More generally, units observed across different subsets of periods can contribute information where their observed periods overlap. The overlap need not occur for every pair of periods: a chain of shared observations can connect them. But comparisons made through a long or thin chain may be sensitive to specification and data quality, so identification is not the same as precision.

Worked Example: A Single-Period Unit Does Not Break Identification

Consider two periods and three hypothetical units. Units A and B are observed in both periods, while unit C is observed only in period 2. Suppose the observed outcomes are A: 4 in period 1 and 6 in period 2; B: 10 in period 1 and 14 in period 2; C: 100 in period 2. Fit unit and period indicators with \(\lambda_1=0\).

For each unit observed in both periods, differencing removes its intercept: A's difference is \(6-4=2\), and B's is \(14-10=4\). Under the model, each difference equals \(\lambda_2\) plus its differenced disturbance. The least-squares estimate averages these two paired differences. Unit C has only one observation; its within-demeaned outcome and period indicator are both zero, so it supplies no within-unit information about \(\lambda_2\).

$$ \hat{\lambda}_2 = \frac{(6-4)+(14-10)}{2} = \frac{2+4}{2} =3. $$

Thus the estimated common shift is 3 outcome units from period 1 to period 2. The unusually large outcome for C is absorbed by C's own intercept in this fixed-effects calculation; it is not evidence that the period effect is 100. Identification comes from A and B, which link the two periods. If neither A nor B had been observed in both periods, C alone could not distinguish a period shift from its unit effect.

When Period Comparisons Are Not Identified

If the observation graph has separate components, each component can be shifted by its own constant: adding a constant to all unit effects in a component and subtracting it from all period effects in that component leaves every fitted value unchanged. A reference-period normalization removes one overall shift, but it cannot remove independent shifts for disconnected components. Consequently, period contrasts within a connected component may be estimable while contrasts between components are not.

Worked Example: Disconnected Period Blocks

Suppose units A and B are observed only in periods 1 and 2, and units C and D are observed only in periods 3 and 4. There are no observations linking either of the first two periods to either of the last two. The observation graph therefore has two components.

Within the first component, observed within-unit comparisons can identify \(\lambda_2-\lambda_1\), if there is sufficient within-unit variation in the period indicators. Within the second, comparisons can identify \(\lambda_4-\lambda_3\) under the same condition. But the data cannot identify \(\lambda_3-\lambda_1\): adding a constant \(c\) to the period effects in the second component and subtracting \(c\) from the unit effects there leaves all observed fitted values unchanged. The relative level of the two blocks is therefore undetermined.

Setting \(\lambda_1=0\) does not solve this problem. It fixes the period-effect origin in the first component but supplies no information that connects the second component to that origin. A software package may omit a redundant indicator or report a normalization-dependent coefficient, but that convention does not create the missing comparison.

Estimation, Interpretation, and a Common Pitfall

A practical analysis should first describe which periods each unit contributes, then check whether the observed pattern connects the periods whose effects are to be compared. After fitting the model, report the reference period and the scale of the outcome. Inference also requires an appropriate variance estimate: panel disturbances may be correlated within units, so standard errors should reflect the panel structure and the design. The least-squares equations alone do not justify treating all observed cells as independent.

Common mistake: “The panel is unbalanced, so period effects cannot be estimated,” or the reverse, “There is at least one observation in every period, so all period contrasts are identified.” Neither conclusion follows. Balance is not required, but each period having observations is not enough. The relevant check is overlap and rank: observed cells must link the periods, after accounting for unit indicators and any other regressors in the model.

A second mistake is to interpret a fitted period effect as an observed change for every unit, including units missing that period. The estimate is a common additive contrast fitted from the available overlap under the model. It does not supply a directly observed counterfactual outcome for an absent cell, and it does not by itself remove bias from selective observation.

Key takeaway: Common time effects can be estimated in an unbalanced panel when observed overlap distinguishes period indicators from unit indicators. Missing cells are compatible with identification, but disconnected observation patterns leave cross-component period contrasts unidentified.

Check Your Understanding

Answer each question using the additive unit-and-period-effects model.

  1. Why does a unit observed in only one period contribute no within-unit contrast for estimating period effects?
  2. With period 1 as the reference, how should \(\lambda_3=2.4\) be interpreted when the outcome is measured in dollars?
  3. Three periods are connected by units observed in periods 1 and 2, and other units observed in periods 2 and 3. Is a direct unit observed in periods 1 and 3 necessary for identification? Explain.
  4. Why does having at least one observation in every period fail to guarantee that all period contrasts are identified?
  5. What does a missing unit-period cell mean for the interpretation of the estimated common period effect?
Clinical Biostats