Estimands Under Changing Panel Composition

Why Panel Composition Changes the Question

A panel follows units across waves, but it does not guarantee that the same units contribute records at every wave. Some may leave the target population, and some eligible units may not have an outcome recorded. Consequently, a mean calculated at one wave can describe a different set of units from a mean calculated at another. The numerical change may reflect changing outcomes, changing composition, or both.

The first task is therefore not to calculate a mean or fit a regression. It is to state which units the quantity is intended to describe. A fixed baseline cohort, the units still eligible at a later wave, and the units with recorded outcomes are different targets. Their summaries need not agree, even when all are calculated correctly.

Definition: Let $U_0$ be a defined baseline cohort and $Y_{it}$ the outcome for unit $i$ at wave $t$. A fixed-cohort population mean is the mean over the same baseline units at each wave, when the outcome is defined for all of them. Let $S_{it}=1$ indicate that unit $i$ remains in the population of interest at wave $t$. The survivor mean is the mean among units with $S_{it}=1$. Let $R_{it}=1$ indicate that the outcome is recorded for an eligible unit. The observed-sample mean is the mean among units with both $S_{it}=1$ and $R_{it}=1$.

In this terminology, “survivor” means retained in the defined target population; it need not mean biological survival. It could mean remaining enrolled, employed, or otherwise eligible. The definition must match the application. Likewise, a fixed-cohort mean is meaningful only if the outcome is defined for every baseline unit at that wave. If an outcome is undefined after death or another event, the researcher must define a different outcome or target rather than silently treating the missing value as an ordinary unobserved outcome.

A separate target may be the current-wave population: all units who belong to the population at wave $t$, including new entrants who were not in $U_0$. This can be appropriate for questions about the population at each date, but it is not a fixed-cohort comparison. Naming the target prevents a change in membership from being mistaken for change within the same units.

Three Means, Three Populations

The distinctions can be written directly. For a finite baseline cohort of size $N_0$, let $N_{S,t}$ be the number of baseline units still in the target population and $N_{O,t}$ the number with recorded outcomes among them. The corresponding means are:

\mu_{0t}=\frac{1}{N_0}\sum_{i\in U_0}Y_{it},\qquad \mu_{S,t}=\frac{1}{N_{S,t}}\sum_{i\in U_0:S_{it}=1}Y_{it},\qquad \bar{Y}_{O,t}=\frac{1}{N_{O,t}}\sum_{i\in U_0:S_{it}=1,R_{it}=1}Y_{it}.

Here $\mu_{0t}$ is the fixed-cohort population target, $\mu_{S,t}$ is the survivor target, and $\bar{Y}_{O,t}$ is the mean of the observed records. In a finite data set, the last quantity is directly calculable. It describes the observed units; by itself, it does not establish what the mean would have been for units without recorded outcomes. Whether it can represent a broader population depends on additional assumptions or adjustments, topics that require separate treatment.

Worked Example: Comparing Cohort, Survivor, and Observed Means

Suppose a hypothetical baseline cohort has four units. At wave 2, their outcomes are defined as 10, 12, 18, and 20. The first three units remain eligible for the target population; the fourth does not. Among the three eligible units, outcomes are recorded for units 1 and 3 only.

Unit	Wave-2 outcome	Still eligible, $S_{i2}=1$?	Recorded, $R_{i2}=1$?
1	10	Yes	Yes
2	12	Yes	No
3	18	Yes	Yes
4	20	No	No

The fixed-cohort mean is

\mu_{02}=\frac{10+12+18+20}{4}=\frac{60}{4}=15.

The survivor mean includes units 1, 2, and 3:

\mu_{S,2}=\frac{10+12+18}{3}=\frac{40}{3}\approx 13.33.

The observed-sample mean uses only units 1 and 3:

\bar{Y}_{O,2}=\frac{10+18}{2}=14.

All three values are correct for their stated targets. They differ because they average different sets of units, not because one calculation is inherently preferable. If the research question concerns the original cohort, 14 is not automatically an estimate of 15 merely because it is based on recorded data.

How Composition Changes a Wave-Specific Mean

A useful way to diagnose a changing mean is to divide the target population into groups $g$, such as regions, age categories, or baseline risk strata. Write $\pi_{gt}$ for group $g$'s share of the target at wave $t$, and $\mu_{gt}$ for its mean outcome. The overall mean is the weighted average of the group means:

\mu_t=\sum_g \pi_{gt}\mu_{gt},\qquad \sum_g\pi_{gt}=1.

Thus, an overall mean can change because group means change, because group shares change, or both. An exact decomposition between waves 1 and 2 is:

\mu_2-\mu_1 = \sum_g \pi_{g1}(\mu_{g2}-\mu_{g1}) + \sum_g(\pi_{g2}-\pi_{g1})\mu_{g2}.

The first term measures changes in group means, holding group shares at their wave-1 values. The second measures the composition change, evaluated using wave-2 group means. This is an accounting identity, not a causal decomposition; the particular allocation between terms depends on the chosen reference shares and means.

Worked Example: A Mean Rises With No Within-Group Change

Consider two groups in a hypothetical panel. The low-outcome group has a mean of 2 at both waves, and the high-outcome group has a mean of 8 at both waves. At wave 1, one quarter of the target population is in the high group. At wave 2, three quarters is in the high group.

At wave 1, the overall mean is

\mu_1=(0.25)(8)+(0.75)(2)=2+1.5=3.5.

At wave 2, it is

\mu_2=(0.75)(8)+(0.25)(2)=6+0.5=6.5.

The overall mean rises by $6.5-3.5=3.0$ outcome units even though neither group's mean changes. The entire difference comes from the high-outcome group's larger share. If the target is a fixed cohort whose group membership does not change, this example would not describe a change in that cohort's group shares; it instead illustrates why changing-wave or observed-sample averages can move when composition changes.

The distinction matters for reporting. A wave-specific mean can be a useful description of the people represented at that wave. But interpreting its difference from an earlier mean as improvement or deterioration for a stable population requires a stable target or an explicit standardization to a common composition. Otherwise, the comparison mixes outcome change with membership change.

Pooled Regression Targets the Observed Person-Period Distribution

Composition also matters in pooled panel regression. Before interpreting a pooled slope, define its target. With an intercept and one regressor $X_{it}$, the linear-projection slope in a specified target distribution is the coefficient that minimizes mean squared prediction error over that distribution. Equivalently, when the variance exists and is positive, it is the covariance of $X$ and $Y$ divided by the variance of $X$.

Definition: For a distribution of person-period records with finite second moments and $\operatorname{Var}(X)>0$, the pooled linear-projection slope is $$ \beta^{LP}=\frac{\operatorname{Cov}(X,Y)}{\operatorname{Var}(X)}. $$ The observed-record projection uses the distribution of records that are actually observed. Its finite-sample pooled OLS counterpart, with an intercept, is the centered cross-product of observed $X$ and $Y$ divided by the centered sum of squares for observed $X$.

The defining distribution matters: if the mix of units or groups contributing person-period records changes, then the covariance and variance in this expression can change. The pooled slope may therefore change even when the relationship within each group is stable. A pooled slope is not automatically the effect for a fixed cohort, a within-unit effect, or a causal effect. Those interpretations require a target and assumptions appropriate to them; a descriptive linear projection alone supplies none of those claims.

Worked Example: A Pooled Slope Changes With Group Representation

Consider hypothetical pooled person-period records from two groups. Each pair below consists of records with $X=0$ and $X=1$. In group A, the corresponding outcomes are 0 and 1. In group B, they are 0 and 3. Thus, the group-specific slopes in these constructed records are 1 and 3, respectively, and both groups have mean $X=0.5$. Fit a pooled linear projection with an intercept, without a group indicator.

First suppose there are three such pairs from group A and two from group B. There are 10 records, and the overall mean of $X$ is $0.5$. The centered sum of squares for $X$ is

S_{XX}=10(0.5)^2=2.5,

because each record has $X$-deviation $-0.5$ or $0.5$. A group-A pair contributes $0.5$ to $S_{XY}$, while a group-B pair contributes $1.5$. Therefore,

S_{XY}=3(0.5)+2(1.5)=4.5,\qquad \hat{\beta}_{pool}=\frac{S_{XY}}{S_{XX}}=\frac{4.5}{2.5}=1.8.

Now suppose the observed records contain one group-A pair and four group-B pairs. There are still 10 records, and $S_{XX}=2.5$, but

S_{XY}=1(0.5)+4(1.5)=6.5,\qquad \hat{\beta}_{pool}=\frac{6.5}{2.5}=2.6.

The pooled slope changes from 1.8 to 2.6 solely because group B contributes a larger share of the records. The within-group patterns were held fixed. These slopes describe different pooled record distributions; neither is, without further assumptions, a causal effect or a fixed-cohort target.

A Practical Workflow and a Common Pitfall

When panel composition varies, state the target before presenting a wave mean or regression coefficient. A compact workflow is:

Name the target population.
Specify whether the question concerns the baseline cohort, the units eligible at each wave, or the units with recorded outcomes.

Define the outcome for that target.
Check that the outcome is meaningful for every unit included in the estimand. If not, state the alternative outcome or target explicitly.

Describe composition across waves.
Report which units or groups contribute, and whether their shares differ. For group summaries, distinguish changes in group means from changes in group shares.

Label the regression distribution.
For a pooled slope, state that it is based on the observed person-period records, unless a different target distribution has been defined.

Common mistake: “The mean increased, so outcomes improved for the panel.” This reasoning treats the wave-specific averages as if they necessarily described the same units. The correction is to check the target and composition first. If group shares changed, calculate or report that change; if the intended comparison is for a fixed cohort, use a clearly defined fixed-cohort target and address how its outcomes are available.

A related mistake is to call a pooled slope “the panel effect” without qualification. Pooled OLS summarizes the linear relationship in the records included in its target distribution. Changing who contributes records can change that distribution and its slope. State the population and the interpretation instead of allowing “pooled” to imply a causal or within-unit effect.

Key takeaway: A panel does not by itself fix the population being summarized. Fixed-cohort, survivor, and observed-sample estimands answer different questions. Changing composition can alter both wave-specific means and pooled regression targets, even when outcomes or relationships within groups do not change.

Check Your Understanding

Use the definitions and calculations above to answer these questions.

In one sentence each, distinguish a fixed-cohort mean, a survivor mean, and an observed-sample mean.
A two-group population has unchanged group means, but the share of the higher-mean group rises. What can happen to the overall mean, and why?
For the two-group regression example, explain why changing the number of records from each group can alter the pooled slope even though the group-specific patterns stay fixed.
What condition on $X$ is needed for the stated linear-projection slope to be defined, and why?
Why does a change in a pooled panel slope not, by itself, establish that the causal effect changed?