Why Complete Cases Can Change the Answer
A complete-case panel analysis keeps only units observed at every period required for the analysis. That restriction can seem like a harmless way to obtain a rectangular data set. It is not harmless when the chance of remaining observed is related to outcomes, regressors, or unobserved determinants of the outcome. The retained units can then have a different outcome distribution or a different regressor–error relationship from the target population.
The key question is not simply how many units are missing. It is whether the units retained by the complete-case rule still identify the quantity of interest. The formulas below make that question precise for a panel mean, a regression coefficient, and a fixed-effects slope.
For a mean at a particular period, the relevant selection indicator is whether that period's outcome is observed. For a panel regression requiring the full trajectory, the relevant indicator is \(R_i\). These indicators need not be the same. In this tutorial, \(R\) denotes whichever retention indicator corresponds to the analysis being discussed, and \(\pi=P(R=1)>0\) is the probability of being retained.
Complete-Case Means
Let \(Y\) be the outcome whose population mean is of interest, with \(\mu=E[Y]\). The mean among retained units is \(E[Y\mid R=1]\). Because \(R\) is binary, \(E[RY]=P(R=1)E[Y\mid R=1]\), so the difference between the retained mean and the population mean has a simple expression.
This is an exact population identity, not an approximation. If higher outcomes make retention more likely, then \(\operatorname{Cov}(Y,R)>0\), and the complete-case mean is higher than the population mean. If lower outcomes make retention more likely, the direction is reversed. The identity also shows that a low attrition rate alone does not establish negligible bias: the covariance can be substantial even when only a modest fraction of units is missing.
Worked Example: A Complete-Case Mean After Outcome-Related Attrition
Suppose a hypothetical panel has six units, with outcome values at the target period of 2, 4, 6, 8, 10, and 12. Only the four units with outcomes 6, 8, 10, and 12 are complete cases. Compare the complete-case mean with the mean for all six units.
The full-population mean is \(\mu=(2+4+6+8+10+12)/6=42/6=7\). The retained mean is \((6+8+10+12)/4=36/4=9\). The complete-case mean is therefore 2 outcome units above the full-population mean.
The covariance identity gives the same result. Here \(E[R]=4/6=2/3\), \(E[RY]=(0+0+6+8+10+12)/6=6\), and \(E[Y]=7\). Thus \(\operatorname{Cov}(Y,R)=6-(2/3)(7)=4/3\), and \(\operatorname{Cov}(Y,R)/P(R=1)=(4/3)/(2/3)=2\). The discrepancy occurs because retention is associated with the outcome, not because the complete-case mean was calculated incorrectly.
For a finite sample, the complete-case mean is \(\bar{Y}_{R=1}=\sum_i R_iY_i/\sum_iR_i\), provided at least one unit is retained. It describes the retained units directly. Treating it as an estimate of the full-population mean requires a reason that the retained outcomes are representative for that target, such as \(E[Y\mid R=1]=E[Y]\). The stronger condition \(Y\) independent of \(R\) is sufficient, but not necessary.
Regression Coefficients Under Selection
Consider a correctly specified linear conditional-mean model for an outcome observed on a unit-period pair: \(Y_{it}=W_{it}'\beta+u_{it}\), where \(W_{it}\) contains the intercept and regressors, and \(E[u_{it}\mid W_{it}]=0\) in the target population. The complete-case pooled regression uses only units with \(R_i=1\). For clarity, suppose the required population moments exist and the selected design matrix has full rank. Its population coefficient is the selected-sample linear regression coefficient.
To see why, the selected regression coefficient solves the population normal equations \(E[\sum_t R_iW_{it}(Y_{it}-W_{it}'\beta_R)]=0\). Substitute \(Y_{it}=W_{it}'\beta+u_{it}\), move the terms involving \(\beta\) to the other side, and multiply by the inverse selected second-moment matrix. The displayed expression follows. The second term is the selection bias in the coefficient relative to \(\beta\). It vanishes if the selected regressor–error cross-products are zero and the selected design has full rank.
This condition distinguishes two situations. Selection that depends only on regressors can preserve the coefficient if the conditional-mean model remains correct among retained units, \(E[u_{it}\mid W_{it},R_i=1]=0\), and the retained regressors still vary enough to identify the coefficient. Selection related to outcomes or errors can instead make the selected cross-products nonzero. Even selection on regressors does not guarantee that a coefficient is comparable to the full-population coefficient if slopes differ across units or the fitted linear relationship is only an approximation: changing the regressor distribution can then change the regression projection.
Worked Example: Outcome-Based Selection Changes a Regression Slope
Consider four hypothetical observations from a population model \(Y=1+2X+u\). Their values are \((X,u)=(0,-1),(0,1),(1,-1),(1,1)\), giving outcomes \(Y=0,2,2,4\). Suppose the complete-case rule retains observations with \(Y\geq 2\). Compute the full-sample and retained-sample ordinary least-squares slopes.
In the full sample, \(\bar{X}=0.5\) and \(\bar{Y}=2\). The sum of centered cross-products is \(\sum(X-\bar{X})(Y-\bar{Y})=1+0+0+1=2\). The sum of centered squares for \(X\) is \(\sum(X-\bar{X})^2=1\). Thus the full-sample slope is \(2/1=2\), matching the specified slope.
The retained observations are \((0,2),(1,2),(1,4)\), with \(\bar{X}=2/3\) and \(\bar{Y}=8/3\). The centered cross-product sum is \((-2/3)(-2/3)+(1/3)(-2/3)+(1/3)(4/3)=2/3\). The centered \(X\) sum of squares is \((-2/3)^2+(1/3)^2+(1/3)^2=2/3\). The retained slope is therefore \((2/3)/(2/3)=1\), not 2. The selection rule has retained an error pattern associated with \(X\): among selected observations, the regressor–error cross-product is nonzero. The complete-case regression consequently does not recover the full-sample slope.
Selection and Fixed-Effects Slopes
Panel fixed-effects estimation first removes each unit's time-invariant component by subtracting its time average. For a balanced panel with \(T\) required periods, write \(\widetilde{X}_{it}=X_{it}-\bar{X}_i\) and \(\widetilde{u}_{it}=u_{it}-\bar{u}_i\), where the bars here are averages over the \(T\) periods. For a complete-case fixed-effects regression, the corresponding slope is centered around \[ \beta+ \left\{E\!\left[R_i\sum_t\widetilde{X}_{it}\widetilde{X}_{it}'\right]\right\}^{-1} E\!\left[R_i\sum_t\widetilde{X}_{it}\widetilde{u}_{it}\right], \] assuming the within-regressor matrix is nonsingular. Removing unit effects does not, by itself, remove selection bias: retention can make the within-regressor and within-error cross-products nonzero.
Worked Example: Complete-Case Selection Alters a Fixed-Effects Slope
Consider three hypothetical units observed in two periods. For each unit, \(X\) changes from 0 to 1. Outcomes follow \(Y_{it}=\alpha_i+2X_{it}+u_{it}\). Let the unit-specific effects be zero, and let the error pairs across the two periods be A: \((0,0)\), B: \((0,2)\), and C: \((0,-2)\). Suppose a unit is retained only if its second-period outcome is at least 2. Compute the fixed-effects slope using all units and using only complete cases.
For each unit, \(\widetilde{X}=(-0.5,0.5)\), so its within-regressor sum of squares is \(0.25+0.25=0.5\). The within-error pairs are A: \((0,0)\), B: \((-1,1)\), and C: \((1,-1)\). The within-error cross-products with \(\widetilde{X}\) are respectively \(0\), \(0.5(-1)+0.5(1)=0\)? No: using the ordered pairs gives \((-0.5)(-1)+(0.5)(1)=1\) for B, and \((-0.5)(1)+(0.5)(-1)=-1\) for C. These sum to zero across all three units.
The full-panel within-regressor sum of squares is \(3(0.5)=1.5\), and the within-error cross-product is \(0+1-1=0\). Hence the full-sample fixed-effects slope is \(2+0/1.5=2\). The second-period outcomes are 2 for A, 4 for B, and 0 for C, so the retention rule keeps A and B and drops C. Among retained units, the within-regressor sum of squares is \(1.0\), while the within-error cross-product is \(0+1=1\). The complete-case fixed-effects slope is \(2+1/1=3\). The rule selects on an outcome and removes the unit whose within-error cross-product offsets B's, so the within transformation does not protect the estimate.
When Is Complete-Case Analysis Valid?
Validity depends on the estimand and estimator. A condition adequate for one target need not be adequate for another. The following conditions are useful sufficient conditions, rather than an exhaustive list of all possible cases.
| Target | Sufficient condition for complete-case validity | What must also hold |
|---|---|---|
| Mean at a specified period | \(E[Y\mid R=1]=E[Y]\); independence of \(Y\) and \(R\) is stronger and sufficient. | The retained sample must contain observations, and the target period and outcome must be defined consistently. |
| Pooled regression coefficient | \(E[\sum_t R_iW_{it}u_{it}]=0\). | The conditional-mean model must apply to the target coefficient, and the retained design must have full rank. |
| Fixed-effects slope | \(E[R_i\sum_t\widetilde{X}_{it}\widetilde{u}_{it}]=0\). | There must be sufficient within-unit regressor variation among retained units. |
Missingness completely at random is a familiar sufficient condition for representativeness, subject to the sampling and model assumptions relevant to the analysis. But it is not necessary. For example, a mean may remain valid when retention depends on variables associated with the outcome if the retained and target outcome means nevertheless agree. Likewise, regression can remain valid when retention depends on regressors, provided the conditional mean is correctly specified among retained units and the selected design has adequate variation. In contrast, saying that missingness is conditionally independent of an outcome given observed history does not automatically make an unadjusted complete-case mean or regression valid. The estimator must still account for the conditioning information in a way that identifies its stated target.
Specify whether the goal is a population mean, a pooled regression coefficient, or a within-unit slope, and specify the periods and units to which it refers.
Define the indicator for inclusion in the analysis. For a full-trajectory complete-case regression, this is \(R_i=1\) only if every required outcome is observed.
For a mean, compare \(E[Y\mid R=1]\) with \(E[Y]\). For regression, assess whether selection leaves the appropriate regressor–error cross-products equal to zero and preserves identification.
Common Mistake
A common mistake is to argue that complete-case analysis is unbiased because the retained sample is large, or because the proportion missing is small. Neither fact establishes the required moment conditions. A large selected sample can estimate the wrong mean or regression coefficient very precisely. Another mistake is to assume that fixed effects solve all attrition problems: the transformation removes time-invariant unit components, but it does not force \(E[R_i\sum_t\widetilde{X}_{it}\widetilde{u}_{it}]\) to be zero. The correction is to state the target and examine how retention relates to the outcome or the regression errors relevant to that target.
Check Your Understanding
Answer these questions by identifying the target and the selection condition that matters.
- Suppose retention is more likely for units with high outcomes. What does the covariance identity imply about the direction of complete-case mean bias?
- For a pooled regression, which selected-sample cross-product appears in the coefficient bias formula?
- Why can selection based only on regressors preserve a correctly specified regression slope, and what additional identification condition is needed?
- In the fixed-effects example, why did retaining only units A and B change the slope even though the within transformation removed unit-specific means?
- State one sufficient condition for a complete-case mean to equal the population mean, and explain why a small missing fraction alone is not such a condition.