Why the Complete-Case Group Can Mislead
Longitudinal studies aim to describe how health changes over time, but some participants stop providing measurements. If people with worsening health are more likely to drop out, those who remain measurable may have less progression than the original cohort. A comparison based only on participants with measurements at every scheduled visit can therefore give a misleading picture of change.
The key issue is not simply that some measurements are absent. It is that inclusion in the complete-case group may be related to the progression being estimated. A complete-case calculation can be perfectly correct for the participants it includes and still fail to represent the cohort from which they came.
A complete-case mean directly describes the observed completers. It describes the baseline cohort's mean change only when the completers' average change is representative of the whole cohort. For example, this holds in expectation under missing completely at random, but can fail when dropout is associated with current health or unobserved progression. Even when dropout is related to measured information, a raw complete-case mean need not represent the cohort.
A Numerical Demonstration
Suppose a higher biomarker value indicates worse health. All participants have a baseline measurement, and the study plans to measure the biomarker again one year later. The values below are hypothetical. To show the distortion transparently, the table includes the follow-up values that would have been observed for everyone, although the values for those who dropped out would not be available to an analyst in an actual study.
Worked Example: High Progression Is More Likely to Be Missing
Eight participants begin the study with a biomarker value of 10 units. Four complete the one-year measurement; four drop out as their health worsens. Calculate the complete-case mean change and the cohort mean change.
| Participant | Baseline (units) | One-year value (units) | Change (units) | Follow-up observed? |
|---|---|---|---|---|
| A | 10 | 11 | 1 | Yes |
| B | 10 | 12 | 2 | Yes |
| C | 10 | 11 | 1 | Yes |
| D | 10 | 12 | 2 | Yes |
| E | 10 | 15 | 5 | No |
| F | 10 | 16 | 6 | No |
| G | 10 | 15 | 5 | No |
| H | 10 | 14 | 4 | No |
For the four complete cases, the observed changes are 1, 2, 1, and 2 units. Thus the complete-case mean change is
For the full hypothetical cohort, the changes sum to \(6+20=26\) units across eight participants. The cohort mean change is therefore
The complete-case result understates the cohort's mean progression by \(3.25-1.5=1.75\) units. The calculation is not evidence that every real complete-case analysis has this degree of bias. It demonstrates how selection can arise: participants with the largest worsening are absent from the group used to calculate the complete-case result.
Why Selection Changes the Mean
The complete-case mean is a conditional mean: it averages change among those with \(R=1\). The cohort mean averages change without conditioning on being observed. Their difference can be expressed directly in terms of the relationship between change and complete-case status.
To verify this, because \(R\) is an indicator, \(E(\Delta R)=P(R=1)E(\Delta\mid R=1)\). Also, \(\operatorname{Cov}(\Delta,R)=E(\Delta R)-E(\Delta)E(R)\), and \(E(R)=P(R=1)\). Substituting and dividing by \(P(R=1)\) gives the displayed expression. If larger changes tend to occur among participants who are not complete cases, then change and \(R\) tend to be negatively associated, and the complete-case mean is lower than the cohort mean.
This identity describes the source of the distortion, not a way to calculate the bias from an ordinary incomplete data set. For participants with missing follow-up, their changes are unknown. Therefore the covariance, and hence the difference between the two means, is generally not identified from the observed measurements alone without additional assumptions or information.
Paired Comparisons and Changing Groups
A complete-case longitudinal comparison should use the same participants at baseline and follow-up. This paired comparison calculates each completer's change and then averages those changes. A different calculation—subtracting the mean baseline value for the whole cohort from the mean follow-up value among completers—uses different groups at the two times. That difference mixes within-person change with a change in group composition.
Worked Example: A Changing Group Can Reverse the Apparent Direction
Six participants have baseline biomarker values of 2, 3, 4, 8, 9, and 10 units. The first three complete follow-up, each with a value one unit higher than baseline. The other three drop out; for this hypothetical illustration, suppose their changes would also have been positive, each four units.
The all-participant baseline mean is
The completers' follow-up values are 3, 4, and 5 units, so their follow-up mean is
Subtracting these means gives \(4-6=-2\) units, which might be misread as an average improvement of 2 units. But the paired change among the same three completers is
The apparent improvement came from comparing a baseline group that included participants with high values to a follow-up group containing only the lower-baseline completers. Even the paired result of 1 unit describes only those completers. In this hypothetical cohort, the full-cohort mean change is \((1+1+1+4+4+4)/6=2.5\) units of worsening.
A paired complete-case comparison avoids the particular group-mismatch error in this example; it does not remove selection bias. If participants who complete follow-up have systematically different changes from those who drop out, their paired mean change remains a selected-group summary.
Assessing the Possible Distortion
An actual study cannot calculate the full-cohort mean change when follow-up values are missing unless it makes assumptions or uses additional information. A useful first step is to make the dependence on the missing changes explicit rather than treating them as if they were known.
Worked Example: A Tipping-Point Calculation
Suppose 10 participants enter a study. Six are complete cases, with a mean observed change of 1.5 biomarker units. The other four have missing follow-up values. Let \(d\) be the average change, in units, among those four participants. What cohort mean change corresponds to a mean of 2 units?
The six observed changes sum to \(6(1.5)=9\) units. If the four missing changes average \(d\), their total is \(4d\). The cohort mean, as a function of this unknown average, is
Set this equal to 2 units and solve:
Thus, if the missing participants' average change were 2.75 units, the full-cohort mean would be 2 units. If their average change were greater than 2.75 units, the cohort mean would exceed 2 units. This is a sensitivity calculation, not an estimate of the missing participants' actual progression. Its value is that it makes clear which assumptions about unobserved changes would alter the interpretation.
Diagnostics, Alternatives, and Limitations
Observed data cannot prove that complete-case selection is harmless, but several checks can reveal reasons for concern. Compare dropout rates across baseline health groups and across levels of earlier observed biomarker values. Examine whether participants who later drop out had different prior trajectories, and summarize stated reasons for loss to follow-up. These checks can show that the complete-case group differs from those with incomplete records; similar observed histories do not establish that their unobserved changes are similar.
The main approaches differ in what they estimate and what assumptions they require:
| Approach | What it summarizes | Main caution |
|---|---|---|
| Complete-case paired mean change | Mean change among participants observed at both times | May not represent the baseline cohort if completeness is related to progression |
| Observed-data longitudinal model | Can use incomplete records to estimate a specified mean trajectory | Its interpretation depends on the model and assumptions about the missingness process |
| Inverse-probability weighting | Reweights observed participants to represent a target cohort | Requires an adequately specified observation model and sufficient probability of observation across relevant groups |
| Sensitivity analysis | Shows how conclusions change under alternative assumptions about missing outcomes | Does not reveal which assumption is correct from the observed outcomes alone |
A small dropout percentage does not guarantee negligible bias: a small number of missing participants can matter if their progression differs substantially. Conversely, a large dropout percentage does not automatically prove a particular direction or magnitude of bias. The direction depends on how missingness relates to change, and the size depends both on that relationship and on how many observations are missing.
Complete-case results can still be useful when the target is explicitly the group of completers, or when justified assumptions make completers representative for the cohort-level quantity of interest. The crucial practice is to name the target. “Mean change among completers” is a direct description of the selected data; “mean change in the baseline cohort” needs stronger support.
Common Mistakes
- Equating a paired calculation with an unbiased cohort estimate. Pairing ensures the baseline and follow-up values come from the same people. It does not ensure those people represent everyone enrolled.
- Subtracting means from different groups. A whole-cohort baseline mean minus a completer-only follow-up mean combines progression and group-composition differences.
- Treating missing outcomes as if they were observed. A numerical illustration may specify hypothetical missing values, but an actual study cannot know those values from the observed data alone.
- Claiming that dropout proves bias in a particular direction. Bias depends on the relation between dropout and change. Health-related dropout makes distortion plausible, but observed data and assumptions determine what can be concluded.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In a study where higher biomarker values indicate worse health, what does a complete-case mean change directly describe?
- Six complete cases have changes of 0, 1, 1, 2, 2, and 3 units. What is their mean change? Does that calculation alone establish the cohort mean change?
- Why can subtracting the completers' follow-up mean from the full cohort's baseline mean give a misleading result?
- In the tipping-point example, if the four missing participants average 4 units of change, what is the implied cohort mean change?
- Name one observed-data diagnostic for health-related dropout and state one limitation of that diagnostic.
Solutions. (1) It describes mean change among participants whose baseline and follow-up measurements were both observed. (2) The mean is \((0+1+1+2+2+3)/6=9/6=1.5\) units. It does not by itself establish the cohort mean, because the missing participants' changes may differ. (3) The two means refer to different groups, so their difference mixes within-person progression with a change in group composition. (4) Using \((9+4d)/10\), the implied mean is \((9+4(4))/10=25/10=2.5\) units. (5) One can compare prior observed biomarker trajectories between participants who later complete follow-up and those who drop out. This can reveal observed differences, but cannot establish that their unobserved changes are alike or different.