Why the Observed Group Can Change
Longitudinal studies aim to describe change over time, but not every participant contributes data at every visit. If participants who remain under observation differ in health or event risk from those who are no longer observed, the follow-up data describe a selected subset of the original cohort. A mean calculated correctly from that subset can still be a poor summary of the original cohort.
The key issue is not simply that the number of observations falls. It is whether the chance of being observed is related to the health or outcome values the analysis aims to describe. The earlier material on informative loss to follow-up and shared latent health provides the setting: observed follow-up can depend on health, while health can also be related to subsequent outcomes. This tutorial focuses on what that dependence does to the composition of the observed group.
The target matters. The average among participants observed at \(t\) is a valid description of those participants. It is not automatically an estimate of the average that would describe the original cohort at \(t\). These are different quantities. Selection is a problem when the analysis interprets the observed-participant summary as representing a broader target without sufficient justification.
For the examples below, assume a fixed visit, a well-defined outcome for every member of the target cohort, and no measurement error. These simplifications isolate the selection issue. In real studies, event occurrence can make some later outcomes undefined, and that requires careful definition of the target; it is not resolved merely by calling the missing measurement dropout.
How Retention Changes Composition
Let \(H\) denote a health or risk category at the visit, and suppose it has a small number of possible values. Among the participants who remain observed, the probability of a category is determined both by how common that category was in the cohort and by the category-specific retention probability:
This is Bayes' rule applied to subset membership. If high-risk participants are less likely to remain observed than low-risk participants, high risk will generally make up a smaller share of the observed group than of the cohort. The reverse happens if high-risk participants are more likely to remain. A difference in composition is not guaranteed just because retention varies: it depends on the prevalence of the health categories and their retention probabilities.
To connect composition to an outcome \(Y(t)\), suppose, for illustration, that within each health category the observed participants have the same mean outcome as all cohort members in that category. Then the observed mean combines the category means using the observed group’s category proportions:
The cohort mean uses \(P(H=h)\) instead of \(P(H=h\mid R(t)=1)\). Thus, even if the mean within each category is represented accurately, a change in category proportions can change the overall mean. If retention also selects participants within categories according to their unobserved outcomes, this simple within-category assumption fails, and the observed data alone do not reveal the missing outcomes.
Worked Example: A Retained Mean Below the Cohort Mean
Consider a hypothetical cohort of nine participants with follow-up scores in points. Six are classified as lower risk, with scores 2, 4, 6, 8, 10, and 12. Three are higher risk, with scores 14, 16, and 18. For this illustration, suppose the scores for all nine are known, although in an actual follow-up study the scores of participants who leave would not be observed.
The original-cohort total is \(2+4+6+8+10+12+14+16+18=90\) points, so its mean is \(90/9=10.00\) points. Suppose all six lower-risk participants remain observed, but among the higher-risk participants only the person with score 14 remains. The observed total is \(42+14=56\) points, and the observed mean is \(56/7=8.00\) points.
The observed-participant mean is therefore \(8.00-10.00=-2.00\) points below the cohort mean. Here the difference comes from selective retention of participants with lower scores, not from a change in the scores of the people who remained. The calculation demonstrates the mechanism; in real data, the cohort mean at follow-up is generally unknown precisely because some scores are missing.
Risk Composition in Counts and Probabilities
The same selection can be expressed as a change in the proportion at higher risk. Counts are often useful for checking whether a reported average is plausible and for communicating the mechanism without relying only on a model.
Worked Example: Higher Risk Becomes Less Common Among Those Retained
Suppose a hypothetical cohort has 100 participants: 40 at higher risk and 60 at lower risk. By the follow-up visit, 20 of the 40 higher-risk participants and 54 of the 60 lower-risk participants are observed.
The retention proportions are \(20/40=0.500\), or 50.0%, in the higher-risk group and \(54/60=0.900\), or 90.0%, in the lower-risk group. The observed group has \(20+54=74\) participants. Its higher-risk proportion is \(20/74=0.2703\), or 27.0%, compared with \(40/100=0.4000\), or 40.0%, in the original cohort.
Now suppose, just for a numerical illustration, that every higher-risk participant has a risk score of 8 points and every lower-risk participant has a score of 2 points. The original-cohort mean is \((40\times8+60\times2)/100=440/100=4.400\) points. The observed-group mean is \((20\times8+54\times2)/74=268/74=3.622\) points. The observed mean is lower because higher-risk participants are underrepresented. This does not show that risk scores improved; it shows that the people contributing measurements have a different risk composition.
When Measured Health Helps—and When It Does Not
If retention differs according to a measured health category, it can be useful to compare people within that category rather than comparing only overall means. One approach is direct standardization: calculate a mean in each category among observed participants, then combine those means using the category proportions in the target cohort. This changes the weights assigned to categories; it does not create information about outcomes that are unobserved within a category.
Worked Example: Standardizing to the Cohort’s Risk Mix
In a hypothetical cohort, 60 participants are lower risk and 40 are higher risk. At a follow-up visit, 54 lower-risk and 20 higher-risk participants are observed. Suppose their observed mean scores are 3 points and 9 points, respectively. The observed group’s overall mean is
To standardize to the original cohort’s risk mix, use its proportions: 60% lower risk and 40% higher risk. The standardized mean is \(0.60(3)+0.40(9)=1.80+3.60=5.400\) points. It differs from the observed mean because it gives the higher-risk category more weight.
This calculation represents the cohort mean only under assumptions. In particular, the observed mean within each risk category must represent the corresponding cohort mean, and each category in the target cohort must have a positive chance of being observed. Here those within-category representativeness assumptions are stipulated for the example. In practice, measured risk categories may not capture all health differences that affect both retention and the outcome. Standardizing on measured categories alone cannot repair selection within those categories.
Checks, Alternatives, and Limits
A useful first diagnostic is to describe who is and is not observed at each visit. Compare retention proportions across baseline health and risk groups, and, where available, across previous measurements. Report the numbers observed and the reasons for loss where those reasons are recorded. A changing risk composition across visits is a warning that the observed-group trajectory may not be the original cohort’s trajectory.
These checks have limits. Similar observed and unobserved groups on measured baseline characteristics do not prove that retention is unrelated to unmeasured current health or future event risk. Also, an association between a measured characteristic and retention does not by itself quantify the resulting bias. The MCAR, MAR, and MNAR distinctions describe different relationships between missingness and data; observed retention patterns alone generally cannot establish that missing outcomes are unrelated to their unobserved values.
| Approach | What it describes or assumes | Main caution |
|---|---|---|
| Observed-participant summary | The people who contributed measurements at that visit | It need not represent the original cohort. |
| Comparison by measured health strata | Differences among observed people within recorded categories | Unmeasured differences within categories may remain. |
| Standardization or weighting | A target distribution reconstructed using measured factors | Requires adequate measurement, appropriate assumptions, and a positive observation chance in relevant groups. |
| Joint modeling | A model for related longitudinal and event processes under specified assumptions | It is not an automatic correction; results depend on the model and its assumptions. |
No method makes the target irrelevant. If the scientific question concerns only the people observed at a visit, their mean may be the intended summary. If it concerns the original cohort’s health trajectory, an observed-only mean can be misleading when retention selects on health or risk. Alternative analyses may address measured selection or model related processes, but they require assumptions that should be stated and assessed. A common pitfall is to interpret a better observed mean over time as evidence that the cohort improved. The correction is to check who contributed at each time and whether the composition changed before interpreting the trend.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- A cohort is 30% high risk at enrollment. At follow-up, high-risk participants are less likely than low-risk participants to be observed. What happens to the high-risk share among observed participants, in general?
- In a hypothetical cohort, 25 of 50 high-risk participants and 45 of 50 low-risk participants are observed. What proportion of the observed group is high risk?
- Why can a retained-group mean be correct for the people observed but still fail to describe the original cohort?
- If retention differs by a measured risk group, why might standardizing the observed group to the cohort’s risk proportions still fail to recover the cohort mean?
Solutions. 1. It will generally be below 30%, because high-risk participants have a lower chance of entering the observed subset; the exact proportion also depends on the group-specific retention rates. 2. The observed total is \(25+45=70\), so the high-risk proportion is \(25/70=0.3571\), or 35.7%. 3. The mean can accurately summarize observed participants while their health or risk composition differs from that of the original cohort. The two means describe different populations. 4. Standardization addresses the measured group proportions, but it may not address selection within risk groups or differences due to unmeasured health factors.