What Repeated Outcomes Reveal—and What They Do Not
A longitudinal study records outcomes over time, so an earlier outcome can be informative about a later one. That predictive information is useful, but it does not automatically reveal how people who drop out would have fared at the next measurement. The key distinction is between observed histories and the complete-data relationship between dropout and an outcome that was not observed.
Consider a two-wave study. Let \(Y_1\) be the first outcome, recorded for everyone, and let \(Y_2\) be the second outcome. Let \(R=1\) if \(Y_2\) is observed and \(R=0\) if the participant drops out before it is measured. For clarity, suppose the outcomes are binary, with values 0 and 1. The observed record is \((Y_1,R,Y_2)\) when \(R=1\), and \((Y_1,R)\) when \(R=0\).
The distinction matters even when response and dropout both occur among people with each first-wave outcome. In other words, having participants who remain and participants who drop out in every observed \(Y_1\) group does not fill in the missing \(Y_2\) values. It only ensures that both response patterns are represented in those groups.
A Two-Wave Example: Same Histories, Different Complete Data
Suppose a hypothetical study has the following observed-data probabilities. Each row gives a joint probability in the full study population. For example, 0.12 is the probability that \(Y_1=0\), the second outcome is observed, and \(Y_2=1\).
| First outcome \(Y_1\) | Observed at wave 2, \(Y_2=1\) | Observed at wave 2, \(Y_2=0\) | Dropped out before wave 2 |
|---|---|---|---|
| 0 | 0.12 | 0.18 | 0.20 |
| 1 | 0.24 | 0.06 | 0.20 |
The probabilities sum to 1. The dropout cell for \(Y_1=0\), for instance, establishes that 20% of the population had \(Y_1=0\) and dropped out. It does not establish how many of those people would have had \(Y_2=1\). That missing split can vary without changing any entry in the table.
Worked Example: Completing the Same Observed Distribution in Two Ways
Construct two complete-data laws that agree with the table but allocate missing second-wave outcomes differently. In each law, let \(q_h=P(Y_2=1\mid Y_1=h,R=0)\), the probability of a positive second outcome among dropouts with first outcome \(h\).
In Model A, set \(q_0=0.20\) and \(q_1=0.70\). For \(Y_1=0\), the dropout mass is 0.20, so the missing \(Y_2=1\) mass is \(0.20(0.20)=0.04\); the missing \(Y_2=0\) mass is \(0.20-0.04=0.16\). For \(Y_1=1\), those masses are \(0.20(0.70)=0.14\) and \(0.20-0.14=0.06\).
In Model B, set \(q_0=0.80\) and \(q_1=0.30\). The dropout masses split as \(0.20(0.80)=0.16\) and \(0.20-0.16=0.04\) for \(Y_1=0\), and as \(0.20(0.30)=0.06\) and \(0.20-0.06=0.14\) for \(Y_1=1\).
Both models leave the observed cells unchanged. They therefore imply exactly the same probabilities for every recorded history, even though their unobserved outcomes among dropouts differ. The total probability of \(Y_2=1\) is 0.54 in Model A and 0.58 in Model B:
The difference in these complete-data means is not a discrepancy in the observed histories. It comes entirely from allocating different \(Y_2\) values to people whose \(Y_2\) was not recorded.
Why the Missing Association Is Not Identified
The missing split can be seen directly by writing the observed probabilities and the unknown dropout probabilities separately. For each \(h\in\{0,1\}\), the observed data reveal \(P(Y_1=h,R=0)\), but not the probability of \(Y_2=1\) within that dropout cell. Given any \(q_h\) between 0 and 1, that cell can be completed as follows:
Neither expression is separately observed. Their sum is the observed dropout probability \(P(Y_1=h,R=0)\), whatever value \(q_h\) takes. By contrast, the probabilities in the responding group, such as \(P(Y_1=h,R=1,Y_2=y)\), are observed. This gives a direct justification for nonidentification: varying \(q_h\) changes the complete-data law while leaving the observed-data law fixed.
The same point can be expressed as a short derivation. The observed likelihood contribution for a dropout with \(Y_1=h\) sums over the unobserved value of \(Y_2\):
The unknown \(q_h\) cancels from the observed contribution. More observations of the same recorded variables can estimate the observed probabilities more precisely, but they do not create a separate observed contribution that depends on \(q_h\). Thus, the issue is structural nonidentification, not merely an imprecise estimate.
Worked Example: The Dropout Association Can Reverse
Use the two compatible models above to calculate whether response is more or less common for people with \(Y_2=1\) than for people with \(Y_2=0\), within each \(Y_1\) group. Although \(Y_2\) is missing for dropouts, the probability of response conditional on \(Y_2\) is a feature of the complete-data law.
For \(Y_1=0\) in Model A, the response probability among people with \(Y_2=1\) is \(0.12/(0.12+0.04)=0.75\). Among those with \(Y_2=0\), it is \(0.18/(0.18+0.16)=0.529\). Response is more common for \(Y_2=1\) in this group. In Model B the corresponding probabilities are \(0.12/(0.12+0.16)=0.429\) and \(0.18/(0.18+0.04)=0.818\). Now response is less common for \(Y_2=1\).
For \(Y_1=1\) in Model A, the probabilities are \(0.24/(0.24+0.14)=0.632\) for \(Y_2=1\) and \(0.06/(0.06+0.06)=0.500\) for \(Y_2=0\). In Model B, they are \(0.24/(0.24+0.06)=0.800\) and \(0.06/(0.06+0.14)=0.300\), respectively. The computed response–outcome association is not the same across the two complete-data laws, despite identical observed histories.
The denominators include both response patterns for the specified complete-data outcome and first outcome. For example, in Model A with \(Y_1=0,Y_2=1\), the total probability is \(0.12+0.04=0.16\), of which 0.12 has \(R=1\). The calculation is therefore \(P(R=1\mid Y_1=0,Y_2=1)=0.12/0.16=0.75\).
More Waves Add History, Not the Missing Future Outcome
With more measurement occasions, a person may have a longer observed history before dropping out. That history can be informative about the next outcome. But the same identification issue remains within any history group: if the next outcome is absent for dropouts in that group, the data do not directly show its distribution among them.
Worked Example: A Third Outcome After an Observed History
Suppose a hypothetical three-wave study considers one particular history \(H=(Y_1,Y_2)\), observed for everyone in the group under consideration. Half of this group has \(Y_3\) recorded, and among those responders 60% have \(Y_3=1\). The other half drops out before wave 3. Compare two completions: \(P(Y_3=1\mid H,R_3=0)=0.20\) and 0.80, where \(R_3=1\) denotes observation of \(Y_3\).
In the first completion, the population probability of \(Y_3=1\) in this history group is \(0.50(0.60)+0.50(0.20)=0.40\). In the second, it is \(0.50(0.60)+0.50(0.80)=0.70\). Both completions preserve the same history distribution, response rate, and observed \(Y_3\) distribution. The added earlier outcome \(Y_2\) defines a more detailed history group, but it does not reveal the unobserved \(Y_3\) values within that group's dropouts.
This is why repeating measurements does not, by itself, identify dependence between dropout and the unobserved future outcome. Identification would require additional information or a restriction linking the missing-outcome distribution to quantities that are observed.
Assumptions, Diagnostics, and Alternatives
The argument assumes that \(Y_1\) is observed for all participants, that \(R\) correctly records whether \(Y_2\) is observed, and that observed \(Y_2\) values are measured consistently. The examples also have both response and dropout in each \(Y_1\) group. Thus, the nonidentification is not caused by an empty observed history group. It persists even with positive representation of both patterns.
A missing-at-random restriction, for example, would set the distribution of \(Y_2\) among dropouts equal to its distribution among responders after conditioning on the observed history. In the two-wave binary setup, that restriction would impose \(P(Y_2=1\mid Y_1=h,R=0)=P(Y_2=1\mid Y_1=h,R=1)\). It can supply the missing \(q_h\), but the equality is an assumption about unobserved outcomes, not a fact established by the table. A parametric MNAR model can instead impose a different relationship, but its extrapolation likewise depends on restrictions that the observed histories alone do not determine.
Useful diagnostics can expose data problems but cannot test the missing split directly. Check that response indicators and outcome histories are coded as intended, tabulate response patterns within prior-history groups, and look for histories with very few responders or dropouts. Such checks can reveal empty or sparse groups and unstable estimation. They cannot establish how the unobserved outcomes among dropouts are distributed. In particular, a good fit to observed histories does not distinguish two complete-data laws that agree on those histories.
The main alternatives answer different questions. A model based on a missingness restriction produces point estimates conditional on that restriction. Sensitivity analysis varies the unidentified dropout-outcome relationship to show how conclusions change. Bounds describe a range of values compatible with the observed data and specified restrictions, rather than selecting one completion. None of these approaches makes repeated outcomes alone reveal the missing future outcomes.
A common mistake is to reason that because \(Y_1\) predicts \(Y_2\) among responders, the observed \(Y_1\) values must determine \(Y_2\) among dropouts. Prediction is not identification: responders and dropouts with the same \(Y_1\) can have different distributions of unobserved \(Y_2\). The correction is to separate the observed conditional distribution from the missing one, then state what assumption, if any, connects them.
Check Your Understanding
Answer each question, then compare with the solutions below.
- In the two-wave setup, what does the observed dropout probability \(P(Y_1=h,R=0)\) tell us, and what does it not tell us about \(Y_2\)?
- In the third-outcome example, calculate the population probability of \(Y_3=1\) when the dropout-group probability is 0.50 rather than 0.20 or 0.80.
- Why does having both responders and dropouts in each first-outcome group fail to identify the dropout-outcome association?
- Does a strong association between \(Y_1\) and \(Y_2\) among responders establish the distribution of \(Y_2\) among dropouts with the same \(Y_1\)? Explain.
Solutions. (1) It gives the proportion with first outcome \(h\) who dropped out, but not how that probability is divided between \(Y_2=0\) and \(Y_2=1\). (2) The probability is \(0.50(0.60)+0.50(0.50)=0.30+0.25=0.55\). (3) Both patterns being represented avoids an empty group, but \(Y_2\) remains unobserved for dropouts; its distribution within that pattern can still vary while preserving the observed law. (4) No. The responder association describes observed outcomes among responders. Equating it to the dropout distribution requires an additional assumption, not a consequence of the observed data.