Prediction and mechanism answer different questions
A dropout model can predict very well who will leave a longitudinal study. That is useful for planning follow-up, describing attrition, or identifying histories associated with withdrawal. It does not, by itself, establish that dropout is missing at random (MAR). Prediction asks whether recorded information distinguishes people who remain from people who leave. MAR asks whether, after conditioning on recorded information, dropout still depends on outcomes that were not observed.
The distinction matters because the outcome that is missing for a dropout is precisely the information needed to assess whether dropout depends on that outcome. A model can be highly accurate using observed history while the missing outcomes still matter to the response mechanism. Conversely, history may predict dropout under MAR because MAR allows dropout probabilities to vary with recorded history.
The relevant history must be specified. If an earlier outcome is recorded and included in \(H\), dropout may depend on that outcome and still satisfy MAR. If a variable that predicts dropout is available but omitted from \(H\), the condition above may fail for the chosen conditioning set. This is why the earlier material on mechanism labels and conditioning sets matters: “MAR” is shorthand for a particular conditional-independence assumption, not a label inferred from a good prediction score.
Why prediction from history is compatible with MAR
Under MAR, the probability of response can vary across recorded histories. For example, participants with a poor last-observed symptom score may be more likely to leave than those with a good score. The model \(P(R=1\mid H)\) can therefore have substantial predictive power. What MAR rules out is additional dependence on \(Y\) after conditioning on \(H\), not dependence on \(H\) itself.
The following decomposition makes the distinction explicit. The probability of response given history alone averages over possible outcomes:
with an integral replacing the sum for a continuous outcome. A strong relationship between \(H\) and dropout may arise because response probabilities differ by \(H\). It may also arise when response depends on \(Y\), since the distribution of \(Y\) can differ across histories. Observing the averaged probability does not reveal whether \(P(R=1\mid H,Y)\) changes with \(Y\) within a history.
Worked Example: A Predictive History That Is Compatible With MAR
Suppose a hypothetical follow-up study records each participant’s last observed symptom category, \(H\), and whether the next outcome is observed, \(R\). Among 100 participants with a poor last-observed score, 60 drop out; among 140 with a good score, 12 drop out. Calculate the dropout risks by history and explain what the results do—and do not—show.
For the poor-score group, the estimated dropout proportion is \(60/100=0.600\), or 60%. For the good-score group, it is \(12/140=0.0857\), or about 8.57%. The risk difference is \(0.600-0.0857=0.5143\), a difference of 51.4 percentage points. Thus, the recorded history strongly predicts dropout in these hypothetical data.
This pattern is compatible with MAR: dropout may depend on the recorded symptom category, while being independent of the next outcome conditional on that category. But the same observed dropout rates are also compatible with a mechanism that, within each symptom category, gives people with worse unobserved next outcomes a higher probability of leaving. The counts alone cannot distinguish those possibilities. A prediction based on \(H\) establishes an association with dropout, not conditional independence from the unseen outcome.
Dropout may be predictable and still depend on an unseen outcome
A prediction model uses variables that are recorded. It cannot directly use an outcome that is missing for the person whose dropout it predicts. That does not mean the missing outcome is irrelevant to the mechanism. A latent, unobserved outcome can affect the probability of dropout, while recorded history remains predictive because it is associated with that outcome or has a separate relationship with dropout.
Worked Example: History Prediction Does Not Remove Outcome Dependence
Consider a hypothetical study with 100 participants in each of two recorded-history groups. For illustration, suppose the complete outcomes and response decisions were generated as follows. The outcome \(Y\) is either high or low; \(R=0\) indicates dropout before \(Y\) is observed.
| Recorded history \(H\) | Outcome \(Y\) | Number | Dropouts |
|---|---|---|---|
| Poor | High | 40 | 30 |
| Poor | Low | 60 | 15 |
| Good | High | 10 | 5 |
| Good | Low | 90 | 9 |
Within the poor-history group, the dropout probabilities are \(30/40=0.750\) for a high outcome and \(15/60=0.250\) for a low outcome. Within the good-history group, they are \(5/10=0.500\) and \(9/90=0.100\), respectively. In both groups, response depends on \(Y\) after conditioning on \(H\); this mechanism is not MAR for this conditioning set.
History also predicts dropout: the poor-history group has \(45/100=0.450\) dropouts, while the good-history group has \(14/100=0.140\). An analyst who fits a dropout model using only \(H\) could therefore find a clear predictive association. But the model’s prediction does not remove the within-history dependence on \(Y\).
In an actual incomplete dataset, the analyst would not see \(Y\) for the 59 dropouts. The complete table is shown only to make the hypothetical generating mechanism explicit; its unseen outcomes could not be used to test that mechanism from the observed data alone. The example’s point is not that a predictive model must be wrong, but that predictive success is insufficient evidence for MAR.
The same observed data can fit different assumptions
The missing outcomes limit what can be learned from dropout prediction. Even with a very simple binary outcome and no additional history, the observed data may be consistent with both an MAR completion and an alternative completion in which response depends on the unseen outcome. The following calculation shows why a prediction model cannot settle the question.
Worked Example: Two Mechanisms, One Observed Table
Suppose a hypothetical sample of 100 participants has 60 observed outcomes: 30 high and 30 low. The other 40 outcomes are missing. Under an MAR completion, divide the missing outcomes equally: 20 high and 20 low. Then there are 50 high and 50 low outcomes overall, with 30 of each observed. The response probabilities are \(30/50=0.600\) for high outcomes and \(30/50=0.600\) for low outcomes, consistent with MAR.
Now consider a different completion of the same observed data: 25 of the 40 missing outcomes are high and 15 are low. There are then 55 high and 45 low outcomes in all. The response probabilities would be $$ P(R=1\mid Y=\text{high})=\frac{30}{55}=0.5455, \qquad P(R=1\mid Y=\text{low})=\frac{30}{45}=0.6667. $$ These probabilities differ, so response depends on the outcome in this completed dataset. Yet the observed records are unchanged: 30 observed high outcomes, 30 observed low outcomes, and 40 missing outcomes.
The two completions are not claims about which mechanism generated a real dataset. They demonstrate that observed response counts and observed outcomes do not by themselves reveal the distribution of outcomes among dropouts. A dropout prediction model may describe observable differences in response, but it cannot choose between these unobserved-outcome assumptions solely through its predictive accuracy.
What prediction diagnostics can and cannot establish
Prediction diagnostics remain useful. Calibration asks whether predicted dropout probabilities agree with observed dropout frequencies across groups; discrimination asks whether the predictions rank participants with higher dropout risk above those with lower risk. Both are assessed using observed histories and response indicators. Good calibration and discrimination support the practical usefulness of a prediction model for those purposes. They do not assess whether, at the same history, response depends on an outcome missing for dropouts.
A very accurate model deserves careful inspection, not an automatic MAR label. Check whether its apparent performance comes from genuinely available history, whether predictions are calibrated in relevant history groups, and whether groups with few or no observed outcomes are being extrapolated over. Sparse response within a history can make outcome analysis unstable even if the MAR assumption is adopted. That lack of support is a separate concern: it does not prove MNAR, and good support does not prove MAR.
The main alternatives answer different questions. A dropout prediction model estimates response patterns from recorded information. A MAR analysis assumes that recorded information suffices to account for outcome-related differences in response. An MNAR sensitivity analysis examines how conclusions could change under specified dependence on missing outcomes. The first is a predictive task; the latter two concern assumptions about the unobserved part of the data. A model’s prediction score cannot substitute for the sensitivity questions raised by those assumptions.
A practical reasoning sequence
When a dropout model performs well, keep its interpretation separate from the mechanism claim. State what was observed, identify the history being conditioned on, and then state what additional assumption would be needed to call the mechanism MAR. In particular, ask whether dropout could still vary with the unobserved outcome among participants with the same recorded history. If the answer cannot be learned from the observed data, say so rather than treating predictive fit as a test.
For example, specify that the model predicts whether the next outcome is observed, using baseline information and previously recorded outcomes.
Report calibration or discrimination in context, without interpreting it as evidence that the unseen outcome is irrelevant.
For MAR, specify that response is independent of the missing outcome conditional on the stated recorded history.
Observed-history prediction does not reveal how dropout varies with unseen outcomes within the same history; conclusions depending on that distinction require an assumption or sensitivity analysis.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A dropout model predicts response well using the last recorded symptom score. Does this establish MAR? State the distinction.
- Within a fixed recorded history, suppose dropout probabilities are 0.20 for low unseen outcomes and 0.60 for high unseen outcomes. Is the mechanism MAR for that history?
- In a hypothetical group of 80 participants with a poor recorded history, 32 drop out. What is the observed dropout proportion, and what does it establish?
- Two hypothetical completions have identical observed outcomes and response indicators but assign different outcomes to dropouts. What does this imply about using observed prediction alone to identify MAR?
Solutions. (1) No. It shows that recorded symptom history predicts response; MAR additionally requires response to be independent of the missing outcome conditional on that history. (2) No. The response probabilities differ by the unseen outcome within the same history, so the stated conditional-independence condition fails. (3) The proportion is \(32/80=0.400\), or 40%. It describes dropout in that recorded-history group; it does not establish whether dropout depends on unseen outcomes within the group. (4) Observed prediction alone cannot distinguish those completions. The MAR claim requires an assumption about the unobserved outcomes or additional information, not merely accurate prediction of observed response.