Sensitivity to Unobserved Outcomes After Dropout

Why Sensitivity to Unobserved Outcomes Matters

When participants leave follow-up, their later biomarker values are not recorded. The mean among participants who remain can still be calculated, but it need not describe the trajectory of the original cohort. As the earlier material on observed follow-up and dropout emphasizes, the people still observed may differ from those no longer contributing measurements. The size and direction of that difference cannot generally be learned from the observed later measurements alone.

A sensitivity analysis makes this uncertainty explicit. Instead of treating one unverifiable assumption about missing outcomes as known, it asks how conclusions would change under a range of plausible hypothetical values. The purpose is not to reconstruct each person's actual unobserved value. It is to show how much a cohort-level summary depends on assumptions about the outcomes that dropout concealed.

This distinction requires care. The examples below concern nonterminal dropout while the biomarker remains conceptually defined. After a terminal event such as death, a later biomarker may not be a missing value at all: it may be undefined. Treating such an outcome as if it could simply be imputed would change the scientific question.

A Simple Pattern-Mixture Sensitivity Model

Fix a follow-up time \(t\). Let \(q(t)\) be the proportion of the original cohort whose biomarker is unobserved at that time, and let \(m_{\mathrm{obs}}(t)\) be the mean among those observed. Let \(m_{\mathrm{mis,MAR}}(t)\) be a benchmark mean for those missing, predicted from their observed information under a missing-at-random (MAR) assumption. In this setting, MAR means that, conditional on the observed information used in the model, the probability of a missing outcome does not depend on its unobserved value.

The benchmark is an assumption, not something established by the observed data. A pattern-mixture sensitivity analysis shifts the benchmark missing-outcome mean by a chosen amount \(\delta(t)\), measured in the biomarker's units:

A negative \(\delta(t)\) represents a missing group whose average is lower than the benchmark; a positive value represents a higher average. It is a shift in the group mean, not a claim that every missing participant has exactly the same individual-level change. Combining observed and missing groups gives the sensitivity-adjusted cohort mean:

For a fixed benchmark, changing \(\delta(t)\) by one biomarker unit changes the cohort mean by \(q(t)\) units. Thus the same departure matters more when a larger fraction of the cohort is missing. In practice, the benchmark may come from a model using observed histories, and the missing fraction and group means may be estimated. The arithmetic below treats them as known to keep attention on the sensitivity calculation; it does not represent sampling uncertainty.

From a Single-Time Mean to a Trajectory

A cohort trajectory is a sequence of cohort means across time. Sensitivity assumptions can be specified separately at each time, or summarized by a pattern such as a constant shift over follow-up. A constant \(\delta\) is convenient for illustration, but it is not automatically plausible: the gap between missing and benchmark outcomes could grow, shrink, or change direction over time. Any trajectory comparison must apply assumptions consistently at the times being compared.

For a two-time-point illustration, suppose the baseline biomarker is observed for the full cohort. The change in the cohort mean is the adjusted follow-up mean minus the baseline mean. This change describes a difference in population averages over time; it does not imply that every participant changed by that amount.

Making the Assumptions and Checks Explicit

Before interpreting a sensitivity calculation, verify that its target and inputs are coherent. The biomarker must be defined for the people and times included in the cohort mean; dropout status and the denominator must be clear; the observed measurements must be comparable across participants; and the benchmark for missing outcomes must be described. If that benchmark uses a prediction model, its conditional assumptions and included history matter. A shift \(\delta\) changes the assumption about missing outcomes; it does not fix a poorly chosen benchmark.

Useful checks include confirming that the missing fraction and group weights sum to one, checking that assumed means remain within scientifically possible values, and plotting adjusted means across the chosen \(\delta\) range. If biomarker values are bounded, a scenario that implies a group mean outside those bounds is not plausible. If dropout proportions or dropout patterns change sharply over time, a single constant shift may conceal important variation. These checks can expose an incoherent scenario, but they cannot reveal the unobserved outcomes.

What Sensitivity Analysis Can and Cannot Establish

Sensitivity analysis is most informative when the assumptions are stated in recognizable units and the range is justified using subject-matter knowledge, earlier measurements, or scientifically meaningful limits. Reporting only the MAR result can make an unverifiable assumption look like a data-driven certainty. Reporting a range shows whether a conclusion is robust to departures from that assumption.

The method has important limits. Observed data alone generally cannot identify the mean of outcomes that remain unobserved under nonrandom dropout. A selected set of \(\delta\) values does not turn those unknown outcomes into known data, and a conclusion that persists over a narrow or implausible range is not evidence of robustness over all possible departures. Sensitivity scenarios also do not remove ordinary sampling error. In a full analysis, uncertainty in estimated observed means, dropout proportions, benchmark predictions, and the sensitivity-adjusted summary should be reflected in intervals or other uncertainty measures.

Several approaches answer related but different questions. A complete-case mean describes those observed and does not, by itself, recover the original-cohort mean. A MAR-based analysis estimates missing outcomes using observed information but depends on the MAR assumption. Sensitivity analysis varies that assumption to assess robustness. Bounds can instead report a range of possible summaries under explicit outcome limits, sometimes at the cost of being wide. These are alternatives or complements, not interchangeable guarantees. A sensitivity analysis is useful precisely because it communicates what the data do not determine.

Common Pitfall

A related error is to describe a sensitivity scenario as though it were a finding about what actually happened after dropout. The accurate interpretation is conditional: if the missing-group mean were a specified amount below the benchmark, then the cohort mean would take the calculated value. That conditional statement is useful, but it is not evidence that the assumed post-dropout values occurred.