Simulating Dropout and Informative Missingness

Why Simulate Missingness Explicitly?

A simulated trial may generate every participant’s outcome at every planned visit, but a real trial may not observe all of those outcomes. The reasons matter. A participant may have no opportunity to complete a visit because data collection ended at a fixed cutoff; may drop out for reasons unrelated to outcome; or may be more likely to miss a visit because their current or future outcome is poor. These mechanisms can produce different observed data, even when the underlying treatment effect is identical.

For trial simulation, it is useful to generate the full outcome history first and then apply a specified observation process. This separates the scientific data-generating model from the process that determines which data are available to an analysis. Comparing full-data results with observed-data results helps reveal both loss of precision and potential bias.

Representing the Observation Process

Let \(Y_{ij}\) denote the outcome for participant \(i\) at planned visit \(j\), and let \(R_{ij}=1\) if that outcome is observed and \(R_{ij}=0\) otherwise. For administrative censoring, define \(A_{ij}=1\) when visit \(j\) occurs on or before the applicable data cutoff, and \(A_{ij}=0\) otherwise. For dropout, define \(D_{ij}=1\) if the participant remains under follow-up through visit \(j\), and \(D_{ij}=0\) after dropout. The combined observation indicator can be written as:

This representation makes the mechanisms distinct. A participant whose visit falls after the cutoff has \(A_{ij}=0\), even if they would otherwise have stayed in the study. A participant who drops out before the cutoff has \(D_{ij}=0\) for subsequent visits. In a dropout process that is absorbing, once \(D_{ij}=0\), it remains zero at all later visits.

For a fixed endpoint time, administrative censoring can be implemented by generating enrollment dates and a study cutoff, then marking the endpoint unavailable for participants whose scheduled endpoint occurs after the cutoff. In a longitudinal trial, dropout can be simulated at each interval. For example, let \(h_{ij}\) be the probability of dropout between visits \(j-1\) and \(j\), conditional on remaining in follow-up through visit \(j-1\). Draw a dropout indicator with probability \(h_{ij}\); if dropout occurs, set all subsequent \(D_{ij}\) values to zero.

The dropout probability can be constant, vary by visit, or depend on participant information. A constant probability is a simple independent-dropout scenario. To represent dropout depending on observed history, one might specify \(h_{ij}\) as a function of treatment assignment and the participant’s most recent observed outcome. To represent outcome-dependent missingness, the simulation can instead make dropout depend on the current outcome, including an outcome that will not be observed. That is a deliberate assumption about the data-generating process, not something directly testable from the observed outcomes alone.

Independent and Outcome-Dependent Dropout

A common classification distinguishes missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR). Under MCAR, observation does not depend on observed or unobserved outcomes or other participant characteristics. Under MAR, after conditioning on specified observed information, missingness does not additionally depend on the unobserved outcome. Under MNAR, missingness still depends on the unobserved outcome after conditioning on the observed information used to describe the mechanism.

These labels describe assumptions about the relationship between data and missingness; they do not identify the cause of dropout. For example, a dropout probability based on a previously observed symptom score is compatible with MAR if that observed history is included in the conditioning information. A probability based on the current unobserved symptom score creates an MNAR mechanism. In a simulation, both are implementable because the full simulated outcome history is available before outcomes are hidden.

Under MCAR, complete-case analysis—an analysis using only participants with observed outcomes—does not systematically select higher or lower outcomes on average, though it loses precision. Under MAR, complete-case analysis may be biased if observation depends on observed factors related to the outcome; an analysis that appropriately models the observed data can use those factors. Under MNAR, conclusions generally require additional assumptions or sensitivity analyses because the observed data alone do not determine the distribution of unobserved outcomes.

The equality follows from \(E(YR)=\operatorname{Cov}(Y,R)+E(Y)E(R)\) and \(E(R)=P(R=1)\). Therefore, if observation is independent of \(Y\), the covariance is zero and the observed mean equals the full-data mean in expectation. If observation probability increases or decreases with \(Y\), the covariance need not be zero, and the observed mean can differ systematically from the full-data mean.

From Missingness to Decision Rates

A decision rate is the proportion of simulated trial replicates in which a prespecified decision rule is met. Missingness can change that rate by reducing the information available, shifting an estimator, or both. To see the precision effect, consider a simple hypothetical one-sided normal decision rule: declare evidence of a positive treatment effect if the estimated difference exceeds \(1.645\) standard errors. Assume independent normally distributed outcomes with variance 4 units squared in each arm, equal allocation, and a known variance for this calculation.

Implementation, Diagnostics, and Alternatives

A transparent simulation applies the observation mechanism after generating outcomes. This makes it possible to retain a full-data benchmark and to check whether the simulated missingness matches its intended specification.

Useful diagnostics include the fraction observed at each visit, dropout timing, rates by treatment arm and relevant baseline or observed-outcome groups, and the distribution of follow-up time. For outcome-dependent scenarios, also inspect observation rates across the simulated full outcome values; this check is available to the simulator even though the missing outcomes would be unavailable in a real analysis. Verify that no participant has later observed visits after an absorbing dropout, and that administrative censoring follows the cutoff dates exactly.

The main alternatives are not interchangeable. Complete-case analysis is straightforward and can be unbiased under suitable independence conditions, but may discard substantial information or be biased under selective observation. Likelihood-based analyses of observed longitudinal data can use incomplete outcome histories under a correctly specified outcome model and an appropriate MAR assumption. Multiple imputation similarly relies on a defensible model for the missing outcomes and the information related to missingness. For MNAR scenarios, sensitivity analyses can vary assumptions about unobserved outcomes; no single analysis can recover those outcomes from observed data without assumptions.

A common mistake is to assume that random dropout only reduces sample size, so the treatment-effect estimate must remain unbiased. That conclusion requires dropout to be independent of the outcome relevant to the estimator, or other conditions that justify the analysis. If participants with worse unobserved outcomes are more likely to drop out, the observed outcomes can be systematically selected, shifting the estimate as well as increasing uncertainty. Another pitfall is to simulate dropout separately by arm but ignore its dependence on outcomes within each arm: equal dropout percentages do not guarantee equal bias.