Why Missing Outcomes Can Change an Exposure-Response Conclusion
An exposure-response estimate is only as representative as the outcome data used to estimate it. If participants with poor response, high exposure, or adverse events are more likely to stop attending visits, observed outcomes may disproportionately describe participants who tolerated treatment or did well. A complete-case analysis can then give a misleading picture of the exposure-response relationship.
Dropout does not automatically imply bias. The key question is whether the probability that an outcome is missing depends on information that is unobserved, after accounting for the observed history and variables in the analysis. This distinction determines which methods are defensible and which assumptions need sensitivity analysis.
These labels describe assumptions about the data-generating process, not patterns that can usually be established from observed data alone. For example, dropout associated with a low outcome measured at the previous visit can be MAR if that outcome is included in the model. Dropout associated with the outcome that would have been measured after dropout is generally MNAR unless the dependence is explained by observed information.
How Dropout Alters the Exposure-Response Estimate
Suppose the target is the mean outcome at a later visit across the population enrolled at baseline, or the exposure-response coefficient for that population. If participants with a particular exposure-response history are selectively missing, the observed sample at that visit represents a selected subset. A regression among completers can therefore estimate a relationship for that subset rather than the intended population. Conditioning on completion can also induce or distort associations when both exposure and outcome history predict completion.
A useful causal and statistical diagnostic is to describe the dropout process over time: which participants leave, when they leave, and what observed information predicts leaving. Relevant predictors may include the previous outcome, current or cumulative exposure, treatment interruption, and recorded adverse events. An adverse event may predict both discontinuation and a subsequent efficacy outcome; omitting it from the missingness model can leave dropout dependent on observed history not accounted for in the analysis. Including it does not, by itself, make an MNAR process MAR.
Worked Example: Complete-Case Bias After Poor Prior Response
Consider a hypothetical study with 100 participants. At the final visit, the 40 participants whose previous measured response was poor have a mean final response of 4 units; all 40 drop out before that visit. The remaining 60 participants have a mean final response of 10 units, and all are observed. For illustration, suppose response is measured in units where larger values indicate greater benefit. What does the final-visit complete-case mean estimate, and how does it compare with the enrolled-population mean?
The complete-case mean is the mean among the 60 observed participants:
If the final outcomes for the 40 dropouts are as stated, the enrolled-population mean is
Thus the complete-case mean is 2.40 units higher than the enrolled-population mean in this constructed example. Dropout is predictable from the observed previous response, so an analysis that conditions appropriately on that history can be MAR. The difference does not prove that the complete-case estimate is biased in every trial; it shows how it can fail to represent the intended population when dropout selects participants by outcome history.
Likelihood-Based Analysis Under MAR
Likelihood-based longitudinal models use the observed outcomes and their modeled joint distribution. Under MAR and parameter distinctness—the outcome-model parameters are not constrained by the missingness mechanism—the missingness process can be ignored in likelihood inference for the outcome parameters. This is often called an ignorable missingness analysis. It is not a claim that missing outcomes have been recovered without assumptions: the result also requires a correctly specified outcome model and appropriate handling of covariates and exposure.
For example, a repeated-measures model can specify the distribution of outcomes over visits conditional on exposure and other predictors. A participant with some observed and some missing visits contributes the likelihood of the observed components of that outcome vector. The method uses the modeled associations among visits; it does not simply substitute a participant's observed mean for missing values.
The role of observed history can be seen in a simple derivation. Suppose a later outcome \(Y_2\) and a prior outcome \(Y_1\) are jointly normally distributed, with means \(\mu_1,\mu_2\), variances \(\sigma_1^2,\sigma_2^2\), and correlation \(\rho\). Then
If dropout depends on \(Y_1\), but not further on \(Y_2\) after conditioning on \(Y_1\), the expected missing \(Y_2\) values differ across observed \(Y_1\) values. A model using the joint outcome distribution can account for this association under MAR. If dropout also depends on the unobserved \(Y_2\) itself after conditioning on \(Y_1\), this conditional expectation alone does not characterize the missing outcomes; an MNAR assumption or sensitivity analysis is needed.
Worked Example: Using Prior Outcomes Under MAR
In a hypothetical two-visit study, suppose the prior response \(Y_1\) has mean 5 units and standard deviation 2 units, the later response \(Y_2\) has mean 8 units and standard deviation 3 units, and their correlation is 0.60. A participant with observed \(Y_1=3\) units misses the later visit. Under the specified bivariate normal model and MAR conditional on \(Y_1\), calculate the model-based conditional mean for that participant's \(Y_2\).
The conditional-mean formula is
The 6.20-unit value is a conditional expectation under the stated model, not a known replacement for the participant's unobserved outcome. A likelihood analysis uses the conditional distribution and uncertainty, rather than treating 6.20 as error-free observed data. The MAR assumption is plausible here only if the dropout probability has no remaining dependence on \(Y_2\) after conditioning on \(Y_1\) and the other modeled observed history.
Inverse-Probability Weighting
Inverse-probability weighting (IPW) reweights observed outcomes to represent participants with similar observed histories who are less likely to remain observed. At a particular visit, let \(\pi_i\) be the modeled probability that participant \(i\)'s outcome is observed, conditional on measured history. An observed outcome receives weight \(1/\pi_i\). Longitudinal analyses use probabilities of remaining observed through the relevant visit, often constructed from conditional probabilities at each visit.
Weighting can be useful when the observation model is easier to specify than a complete longitudinal outcome distribution. It can also be combined with an outcome model, but such combinations do not remove the need to inspect assumptions. Large weights indicate that a small number of observed participants stand in for many others, making estimates unstable. Weight truncation can reduce variance, but changes the estimator and may introduce bias; it should be reported and examined in sensitivity analyses.
Worked Example: A Weighted Mean After Exposure-Related Dropout
In a hypothetical trial, suppose 20 observed participants with lower prior exposure each have a modeled observation probability of 0.80 and a mean final outcome of 6 units. Ten observed participants with higher prior exposure each have an observation probability of 0.50 and a mean final outcome of 10 units. Calculate the IPW mean using the group summaries.
The lower-exposure group has weight \(1/0.80=1.25\) per participant, for total weight \(20(1.25)=25\). The higher-exposure group has weight \(1/0.50=2\) per participant, for total weight \(10(2)=20\). The weighted outcome total is
The sum of weights is \(25+20=45\), so
The unweighted mean among these 30 observed participants is \(\{20(6)+10(10)\}/30=7.33\) units. The weighted estimate is higher because the higher-exposure group, with a lower probability of observation, receives greater weight. This calculation does not establish that 7.78 is the true population mean: it relies on correct observation probabilities, adequate positivity, and the assumption that measured history captures the relevant selection.
Pattern-Mixture Sensitivity Analysis
A pattern-mixture analysis groups participants by their observed missingness pattern—for example, by the visit at which they discontinued—and models outcomes within each pattern. Because the distribution of unobserved outcomes is not determined by observed data under MNAR, an analyst must specify how missing outcomes differ from predictions under a reference model. A common sensitivity parameter, \(\delta\), shifts the predicted mean for missing outcomes by a chosen amount. The analysis is repeated across plausible values, including values that represent less favorable outcomes among dropouts.
The shift is not estimated from the observed data alone. Its purpose is to make the untestable MNAR assumption explicit and show how conclusions change when that assumption changes. A range of \(\delta\) values should be justified clinically or scientifically, rather than chosen only to preserve a preferred result.
Worked Example: Delta Adjustment for Missing Outcomes
Suppose a hypothetical trial has 80 participants with observed final outcomes averaging 8 units and 20 participants who discontinued. Under a reference MAR model, the 20 missing outcomes are predicted to average 7 units. Consider an MNAR sensitivity assumption that their mean is 2 units lower than that prediction, so \(\delta=-2\) units. Calculate the adjusted enrolled-population mean.
The delta-adjusted mean among the missing participants is \(7+(-2)=5\) units. Combining observed and adjusted missing outcomes gives
With \(\delta=0\), the reference MAR calculation would give \(\{80(8)+20(7)\}/100=7.80\) units. Thus this specified two-unit downward shift changes the mean by 0.40 units. It does not show which mean is correct; it quantifies the consequence of one transparent departure from MAR.
Choosing and Checking an Approach
| Approach | Key assumption | Strength and limitation |
|---|---|---|
| Likelihood-based outcome model | MAR, distinct parameters, and a correctly specified outcome model | Uses observed outcome patterns and modeled within-participant associations; sensitive to outcome-model misspecification and does not identify MNAR behavior on its own. |
| Inverse-probability weighting | Observation probabilities correctly modeled, conditional exchangeability, and positivity | Directly reweights observed participants; can be unstable when probabilities are small or estimated weights are extreme. |
| Pattern-mixture sensitivity analysis | A specified, interpretable assumption for outcomes within missingness patterns | Makes MNAR departures explicit; conclusions depend on sensitivity parameters not identified by observed data. |
Diagnostics cannot prove MAR, but they can reveal weaknesses in an analysis. Compare dropout rates across exposure and observed-outcome histories; inspect whether adverse events or treatment discontinuations predict missingness; examine the distribution of estimated observation probabilities and the largest weights; and check whether outcome-model residuals and predicted trajectories are credible for observed participants. If exposure or history strata have virtually no observed outcomes, positivity is doubtful and neither weighting nor model extrapolation can reliably recover those strata without stronger assumptions.
A common mistake is to reason, “Dropout is related to the previous outcome, so the data are MNAR and complete cases are unusable.” The correction is that dependence on an observed previous outcome can be MAR if that history is adequately included in the analysis. The opposite mistake is to assume that adding the previous outcome automatically solves informative dropout. If dropout still depends on an unobserved future outcome, the process remains MNAR. Observed data alone generally cannot distinguish these possibilities, which is why sensitivity analysis matters.
No method is universally best. Likelihood methods are attractive when the longitudinal outcome model is credible; weighting is attractive when the observation model is credible and probabilities are not too small; pattern-mixture analyses communicate how conclusions depend on unverifiable MNAR assumptions. In practice, a primary MAR analysis can be paired with one or more clinically justified MNAR sensitivity analyses. Report the estimand, the variables used to model outcomes and observation, the assumptions, and how conclusions vary across the sensitivity range.
Check Your Understanding
Answer each question, then compare with the solutions.
- Dropout depends on the last measured response, which is included in the outcome model, and not on the unobserved next response after conditioning on measured history. Which missingness assumption is compatible with this description?
- At a visit, three observed participants have outcomes of 4, 7, and 10 units, with observation probabilities 1.00, 0.50, and 0.25. Calculate the normalized IPW mean.
- Why can very small estimated observation probabilities make an IPW analysis unreliable?
- A pattern-mixture analysis sets \(\delta=-1.5\) units for missing outcomes relative to MAR predictions. What does this mean, and what does it not mean?
- Can observed data alone establish that dropout is MAR rather than MNAR? Explain briefly.
Solutions. (1) MAR is compatible with the description because dropout depends on observed history, with no remaining dependence on the unobserved outcome conditional on that history. (2) The weights are 1, 2, and 4. The weighted mean is \(\{1(4)+2(7)+4(10)\}/(1+2+4)=58/7=8.29\) units. (3) A small probability produces a large inverse weight, so one or a few observed participants can dominate the estimate; variance rises and estimates become sensitive to model error. (4) It assumes missing outcomes average 1.5 units below their MAR predictions for the specified pattern. The value is a sensitivity assumption, not an estimate identified from observed data. (5) No. MAR and MNAR can imply the same distribution of observed data, so the distinction requires assumptions and sensitivity analysis.