Why Stratify a Dropout Sensitivity Analysis?
In a randomized asthma trial, participants with more severe symptoms at baseline may be more likely to stop attending follow-up. Dropout may also differ between treatment arms, perhaps because of different symptom experiences or tolerability. If the analysis ignores baseline severity, the observed outcomes can represent different mixtures of severity across arms. A comparison of observed participants may then differ from a comparison standardized to the trial’s baseline severity distribution.
Stratifying by baseline severity helps make this composition visible. It does not, however, reveal the unobserved outcomes of participants who dropped out. Within every arm and severity stratum, their outcomes may still differ from those of participants who remained. A pattern-mixture sensitivity analysis makes that uncertainty explicit by specifying how the missing-outcome distribution might depart from the observed-outcome distribution.
Throughout, suppose the outcome is an end-of-follow-up asthma symptom score, with higher scores indicating worse symptoms. Let \(A\) denote treatment arm, \(S\) baseline-severity stratum, and \(R=1\) indicate that the end-of-follow-up score is observed. The treatment contrast is the mean score in arm A minus the mean in arm B, so a negative contrast favors A on this outcome.
An overall analysis instead uses an arm-level observed mean \(\bar{Y}^{obs}_{a}\) and an arm-level shift \(\delta_a\), without separating baseline severity. It assumes the missing-outcome mean in arm \(a\) is \(\bar{Y}^{obs}_{a}+\delta_a\). Neither the overall nor the stratified shift is identified from observed outcomes alone. The values must be chosen as sensitivity assumptions, informed where possible by clinical judgment and relevant external evidence.
Estimating an Arm Mean Under a Stratified Shift
Let \(N_{as}\) be the number randomized to arm \(a\) and severity stratum \(s\), \(n^{obs}_{as}\) the number with observed outcomes, and \(r_{as}=n^{obs}_{as}/N_{as}\) the observed fraction. If the observed mean estimates the mean for the observed participants in that cell, and the assumed missing mean is \(\bar{Y}^{obs}_{as}+\delta_{as}\), the cell mean is estimated by weighting the two missingness groups by their cell proportions:
The equality follows by expanding the second expression: the coefficient on \(\bar{Y}^{obs}_{as}\) is \(r_{as}+1-r_{as}=1\). The shift affects the estimated cell mean in proportion to the fraction missing. If no one is missing, the shift has no effect; if everyone is missing, this formula cannot provide an estimate because the observed cell mean is unavailable.
To compare arms for a common target population, standardize both arms to the same baseline-severity weights \(w_s\), where the weights sum to one. For the hypothetical trial below, each severity stratum represents half the randomized participants, so \(w_s=0.5\) for both strata. The standardized arm estimate and contrast are:
Using the same weights prevents the two arm means from being standardized to different baseline-severity mixtures. If the trial’s target population has a different severity distribution, its prespecified target weights should be used instead.
Worked Example: Locating Differential Dropout
Hypothetical data. Each treatment arm randomized 50 participants with mild baseline severity and 50 with severe baseline severity. In arm A, 5 mild and 20 severe participants dropped out. In arm B, 10 mild and 30 severe participants dropped out. Calculate dropout rates by arm and stratum.
For mild participants in A, the rate is \(5/50=0.10\), or 10%. For mild participants in B, it is \(10/50=0.20\), or 20%. For severe participants in A, it is \(20/50=0.40\), or 40%; for severe participants in B, it is \(30/50=0.60\), or 60%.
Overall, arm A has \(5+20=25\) dropouts among 100 randomized participants, giving \(25/100=25\%\). Arm B has \(10+30=40\) dropouts among 100, giving \(40/100=40\%\). Dropout is higher in B in both strata, and the mild-to-severe increase is substantial in each arm. These rates describe the observed follow-up process; by themselves, they do not establish why participants dropped out or what their unobserved scores would have been.
Overall and Severity-Stratified Sensitivity Results
Suppose the observed end-of-follow-up means are 4 for mild A participants, 10 for severe A participants, 5 for mild B participants, and 11 for severe B participants. The counts and means are hypothetical and chosen to make the calculations transparent.
Worked Example: An Overall MNAR Sensitivity Analysis
First calculate the observed mean in each arm, pooling across baseline severity. Arm A has 45 observed mild participants and 30 observed severe participants, so:
Arm B has 40 observed mild participants and 20 observed severe participants:
The overall observed-mean contrast is \(6.40-7.00=-0.60\) points. Now set the overall shifts to \(\delta_A=1.80\) and \(\delta_B=2.75\) points. These hypothetical choices are the dropout-count-weighted averages of the severity-specific shifts used in the next example. Under the overall model, every missing participant in an arm is assigned a mean shifted from that arm’s pooled observed mean.
Arm A has 25% missing, so its sensitivity estimate is \(6.40+(0.25)(1.80)=6.85\) points. Arm B has 40% missing, giving \(7.00+(0.40)(2.75)=8.10\) points. The overall-model contrast is therefore \(6.85-8.10=-1.25\) points. Under these assumptions, the estimated mean symptom score is 1.25 points lower in A than in B. This is a sensitivity result, not a conclusion supported by the observed outcomes alone.
Worked Example: A Severity-Stratified MNAR Analysis
Now specify shifts separately by arm and baseline severity. Suppose the missing-outcome mean is assumed to be 1 point worse than the observed mean for mild A participants and 2 points worse for severe A participants. For B, assume shifts of 2 points for mild and 3 points for severe participants. These values are illustrative assumptions, not estimates from the observed scores.
In mild A, \(45/50\) outcomes are observed, so the estimated cell mean is \(4+(5/50)(1)=4.10\). In severe A, it is \(10+(20/50)(2)=10.80\). Standardizing to equal baseline-severity weights gives:
In mild B, the estimated cell mean is \(5+(10/50)(2)=5.40\). In severe B, it is \(11+(30/50)(3)=12.80\). Therefore:
The stratified MNAR contrast is \(7.45-9.10=-1.65\) points. With these particular shifts and weights, the standardized estimated score is 1.65 points lower in A. The contrast differs from the overall-model result of \(-1.25\) points because the overall model uses pooled observed means as its reference, whereas the stratified model preserves severity-specific reference means and applies shifts within those strata.
For comparison, setting all severity-specific shifts to zero gives standardized means \(0.5(4)+0.5(10)=7.00\) in A and \(0.5(5)+0.5(11)=8.00\) in B, for a contrast of \(-1.00\) point. This reference calculation still assumes that, within each arm and severity stratum, missing participants have the same mean as observed participants. The equality is an assumption; stratifying does not make it empirically certain.
What the Stratification Does—and Does Not—Establish
The observed arm-level means in the overall example are based on different severity mixtures: \(45/75=60\%\) of observed A participants are mild, compared with \(40/60=66.7\%\) of observed B participants. Since severe participants have higher scores, this imbalance contributes to the pooled observed comparison. Stratifying and using common baseline weights removes that particular composition difference from the standardized contrast.
But severity stratification addresses only the measured baseline variable and only at the level represented by the chosen strata. Within a stratum, participants who drop out could still have worse symptoms than those who remain, or dropout could depend on other measured or unmeasured factors. Thus, the data can show that dropout varies by arm and baseline severity, but cannot show that outcomes are missing at random within those strata. A zero shift is a reference scenario, not proof of ignorability.
A careful assessment should check that the baseline-severity definition is available for all randomized participants, that each arm has observed participants in every stratum used, and that the observed means and dropout counts are reported cell by cell. Empty or very small cells make the observed reference means unstable or unavailable. It is also useful to inspect whether the chosen strata are sufficiently informative clinically; a crude mild/severe split can leave meaningful variation within each category.
Sensitivity parameters should be varied over clinically interpretable values, ideally with reasons for the range stated. Report the observed-data reference result and the results under the alternative shifts, with symptom-score units. The point estimates above do not provide standard errors or confidence intervals. In a full analysis, uncertainty from sampling and the estimation procedure must also be addressed; a sensitivity shift does not itself quantify that uncertainty.
Choosing Among Approaches and Recognizing Failure Modes
| Approach | What it represents | Key limitation |
|---|---|---|
| Overall pattern-mixture shift | One observed-mean reference and one missing-outcome shift per arm | Can conceal severity composition and assume a common shift across severity levels |
| Severity-stratified pattern-mixture shift | Observed references and sensitivity shifts are specified within arm and severity cells, then standardized | Requires adequate data in each cell and does not rule out residual MNAR within cells |
| MAR-based analysis | Uses an assumption that missing outcomes are adequately predicted from observed information | Observed follow-up data cannot verify the assumption for unobserved outcomes |
These approaches answer different questions. The overall and stratified models need not agree, even when their sensitivity values are chosen with care, because they use different reference distributions. A large discrepancy is a reason to investigate the role of composition and assumptions, not a reason to select whichever answer is more favorable.
Common warning signs include an arm with much higher dropout, sharply different dropout rates across severity strata, sparse observed cells, and conclusions that change substantially across plausible shift choices. These patterns do not identify the correct MNAR model. They do indicate that a single pooled estimate may hide important structure and that the sensitivity assumptions should be made visible.
Check Your Understanding
Answer each question, then compare with the solutions below.
- In a severity cell with 40 randomized participants, 30 observed outcomes, an observed mean of 8 points, and an assumed shift of 2 points, what is the estimated cell mean?
- Why should both treatment arms use the same severity weights for a standardized comparison?
- In the hypothetical trial, what is the severity-stratified contrast when all shifts are zero?
- Does observing higher dropout in severe participants prove that dropout depends on the unobserved symptom score? Explain.
- Name one data feature that would make a severity-stratified analysis difficult to trust, and explain why.
Solutions. (1) The observed fraction is \(30/40=0.75\), so the estimated cell mean is \(8+(1-0.75)(2)=8.50\) points. (2) Common weights compare the arms over the same baseline-severity distribution; different weights would mix treatment differences with differences in the populations being averaged. (3) The zero-shift standardized means are 7.00 in A and 8.00 in B, so the contrast is \(7.00-8.00=-1.00\) point. (4) No. The dropout rates describe who is observed by baseline severity, but the unobserved symptom scores are not available to establish their relationship with dropout. (5) For example, very few observed participants in a severity-by-arm cell make its observed mean unstable; an empty cell leaves no observed reference mean for the stated stratified formula.