Why Missing Outcomes Affect a Sample-Size Update
A sample-size target usually describes the amount of information needed to answer the trial’s primary question. If some randomized participants will not provide an outcome, the number enrolled may need to exceed the number of analyzable outcomes assumed in the original calculation. The challenge at an interim update is to estimate that loss without confusing it with a change in what the trial is trying to estimate.
The simple calculation is valid only under specific assumptions. In particular, a retention rate can translate recruited participants into expected observed outcomes, but it cannot by itself correct bias when outcome availability depends on the unobserved outcome. Nor can it make an outcome defined only before treatment discontinuation equivalent to an outcome defined regardless of discontinuation.
The target estimand should be specified before deciding how missing outcomes affect the sample-size calculation. An estimand states the treatment comparison the trial seeks to estimate, including the target population, treatment conditions, outcome variable, handling of events such as treatment discontinuation, and summary measure. Missingness describes absent data; it does not automatically redefine that target.
Information Loss Under a Simple Planning Model
Suppose the primary outcome is continuous and the planned comparison is the difference between treatment-arm means, \(\delta=\mu_1-\mu_0\). Assume independent randomized participants, a common outcome assessment, and outcome variances \(\sigma_1^2\) and \(\sigma_0^2\). For a two-sided test at level \(\alpha\) with target power \(1-\beta\), a normal-approximation calculation requires approximately
where \(n_j\) is the number with observed outcomes in arm \(j\), and \(z_u\) is the \(u\)-quantile of the standard normal distribution. If outcomes are observed with probabilities \(R_1\) and \(R_0\), the corresponding planning approximation for recruited sample sizes \(N_1,N_0\) is
This substitution treats the expected observed counts as the effective sample sizes. It is a planning approximation: actual observed counts vary, and a trial should normally round recruitment targets upward and allow for operational uncertainty. Under equal allocation, equal variances, and a common retention probability \(R\), the required recruited sample per arm is approximately the complete-outcome target divided by \(R\). For example, retention of 0.80 implies an inflation factor of \(1/0.80=1.25\), or a 25% increase.
Worked Examples
Worked Example: Updating a Continuous-Outcome Recruitment Target
Hypothetical planning scenario. A two-arm trial compares mean symptom scores, with lower scores favorable. The target difference is 4 points, the standard deviation is assumed to be 10 points in each arm, the two-sided significance level is 0.05, and target power is 90%. The original calculation assumes complete outcome ascertainment. An interim review estimates that 85% of randomized participants in each arm will have the primary outcome observed. Find the revised target.
With equal complete-outcome sample sizes \(n\) per arm, the normal approximation gives
Round up to 132 participants with observed outcomes per arm. At retention \(R=0.85\), the recruitment target per arm is
The revised target is 156 randomized participants per arm, or 312 in total. This calculation assumes the 85% estimate applies similarly to future participants and that missingness does not make the observed outcomes a biased basis for the target comparison. It does not promise that exactly 132 outcomes will be observed.
Worked Example: Arm-Specific Retention and a Precision Check
Hypothetical planning scenario. An ongoing trial has 50 observed outcomes in the control arm and 60 in the treatment arm. Investigators consider randomizing 10 more participants to each arm. Based on interim retention of 80%, they expect 8 of those 10 per arm to have outcomes observed. The outcome variance is assumed to be 64 points squared in both arms. The target treatment difference is 3 points, and the normal critical-value sum for the desired test and power is 2.8. Is the proposed addition sufficient?
After the proposed addition, the expected observed counts are \(50+8=58\) in control and \(60+8=68\) in treatment. The approximate variance of the estimated difference is
To meet the specified target, the variance must be no greater than
Because \(2.0446>1.148\), adding 10 participants per arm is not sufficient. The treatment and control retention rates do not need to be equal for this check: the relevant quantities are the expected observed counts in each arm. The result is a planning calculation, not a guarantee about realized sample sizes.
Worked Example: When More Recruitment Does Not Fix the Target
Hypothetical scenario. Suppose the target is a treatment-policy comparison of mean outcomes at 12 weeks, including outcomes after treatment discontinuation. Among treated participants whose 12-week outcomes are observed, the mean is 8 points. Twenty-five percent of treated participants have missing outcomes after discontinuing treatment. For illustration, suppose their mean 12-week outcome would be 12 points if it were observed. The full treated-arm mean would then be
Using only observed treated participants gives a mean of 8, not 9, under these hypothetical values. If the control mean is 10, the observed-outcome difference is \(8-10=-2\) points, while the target treatment-policy difference is \(9-10=-1\) point. Inflating recruitment may produce more observed outcomes, but a complete-case comparison can still converge on the wrong mean if missingness selects participants with different outcomes. The missing outcomes need an analysis strategy justified for the estimand and missingness assumptions; recruitment inflation alone is not that strategy.
Interim Estimation, Diagnostics, and Failure Modes
An interim retention estimate should use participants who have had a genuine opportunity to complete the outcome assessment. Counting everyone randomized so far as either “observed” or “missing” can misclassify participants whose assessment window is still open. Define the assessment window, maturity rule, and missingness denominator in advance. If retention differs by arm, use arm-specific estimates rather than a pooled rate that can conceal an operational or clinical difference.
Review why outcomes are missing as well as how many are missing. Withdrawals, missed visits, site closure, treatment discontinuation, and administrative end-of-study cutoffs may have different implications. Compare missingness across treatment arms, baseline prognostic factors, sites, and time. Such checks can reveal problems, but they cannot establish from observed data alone that missingness is unrelated to the unobserved outcomes.
The standard missingness terminology helps clarify the limitation. Missing completely at random (MCAR) means outcome availability is independent of both observed and unobserved data. Missing at random (MAR) means that, conditional on observed information included in an analysis, availability does not additionally depend on the missing outcome. Missing not at random (MNAR) means that dependence on the unobserved outcome remains after conditioning on observed information. A complete-case mean comparison is generally justified most simply under MCAR; MAR may support methods such as multiple imputation or likelihood-based analysis under their model assumptions. Under MNAR, sensitivity analyses are often needed. A larger sample does not turn an unjustified assumption into a valid one.
State which participants and outcomes define the treatment comparison, including the handling of treatment discontinuation and other intercurrent events.
Use sufficiently mature participants, report the denominator and assessment window, and inspect arm-specific patterns and reasons for missingness.
Substitute expected observed counts \(N_jR_j\) into the planning variance or the appropriate endpoint-specific calculation; round recruitment targets upward.
Confirm that the missing-data method targets the same estimand and that the adaptation remains compatible with the prespecified final test and error-control plan.
A useful diagnostic is to compare the interim estimate with the assumptions that drove the original calculation. If outcome variance among observed participants has changed, retention alone does not capture the information loss. If retention is still changing with time or differs sharply across sites, a single pooled estimate may poorly predict future ascertainment. If reasons for missingness suggest dependence on unobserved outcomes, an inflation factor is especially inadequate.
Keep Information Loss Separate From the Estimand
A missing outcome is not always the same as an outcome that cannot meaningfully be defined. For a treatment-policy estimand, an outcome after treatment discontinuation may still be part of the target; failure to collect it is missing data. For another estimand, the question might explicitly concern outcomes while participants remain on assigned treatment. That is a different target question, not merely a lower retention rate. Changing from one to the other can change the treatment effect being estimated, even if the sample size is unchanged.
The main alternatives to simple inflation address different issues. Sample-size inflation aims to maintain expected information when some outcomes are unavailable, assuming the analysis remains valid. Likelihood-based methods and multiple imputation use observed information to estimate the target under specified model and missing-at-random assumptions; they do not generally justify the simple complete-case inflation formula. Inverse-probability weighting reweights observed participants based on estimated observation probabilities, but can be unstable when those probabilities are small and relies on a correct observation model. Composite outcomes or other intercurrent-event strategies may define a different estimand rather than recover the original outcome.
| Approach | What it addresses | Key limitation |
|---|---|---|
| Recruitment inflation | Expected loss of observed outcomes | Does not remove selection bias or change the estimand appropriately |
| Likelihood or multiple imputation | Analysis with incomplete outcomes under an explicit model | Depends on model adequacy and assumptions such as MAR |
| Inverse-probability weighting | Unequal probabilities of outcome observation | Depends on the observation model and can be unstable |
| Different intercurrent-event strategy | A deliberately different clinical question | Changes the estimand; it is not simply a missingness adjustment |
Finally, sample-size re-estimation must not be treated as an automatic guarantee of valid inference. A procedure that changes recruitment based on interim information should follow the prespecified adaptation and final-testing plan. In particular, an update based on treatment-effect information can have different inferential consequences from one based on nuisance quantities such as outcome ascertainment. The sample-size calculation answers how many observations may be needed under stated assumptions; the trial’s design and analysis plan determine whether the final comparison retains its intended error properties.
Common Mistake
Wrong reasoning: “Twenty percent of participants are missing, so I will increase the sample size by 20%.” This confuses the missing fraction with the required inflation. If retention is 80%, the multiplier is \(1/0.80=1.25\), a 25% increase. A second, more consequential mistake is to assume that this increase corrects bias. It only addresses expected loss of information under a suitable model; informative missingness or a changed estimand requires a separate analysis and design decision.
Check Your Understanding
Try each problem, then compare your work with the solutions below.
- A trial requires 120 observed outcomes per arm. If expected retention is 80% in both arms, how many participants per arm should be recruited using simple inflation?
- In a continuous-outcome trial, each arm has outcome variance 64 and the target variance of the difference is at most 1.2. After a proposed update, expected observed counts are 58 and 68. Is the target met?
- Why might calculating the interim missing fraction using every randomized participant, including those whose outcome window is not complete, overstate missingness?
- A trial’s target is the mean outcome regardless of treatment discontinuation, but participants who discontinue often have unobserved outcomes. Explain why increasing recruitment does not necessarily make a complete-case estimate valid for the target.
Solutions. (1) Divide the required observed count by retention: \(120/0.80=150\). Recruit 150 per arm, a 25% increase over 120. (2) The projected variance is \(64/58+64/68\approx1.1034+0.9412=2.0446\), which exceeds 1.2, so the target is not met. (3) Participants whose assessment windows remain open have not yet had the opportunity to provide the outcome; counting them as missing treats pending data as unavailable and inflates the estimated missing fraction. (4) Discontinuation-related missingness may select participants whose outcomes differ from those observed. A larger complete-case sample can estimate the observed-participant mean more precisely without estimating the target mean; an analysis justified for the estimand and missingness assumptions is also required.