Information Lost to Missing Outcomes

Why Enrollment Does Not Guarantee Information

In a trial, randomization and enrollment establish how many participants entered each arm, but they do not ensure that the outcome of interest will be observed for everyone. If an outcome is missing, that participant generally contributes no direct information to an analysis based on that outcome. Counting every randomized participant as if the outcome were available can therefore overstate the precision the trial can achieve.

The central calculation is straightforward when missingness is independent of the outcome and the outcome model is otherwise correctly specified: replace the planned number of participants with the number expected to have observed outcomes, then calculate information from that number. This is a planning approximation, not a guarantee about the information in the realized data. It also addresses precision only; it does not make biased observed outcomes representative when missingness depends on outcome or prognosis.

Definition: Let \(n_j\) be the number enrolled in arm \(j\), and let \(q_j\) be the probability that a participant in that arm has the outcome observed. Under independent, outcome-independent ascertainment, the observed count \(M_j\) has \(E(M_j)=n_jq_j\). A planning approximation calculates information using \(n_jq_j\) observed outcomes rather than \(n_j\) enrolled participants. The information fraction is the information under incomplete ascertainment divided by the information under complete ascertainment, for the same enrollment plan and outcome model.

Assumptions and a Basic Derivation

Consider a two-arm comparison of means. Assume independent participants within and between arms, a common known outcome variance \(\sigma^2\), and an estimator equal to the difference between the observed-sample means. Assume further that, within each arm, whether the outcome is observed is independent of the outcome value, and that each participant has the same observation probability \(q_j\) within that arm. These assumptions allow the observed participants to represent their arm for estimating its mean. They do not cover outcome-dependent missingness, clustering, or variance changes caused by other features of the design.

Conditional on observing \(m_T\) treatment outcomes and \(m_C\) control outcomes, the variance of the estimated mean difference is \(\sigma^2/m_T+\sigma^2/m_C\). Information is the reciprocal of that variance. For planning, substitute the expected counts \(n_Tq_T\) and \(n_Cq_C\). This uses expected counts as a planning approximation; because information is a nonlinear function of counts, it is not generally identical to the exact average of the information over all possible realized counts.

I_{\text{obs}}\approx \frac{1}{\sigma^2/(n_Tq_T)+\sigma^2/(n_Cq_C)}, \qquad I_{\text{full}}= \frac{1}{\sigma^2/n_T+\sigma^2/n_C}.

Dividing the first expression by the second gives the information fraction for this comparison. The common variance cancels, but the completion rates and allocation still matter.

\frac{I_{\text{obs}}}{I_{\text{full}}} \approx \frac{1/n_T+1/n_C}{1/(n_Tq_T)+1/(n_Cq_C)}.

If both arms have the same completion probability \(q\), then \(q_T=q_C=q\), and the expression simplifies to \(I_{\text{obs}}/I_{\text{full}}\approx q\). Thus, under these assumptions, losing 15% of outcomes in both arms costs approximately 15% of the planned information. The corresponding standard error is multiplied by approximately \(1/\sqrt{q}\), not by \(1/q\), because standard error is the square root of variance.

Important distinction: The expected observed count is \(n_jq_j\), while the realized count is random. For an interim analysis, use the outcomes actually available under the analysis plan to assess current information; do not treat an expected count or an enrollment count as though it were the realized observed count.

Worked Calculations

Worked Example: Equal Completion Rates in a Mean Comparison

Suppose a hypothetical trial plans to enroll 200 participants per arm. The continuous outcome has a known standard deviation of 12 units, and 85% of outcomes are expected to be observed in each arm. Calculate the approximate information fraction and compare the standard errors with and without missing outcomes.

Check the assumptions. For this calculation, assume independent participants, the same outcome variance of \(12^2=144\) square units in both arms, and outcome observation independent of the outcome within each arm. These are planning assumptions, not facts established by the enrollment numbers.

With complete ascertainment, the variance of the estimated treatment-minus-control mean difference is \(144/200+144/200=1.44\) square units. Thus the information is \(1/1.44=0.6944\) per squared outcome unit, and the standard error is \(\sqrt{1.44}=1.200\) outcome units.

The expected observed count per arm is \(200(0.85)=170\). Using those expected counts, the approximate variance is \(144/170+144/170=288/170=1.6941\) square units. The corresponding information is \(1/1.6941=0.5903\) per squared outcome unit, and the standard error is \(\sqrt{1.6941}=1.302\) outcome units.

The information fraction is \(0.5903/0.6944\approx0.8500\), or about 85.0%. The standard error is about \(1.302/1.200=1.085\) times the complete-ascertainment value. The conclusion is that, under the stated assumptions, the expected information is reduced by about 15%, while the standard error rises by about 8.5%.

Worked Example: Different Completion Rates by Arm

A hypothetical mean-comparison trial enrolls 240 participants in the treatment arm and 200 in the control arm. Assume a common known standard deviation of 10 units. The expected completion rates are 80% and 90%, respectively. Compare the information based on expected observed outcomes with the information calculated from enrollment alone.

Check the assumptions. Assume independent participants, a common variance of \(100\) square units, and outcome-independent missingness within each arm. The calculation also assumes the stated completion rates are suitable planning values. Because the rates differ, do not use a single overall completion percentage without checking how the allocation and variances affect the contrast.

The expected observed counts are \(240(0.80)=192\) in treatment and \(200(0.90)=180\) in control. The approximate variance of the mean difference is \(100/192+100/180=0.5208333+0.5555556=1.0763889\) square units. The resulting information is \(1/1.0763889=0.9290\) per squared outcome unit.

If all enrolled participants had outcomes, the variance would be \(100/240+100/200=0.4166667+0.5000000=0.9166667\) square units, and information would be \(1/0.9166667=1.0909\) per squared outcome unit. Using unrounded expressions, the information fraction is \((1/1.0763889)/(1/0.9166667)\approx0.8516\), or about 85.2%. The standard errors are \(\sqrt{1.0763889}=1.037\) units under the missing-outcome approximation and \(\sqrt{0.9166667}=0.9574\) units with complete ascertainment.

The information fraction is not simply the arithmetic average of 80% and 90%. In a mean difference, each arm’s contribution to variance depends on its sample size and variance; here, the control arm’s smaller expected count contributes more to the variance than the larger treatment count.

Worked Example: Missing Binary Outcomes and a Risk Difference

In a hypothetical trial, 500 participants are enrolled per arm. For planning, the treatment and control event probabilities are 0.20 and 0.30, respectively, and 90% of outcomes are expected to be observed in each arm. Under independent Bernoulli outcomes and outcome-independent ascertainment, estimate the information fraction for the risk difference.

Check the assumptions. Use the stated event probabilities as planning values, assume independent participants and independent outcome observation with probability 0.90 in both arms. The calculation treats the observed participants in each arm as representative for estimating that arm’s event probability.

For a risk difference, the approximate variance is \(p_T(1-p_T)/n_T+p_C(1-p_C)/n_C\). With complete ascertainment, it is \(0.20(0.80)/500+0.30(0.70)/500=0.16/500+0.21/500=0.0007400\). The information is \(1/0.0007400=1351\) per squared proportion unit, and the standard error is \(\sqrt{0.0007400}=0.02720\), or about 2.72 percentage points.

The expected observed count is \(500(0.90)=450\) per arm. The approximate variance becomes \(0.16/450+0.21/450=0.37/450=0.0008222\). Information is \(1/0.0008222=1216\) per squared proportion unit, and the standard error is \(\sqrt{0.0008222}=0.02867\), or about 2.87 percentage points. The information fraction is \(1216/1351\approx0.9000\). Under these assumptions, 10% missing outcomes in both arms reduce information by about 10%; they do not make the estimated risk difference itself 10% smaller.

Using the Calculation in Sequential Monitoring

A group sequential design plans analyses using information available at each look. Enrollment can continue while outcomes are pending, so an enrollment fraction can move ahead of the information fraction. If outcomes are observed only after a fixed follow-up period, participants enrolled recently may contribute no outcome information at that look. The calculation should therefore be tied to outcomes available for the planned analysis, not merely to participants randomized by that date.

For planning, an expected completion rate can translate planned enrollment into an approximate information target. For monitoring, however, the realized number of observed outcomes and the analysis-specific variance determine the information estimate. If the realized counts differ from expectations, the information fraction can differ too. The group sequential design must use its planned information scale consistently; enrollment should not be substituted for information just because enrollment is readily counted.

The distinction also matters when sizing a trial. With a common completion probability \(q\) and otherwise identical participants, obtaining an expected \(n\) observed outcomes requires approximately \(n/q\) enrolled participants. This is a simple inflation calculation, not a universal cure for missing data. It is only defensible when the assumptions behind the information calculation are plausible and the target is stated in terms of observed outcomes.

Diagnostics, Alternatives, and Limitations

Compare actual outcome availability with the planning assumptions by arm and by analysis time. Useful checks include the observed completion proportion, the number of outcomes still pending, and whether missingness differs across treatment groups or participant subgroups. Investigate reasons for missing outcomes when possible. A completion rate that changes over time or differs substantially by arm makes a single common-\(q\) approximation questionable.

The simple calculation is appropriate for quantifying a precision loss when outcomes are missing independently of their values within each arm. If missingness depends on the unobserved outcome or on prognosis in a way that is not handled by the analysis, the observed-sample mean or proportion can be biased. In that situation, replacing \(n\) with \(nq\) estimates neither the full-data information nor the bias. More elaborate missing-data models may be needed, and their assumptions must be examined separately.

Enrollment-based calculations are the main tempting alternative: they use \(n_T\) and \(n_C\) as if all outcomes were observed. That is valid only when outcomes are in fact available for essentially everyone, or when the calculation is explicitly a complete-ascertainment benchmark. Under incomplete ascertainment, it generally overstates information and understates the standard error. A second shortcut is to apply one average completion rate to both arms. That can be misleading when completion differs by arm, because the variance of a contrast depends on each arm’s observed count separately.

Other limitations are equally important. The expected-count approximation may be rough when sample sizes or completion probabilities are small, since realized counts can vary appreciably and a zero count in an arm would prevent estimating its mean. The calculation assumes the variance model used in planning is appropriate. If the observed outcomes have different variances, or the design has additional dependence, the simple formulas do not capture those features. These concerns require modifying the information calculation rather than treating an enrollment-based count as a substitute.

Common mistake: “We randomized 200 participants per arm, so the information is the same as 200 observed outcomes per arm.” This confuses enrollment with outcome availability. If only 170 outcomes per arm are expected and the assumptions above hold, the mean-comparison information is approximately 85% of the complete-ascertainment information. Use expected observed counts for planning and actual observed outcomes for an interim information assessment.

Key takeaway: Under outcome-independent missingness and a correctly specified outcome model, incomplete ascertainment reduces precision through fewer observed outcomes. Quantify that loss on the information scale appropriate to the estimand, and do not interpret the calculation as evidence that the observed outcomes are unbiased when missingness may depend on outcome.

Check Your Understanding

Answer each question, then check the solutions below.

A trial enrolls 300 participants per arm, with an expected 80% outcome completion rate in each arm. Under the simple equal-variance mean-comparison assumptions, what is the approximate information fraction?
Why does a 20% reduction in information not imply a 20% increase in standard error?
For a mean comparison, treatment has 100 expected observed outcomes and control has 200. Both have the same outcome variance. Which arm contributes more to the variance of the estimated difference?
State one reason why replacing enrollment counts by expected observed counts may fail to address the main problem caused by missing outcomes.

Solutions. (1) With the same completion rate in both arms, the information fraction is approximately \(q=0.80\). (2) Standard error is the square root of variance, and information is the reciprocal of variance. An information fraction of 0.80 implies a standard-error multiplier of \(1/\sqrt{0.80}\approx1.118\), an increase of about 11.8%, not 20%. (3) The treatment arm contributes more: with common variance \(\sigma^2\), its contribution is \(\sigma^2/100\), compared with \(\sigma^2/200\) for control. (4) If missingness depends on the unobserved outcome or prognosis, observed outcomes may be biased; changing the count in a precision formula does not correct that bias.