Why Extreme Outcomes Matter in a Blinded Update
A blinded sample-size update uses interim data without revealing which treatment group each participant belongs to. For a continuous outcome, the usual fixed-effect sample-size calculation depends on the outcome variance. An extreme observation can therefore have a large effect: sample variance gives large weight to squared distances from the mean, and the recalculated enrollment can change substantially even when the rest of the interim outcomes look similar.
This tutorial focuses on that sensitivity and on how robust scale summaries can help diagnose it. A robust summary is not automatically a replacement for variance in a sample-size formula. The distinction matters particularly when the target is a difference in means: the variance of individual outcomes, not just their central spread, determines the standard error of the mean difference.
The ordinary variance-based calculation presumes that the outcome is measured consistently, observations are independent, and the mean difference is the target effect. The common-variance version also presumes comparable within-arm variances. Under approximate normality, the sample variance has a direct role in the standard error and in normal-theory planning. Under heavy tails, the variance may still exist, but extreme values can make its interim estimate noisy. If the population variance does not exist, the ordinary variance-based mean calculation is not justified as stated.
Masking does not remove all complications. In a balanced trial, the pooled variance can include both within-arm variability and variation between the two treatment means. In the simple equal-variance setting, its expectation is approximately \(\sigma^2+\Delta^2/4\), where \(\sigma^2\) is the common within-arm variance and \(\Delta\) is the treatment difference. Thus an extreme value can affect the estimate, and treatment-effect contamination can also affect it. The examples below isolate outlier sensitivity; they do not claim that a blinded pooled estimate is a pure estimate of within-arm variance.
How the Sample Variance Responds to an Extreme Value
The squared-deviation definition explains the sensitivity. An observation far from \(\bar{x}\) contributes a term \((x_i-\bar{x})^2\), so increasing its distance by a factor of ten can increase its own squared deviation by roughly a factor of one hundred. The mean also moves when an extreme value is added, so the exact change is not just the square of that value. The calculation must be redone using the full sample.
Worked Example: One Extreme Value Changes the Interim Variance
Suppose a hypothetical masked interim sample contains the ten continuous outcomes \(15,9,12,11,11,11,11,11,11,40\), measured in outcome units. The value 40 is unusual relative to the other nine, but assume for this calculation that it is a valid observation.
The sum is 142 and the sum of squares is 2776. Therefore \(\bar{x}=142/10=14.2\), and the corrected sum of squared deviations is
The sample variance and standard deviation are
outcome units squared and outcome units, respectively. For comparison, the first nine observations have sum 102 and sum of squares 1176. Their mean is \(102/9\), and their sum of squared deviations is \(1176-102^2/9=20\), giving sample variance \(20/8=2.5\) and standard deviation \(\sqrt{2.5}\approx1.581\) outcome units. Adding the single value 40 changes the sample variance from 2.5 to 84.4, a factor of \(84.4/2.5=33.76\).
This comparison diagnoses sensitivity; it does not justify deleting 40. In an actual analysis, the value should first be checked against source records and measurement procedures. If it is valid, it may reflect genuine population variation relevant to planning.
For robust summaries, define the MAD as the median of the absolute deviations from the sample median. A common normal-calibrated scale estimate is \(1.4826\,\mathrm{MAD}\): under a normal model this factor makes the population MAD correspond to the standard deviation. The IQR is \(Q_3-Q_1\); under a normal model, \(IQR/1.349\) is a corresponding scale estimate. These calibrations rely on the normal distribution for interpreting the result as an estimate of \(\sigma\). Without that model, the MAD and IQR remain useful descriptions of central spread, but they do not generally recover the standard deviation.
Quartile calculations can differ slightly among software packages, especially for small samples. When reporting an IQR, state the convention used. The example below uses Tukey hinges: \(Q_1\) and \(Q_3\) are the medians of the lower and upper halves of an even-sized ordered sample.
Worked Example: Comparing Robust and Nonrobust Scales
Consider the hypothetical ordered outcomes \(8,9,10,10,11,11,12,12,13,13,14,30\), in outcome units. The median is \((11+12)/2=11.5\). The absolute deviations from the median, in order, are \(0.5,0.5,0.5,0.5,1.5,1.5,1.5,1.5,2.5,2.5,3.5,18.5\), so the MAD is \((1.5+1.5)/2=1.5\). The normal-calibrated MAD scale is \(1.4826(1.5)=2.224\) outcome units.
Using Tukey hinges, the lower half has median \(Q_1=10\), and the upper half has median \(Q_3=13\). Thus \(IQR=13-10=3\), and its normal-calibrated scale is \(3/1.349\approx2.224\) outcome units.
The sample sum is 153 and the sum of squares is 2309. Hence
outcome units squared and outcome units. The central robust scales are much smaller than the sample standard deviation because the value 30 is far from the rest. This contrast is a warning to investigate the outcome distribution, not proof that the robust value is the correct variance input. If 30 is a valid outcome from a heavy-tailed population, its contribution may be relevant to the variance of future observations and to the precision of a mean difference.
Using Scale Diagnostics in a Sample-Size Update
For a balanced two-arm comparison of means, a commonly used normal-approximation formula for the required number per arm is
where \(\alpha\) is the two-sided significance level, \(1-\beta\) is power, \(\Delta\) is the planned difference in means, and \(\sigma^2\) is the assumed common within-arm variance. This approximation assumes independent observations, equal allocation, a valid common-variance model, and a normal or sufficiently accurate large-sample approximation. An interim blinded estimate is not automatically equal to \(\sigma^2\), because it may be affected by treatment-effect contamination, outliers, and sampling variability. Any update also needs to follow the design’s pre-specified rules for minimum and maximum enrollment and for maintaining the intended error control.
Worked Example: A Variance Update Alters the Calculated Enrollment
Suppose a hypothetical balanced trial plans to detect a mean difference of \(\Delta=5\) outcome units, with two-sided \(\alpha=0.05\) and power \(0.80\). Use \(z_{0.975}=1.960\) and \(z_{0.80}=0.8416\), so their sum is 2.8016. If the interim variance input is \(s^2=84.4\) outcome units squared, the normal-approximation calculation is
The required number rounds up to 53 participants per arm, or 106 total, before applying any design-specific constraints. If instead the input were 2.5 outcome units squared, the same calculation would give
which rounds up to 2 per arm under the formula alone. That tiny result is not a plausible trial recommendation: it shows why this arithmetic must be interpreted alongside the design’s enrollment floor, the reliability of the interim estimate, and the assumptions behind the normal approximation. In particular, \(s^2=84.4\) came from a small sample containing an extreme observation, while 2.5 came from the first nine observations only. Neither number should be chosen merely because it produces a preferred enrollment.
A robust summary can be used as a diagnostic alongside the sample variance. If a pre-specified procedure uses a robust scale to estimate \(\sigma\), it must specify the calibration model and account for the fact that the target is variance relevant to the planned estimator. Replacing \(s^2\) in the formula with MAD squared, or with an uncalibrated IQR, changes the estimand and generally does not preserve the formula’s interpretation. A central-spread estimate can be especially misleading when the tails, rather than the center, determine uncertainty in the mean.
Derivation, Diagnostics, and Failure Modes
The algebra for updating the sample variance also makes its vulnerability visible. If the current \(n\) observations have mean \(\bar{x}\) and sum of squared deviations \(SS=\sum_{i=1}^n(x_i-\bar{x})^2\), and a new value \(y\) is added, the new mean is \(\bar{x}_{new}=(n\bar{x}+y)/(n+1)\). The updated sum of squared deviations is
To see this, write the old deviations around the new mean and add the new observation’s deviation. Since \(\sum_{i=1}^n(x_i-\bar{x})=0\), the cross-product terms cancel, leaving \(SS+n(\bar{x}-\bar{x}_{new})^2+(y-\bar{x}_{new})^2\). Substituting \(\bar{x}-\bar{x}_{new}=(\bar{x}-y)/(n+1)\) and \(y-\bar{x}_{new}=n(y-\bar{x})/(n+1)\) gives \(SS+n(y-\bar{x})^2/(n+1)\), as claimed. The added term grows quadratically with the distance between the new value and the previous mean.
In practice, the blinded analyst cannot use treatment-specific diagnostics without breaking the mask. Useful checks on the pooled outcomes include:
- Verify extreme values against source records, units, data-entry rules, and measurement limits. A verified value is not automatically an error.
- Inspect a histogram, dot plot, or other suitable distribution display for skewness, long tails, multiple clusters, or data-entry patterns.
- Compare \(s\) with the median, MAD, and IQR, and examine whether the difference is driven by one or a few observations.
- Recalculate a leave-one-out variance as a sensitivity diagnostic. A large change signals instability, but does not establish that the omitted value should be excluded.
- Check whether interim sample size is so small that both ordinary and robust scale estimates are imprecise. Robustness to a single extreme value does not make an estimate precise.
Heavy tails require particular care. If the population variance is finite but the fourth moment is very large, sample variance can vary greatly between interim samples. If the variance is infinite, a finite sample still produces a numerical \(s^2\), but that number does not estimate a finite population variance, so the stated variance-based mean-planning formula lacks its usual interpretation. A robust central scale may remain finite in such a setting; that does not make it the variance needed to quantify the sampling variability of the sample mean.
The main alternatives answer different questions. Sample variance directly estimates variance under its assumptions, but is sensitive to extremes. MAD and IQR are more resistant to a small number of outliers and help characterize central spread, but require distributional calibration to serve as estimates of standard deviation. Data verification can correct measurement or recording errors, but should not be used to discard inconvenient valid outcomes. A pre-specified robust or transformed analysis may be appropriate for some scientific targets, but it changes the estimand or analysis and should not be improvised during a blinded update.
The limitation is fundamental: without treatment labels, a pooled distribution cannot reveal whether an extreme value arose in one arm, whether within-arm spreads differ, or how much pooled spread reflects a treatment difference. Robust summaries can flag sensitivity but cannot resolve those masked sources of variation. The subsequent interpretation of pooled variance under unequal variances requires separate care; it should not be inferred from the outlier diagnostics alone.
Check Your Understanding
Answer each question, then compare with the solutions.
- Why can one extreme value have more influence on sample variance than on the median?
- A masked sample has \(n=20\) and sum of squared deviations \(SS=190\). What is its sample variance?
- In a normal model, how is the IQR commonly converted to a scale estimate, and why is that conversion not distribution-free?
- Does a smaller MAD than sample standard deviation prove that the MAD should replace variance in a mean-based sample-size formula?
Solutions. (1) Variance uses squared deviations from the mean, so a distant value can contribute a very large term; the median depends on the middle order of the observations and is less moved by one extreme value. (2) \(s^2=SS/(n-1)=190/19=10.0\) outcome units squared. (3) Use \(IQR/1.349\) under normality; the constant comes from the normal distribution’s quartiles, so another distribution need not have the same IQR-to-standard-deviation ratio. (4) No. MAD is a robust central-spread summary, not automatically the variance governing the sampling variability of a mean. A calibrated, pre-specified method and appropriate assumptions are needed.