Estimating Variance in Unblinded Re-Estimation

Why the Interim Variance Estimate Matters

For a continuous outcome, conditional power depends not only on the interim treatment-effect estimate and the assumed effect for the remaining participants, but also on the outcome variance. A larger variance means more uncertainty in the treatment comparison, generally lowering conditional power for a fixed sample size. If the variance is estimated from interim data, that estimate can therefore influence both a conditional-power calculation and the sample size proposed by a re-estimation rule.

This tutorial considers an unblinded interim analysis in a two-arm, equal-allocation trial. “Unblinded” means that the interim treatment comparison and the within-arm variability can be examined by the people or process making the re-estimation decision. The calculations below are planning approximations, not a guarantee that any particular adaptive design preserves its intended error rates.

Estimating the Common Variance

Let \(s_E^2\) and \(s_C^2\) be the sample variances within the experimental and control arms at the interim analysis. Under the common-variance model, the pooled within-arm variance is

Equivalently, the numerator is the sum of squared deviations from each arm’s own sample mean. It does not include the difference between the two arm means. This distinction matters: the variance estimate is intended to describe within-arm outcome spread, not to treat a treatment-effect difference as additional noise.

Under the normal, common-variance model, \(df\,s_1^2/\sigma^2\) has a chi-square distribution with \(df=2n_1-2\) degrees of freedom. Thus, even when the model is correct, the interim variance estimate varies from one trial to another. For example, the relative standard deviation of \(s_1^2\) is \(\sqrt{2/df}\). Increasing the interim sample size raises \(df\) and reduces this relative variability, but does not remove it.

How the Estimate Enters Conditional Power

Write \(\hat{\delta}_1\) for the interim difference in sample means and \(\delta^*\) for the effect assumed for the future participants. The latter is a planning assumption, not an estimate forced by the observed interim difference. Conditional power is the probability of meeting the final rejection criterion, conditional on the interim data, under the assumed future effect and variance.

For the normal approximation, suppose the final test rejects in the upper tail when its standardized statistic exceeds \(z_{1-\alpha}\), the standard-normal quantile at probability \(1-\alpha\). If \(\sigma\) were known, the final standardized difference in means would be

Here \(\hat{\delta}_2\) is the difference in means among future participants. Conditional on the interim observations, the future difference is independent of them and, under the assumed effect, has mean \(\delta^*\) and variance \(2\sigma^2/n_2\). Consequently, the conditional distribution of the final statistic has mean \(\mu_c\) and variance \(v_c\) as follows.

In an unblinded plug-in calculation, substitute \(s_1\) for \(\sigma\). The upper-tail conditional power is then

This expression makes the variance’s role explicit: for fixed interim results and sample sizes, increasing \(s_1\) reduces the standardized contributions of the observed and assumed effects. A smaller estimated standard deviation usually raises calculated conditional power; a larger one usually lowers it. For a two-sided final test, the probability is the sum of the lower- and upper-tail rejection probabilities, using boundaries \(-z_{1-\alpha/2}\) and \(z_{1-\alpha/2}\).

Using Plug-In Variance to Re-Estimate Sample Size

A deterministic re-estimation rule can evaluate candidate future sample sizes using the same conditional-power formula, then choose a candidate that reaches a prespecified target. The variance estimate affects that choice directly. If the interim variance is larger than planned, more participants may be required to achieve the same target conditional power, subject to any design constraints. If the estimate is smaller, the calculation may indicate fewer participants. The decision rule should state in advance how the variance estimate is formed, what effect assumption is used, and how the calculated sample size is constrained.

The arithmetic must respect the allocation. With \(n_2\) defined as future participants per arm, the future total is \(2n_2\), and the final total is \(2(n_1+n_2)\). In particular, an equal-allocation calculation cannot use an odd total sample size or split an odd number of future participants equally.

Why the Calculation Is an Approximation

The derivation treats \(s_1\) as fixed after observing the interim data. It thereby accounts for the observed interim difference when calculating conditional power, but not the sampling uncertainty in the variance estimate. The chi-square interval in the first example illustrates that this uncertainty can be substantial at modest interim sample sizes. As a result, a conditional-power value near a decision cutoff may be sensitive to the particular variance estimate observed.

There is a second approximation: the final statistic in a real analysis commonly uses an estimated standard deviation, for example a pooled-variance \(t\)-statistic, rather than the known-\(\sigma\) normal statistic used in the derivation. Treating the interim variance estimate as the standard deviation in the normal formula is a planning approximation. When the sample is small, when a decision is close to a cutoff, or when accurate operating characteristics matter, a more complete calculation can account for the distribution of the variance estimate and the final test statistic, often by numerical integration or simulation.

The common-variance assumption also needs scrutiny. If the treatment arms have materially different variances, pooling their within-arm sums of squares does not estimate a shared variance as assumed. A heteroscedastic analysis requires arm-specific variance estimates and a standard error appropriate to that model; the pooled formulas above should not be used unchanged. Strong skewness, outliers, dependence between observations, or interim data that are not representative of future outcomes can also undermine the normal-model calculation.

Diagnostics, Alternatives, and Common Pitfalls

Before using an interim estimate, check that it is based on within-arm residual variation, that both arms contribute the intended data, and that the units are consistent. Compare arm-specific spreads and inspect distributions for unusual outliers or marked departures from the working model. These checks do not prove the assumptions, but large discrepancies are a warning that a single pooled variance may be misleading. Also check the sensitivity of the decision to plausible values of \(\sigma\), especially if the proposed sample size changes sharply across those values.

The plug-in estimate is not the only possible approach. A design may use a prespecified planning variance, which avoids responding to a noisy interim estimate but can be inefficient when the planning value is inaccurate. It may instead use an uncertainty-aware calculation that averages conditional power over a distribution for the unknown variance, or a conservative variance value, at the cost of additional modeling assumptions or larger sample sizes. These approaches answer different planning questions; none makes the variance known.

• Mistake: treating an interim estimate as the population variance. The reasoning “\(s_1=10\), so \(\sigma=10\)” ignores sampling variation. Use \(s_1\) as a plug-in estimate, label the result accordingly, and assess sensitivity or variance uncertainty when consequential.
• Mistake: pooling the treatment-effect difference into the variance estimate. That mixes between-arm signal with within-arm outcome spread. Calculate each arm’s deviations around its own mean before pooling.
• Mistake: mixing total and per-arm sample sizes. If \(n_1\) and \(n_2\) denote participants per arm, keep the final total as \(2(n_1+n_2)\). An odd number of future participants cannot be divided equally between two arms.
• Mistake: interpreting calculated conditional power as guaranteed design power. The result is conditional on the interim data, the assumed future effect, the variance approximation, and the planned test. The adaptive rule’s overall performance must be evaluated for the complete design; recalculating conditional power alone does not establish error-rate validity.

The key practical distinction is between estimating variance and knowing it. Unblinded re-estimation can make the sample-size decision responsive to observed variability, but the resulting decision remains conditional on a model and a noisy estimate. A defensible design prespecifies the estimator and decision rule, respects allocation and sample-size limits, and checks how sensitive the result is to plausible variance values.