Diagnostic: Variance Estimation and Re-Estimation

Why Variance-Estimation Noise Matters

For a continuous outcome, a sample size calculation depends on the variance as well as the effect size, significance level, and target power. In unblinded re-estimation, an interim variance estimate can replace the planning variance in a sample size rule. That substitution is useful when the planning variance is uncertain, but it also makes the revised sample size random: an unusually small estimate can reduce the sample size, while an unusually large estimate can increase it. The diagnostic question is not merely whether the estimate is unbiased. It is how much noise it carries and how that noise propagates into the design.

The calculations below use a normal approximation to a final two-sided test. For a balanced two-arm design, \(n\) denotes the number of participants per arm, \(\delta\) is the treatment-effect difference in outcome units, and \(\sigma\) is the common within-arm standard deviation. These calculations diagnose variance-driven changes; they are not a substitute for evaluating the complete group sequential and re-estimation procedure.

A convenient normal-approximation planning formula for a fixed final test is

Here \(n_0\) is the planned number per arm using planning standard deviation \(\sigma_{\mathrm{plan}}\); \(n^*\) is the unrounded variance-adjusted target, before any minimum, maximum, or rounding rule is applied. This proportional rule follows because the required sample size is proportional to the assumed variance when the other planning inputs are held fixed. Its diagnostic value is that relative variance-estimation error transfers directly to relative sample-size error.

How Interim Degrees of Freedom Control Instability

If the true variance equals the planning variance, define the variance ratio \(R=S_p^2/\sigma_{\mathrm{plan}}^2\). Then \(n^*=n_0R\) and \(R\sim\chi^2_{df}/df\). Since the mean and variance of a chi-square variable with \(df\) degrees of freedom are \(df\) and \(2df\), respectively,

Thus the plug-in target is unbiased before caps and rounding, but it can still be highly variable. The coefficient of variation of \(n^*\) is \(\sqrt{2/df}\). This is a measure of relative spread, not a guarantee that any single estimate will lie within that distance of \(n_0\). With small \(df\), the ratio distribution is also right-skewed: occasional large variance estimates can produce especially large targets.

From a Variance Estimate to Approximate Power

A revised target is not itself a power calculation. To see the consequence of a particular variance estimate, evaluate power at the selected sample size using a stated true variance and effect. Under the balanced two-arm normal approximation, the standardized treatment-effect estimator has noncentrality \(\lambda=\delta\sqrt{n/(2\sigma^2)}\). If the final test uses the fixed two-sided normal critical value \(z_{1-\alpha/2}\), its approximate power is

This expression is a fixed-sample approximation evaluated at a selected \(n\). In a group sequential design, the final boundary need not equal the fixed-test critical value. Moreover, a sample size chosen from interim data makes the full procedure adaptive. Therefore plug-in power at \(n^*\) is a diagnostic, not a proof of the operating power or type I error of the complete design.

A related one-sample or paired-difference calculation helps illustrate a common arithmetic check. If a hypothetical final sample has \(n=118\), \(\delta=5\) outcome units, and true standard deviation \(\sigma=20\) outcome units, then \(\lambda=\delta\sqrt{n}/\sigma=5\sqrt{118}/20=2.715695\). The fixed two-sided normal approximation at \(\alpha=0.05\) is

The one-sample formula is not interchangeable with the balanced two-arm formula: their standard errors differ. In either setting, retain unrounded quantities in intermediate calculations. For example, multiplying \(168.169\) by the rounded coefficient \(0.333\) gives a slightly different result than using \(\sqrt{2/18}\); the latter gives 56.1 participants, not 56.0 when rounded to one decimal place.

Quantiles, Caps, and Practical Diagnostics

A standard deviation summarizes spread but does not show the asymmetry of the variance ratio or the fraction of targets that may be extreme. Quantiles of \(\chi^2_{df}/df\) provide a direct diagnostic range. If a target is capped, the probability of reaching the cap can also be assessed under the variance model. A cap limits the implemented sample size; it does not make the underlying estimate less noisy.

For a real design, check the variance re-estimation rule against the assumptions and implementation, not just its formula. Useful diagnostics include:

• Compute the interim degrees of freedom and the implied coefficient of variation. A large value signals that target variability should be expected.
• Examine quantiles of the variance ratio and the frequency with which the minimum or maximum sample size would bind.
• Repeat the calculation over plausible true variances and interim sample sizes. Report the range of selected targets and their approximate power, rather than only the target at the planning variance.
• Inspect whether outliers, skewness, unequal variances, clustering, or missing outcomes make the pooled interim estimate unrepresentative of the final analysis variance.
• Check whether the estimate is computed from independent within-arm residual information as intended. Under the stated normal model, pooled variance and treatment-effect estimates are independent; that fact does not automatically hold under model departures or every analysis procedure.

Alternatives and Limits of the Diagnostic

The main alternative is to keep the sample size fixed at its planning value. That avoids variance-estimate-driven changes but leaves the design exposed to a planning variance that may be wrong. A second approach is to use a conservative planning variance or a prespecified upper bound, trading expected enrollment for protection against underestimation. A third is to use a stabilized estimate, such as a prespecified combination of interim information and planning information. Stabilization can reduce extreme revisions, but changes the sampling distribution and must be specified and evaluated rather than treated as an automatic correction. Blinded variance re-estimation may reduce access to treatment-effect information, but it does not eliminate variance-estimation noise or guarantee unchanged operating characteristics.

The chi-square diagnostic is exact only for the stated normal, independent, common-variance model. Small interim samples, heavy tails, outliers, variance differences between arms, and changes in outcome measurement can all undermine its usefulness. Rounding, a minimum sample size, and a maximum sample size can also make the implemented target biased relative to the unbounded plug-in target. Finally, fixed-sample power evaluated at each selected \(n\) is not generally the unconditional power of a sequential adaptive design. The latter depends on the complete decision and testing rule and should be assessed with a full design evaluation.

A Short Derivation and a Common Mistake

The sample-size variability result follows directly from the chi-square model. Let \(U=df\,S_p^2/\sigma^2\), so that \(U\sim\chi^2_{df}\). When \(\sigma^2=\sigma_{\mathrm{plan}}^2\), the rule gives \(n^*=n_0U/df\). Because \(E(U)=df\) and \(\operatorname{Var}(U)=2df\),

Taking the square root yields \(\operatorname{SD}(n^*)=n_0\sqrt{2/df}\). This derivation concerns the unbounded, unrounded target. If the true variance differs from the planning variance, then \(E(S_p^2)=\sigma^2\), so \(E(n^*)=n_0\sigma^2/\sigma_{\mathrm{plan}}^2\): the target shifts as well as remaining noisy.