Simulating Blinded Sample-Size Re-estimation

Why simulate blinded sample-size re-estimation?

A trial planned using one estimate of outcome variability and dropout can be underpowered if either nuisance quantity is larger than expected. Blinded sample-size re-estimation (SSR) uses accumulating data, without revealing treatment assignments, to revise the planned sample size. Simulation evaluates whether the rule actually improves performance across plausible conditions—and whether its adaptations affect type I error.

Blinding does not make an adaptive design automatically valid. The re-estimation rule, final analysis, missing-data handling, and any maximum sample size together define the design. In particular, a rule that performs well when the treatment effect is present may still have unacceptable type I error under the null. The operating characteristics must therefore be evaluated separately under null and alternative scenarios.

State the design and the rule before simulating

Consider a two-arm randomized trial with a continuous endpoint, equal allocation, and a target difference \(\delta\) between treatment-group means. For initial planning, assume independent normally distributed outcomes with a common standard deviation \(\sigma\), a two-sided test at significance level \(\alpha\), and no covariate adjustment. A normal-approximation target for the number of evaluable participants per arm is

where \(1-\beta\) is the desired power and \(z_q\) is the \(q\)-quantile of the standard normal distribution. This calculation is a planning approximation, not a guarantee of the operating characteristics of the eventual test. If the expected dropout proportion is \(r\), a simple planning adjustment is \(n_{\mathrm{rand}}=\lceil n_{\mathrm{eval}}/(1-r)\rceil\) per arm. That adjustment assumes the anticipated dropout proportion is relevant to the endpoint and that the analysis remains valid with the resulting missing data.

A simulation specification should make the operational rule explicit. For example: at a prespecified interim time, estimate a blinded pooled outcome standard deviation and the fraction of randomized participants without an observed endpoint; recalculate the evaluable target using the original \(\delta\), \(\alpha\), and power target; inflate for estimated dropout; round up; and apply a maximum. State whether the result replaces the original total or is an additional recruitment target. In the examples below, it replaces the target total number randomized per arm, and participants already randomized count toward that total.

One complication is that a blinded estimate of variability may depend on the treatment effect. Under equal allocation, the variance of all observations pooled without treatment labels includes both within-group variation and variation between the two group means. In a simple balanced setting, its expectation is approximately \(\sigma^2+\delta^2/4\), rather than \(\sigma^2\), when the true mean difference is \(\delta\). Thus, pooled blinded variance can be conservative under the alternative. This is one reason to evaluate the rule under both null and alternative conditions rather than assume the blinded estimate is an unbiased estimate of the within-group variance in every scenario.

Simulate the complete design

For each simulated trial, generate participants under a specified scenario, apply randomization and dropout, perform the blinded re-estimation exactly as planned, and analyze the final data using the prespecified test. Repeat this process many times. Under \(H_0\), the rejection proportion estimates type I error. Under a specified alternative, it estimates achieved power. Simulations should reproduce the timing of the interim, the information available to the re-estimation procedure, and all sample-size limits; replacing these details with an idealized calculation can miss important design effects.

For example, a scenario grid can vary the true standard deviation and dropout proportion while holding the target difference and analysis fixed. Include the planning assumptions, plausible departures, and more adverse but credible values. Under the null, set the treatment difference to zero while retaining relevant nuisance conditions. Under the alternative, use the effect size or range of effects that the design is intended to detect. If the analysis is intended to address a particular missing-data mechanism, simulate that mechanism rather than assume dropout is harmless.

Justification, diagnostics, and failure modes

The sample-size formula follows directly from the approximate standard error of a difference between two independent group means. With \(n\) evaluable participants per arm, that standard error is \(\sigma\sqrt{2/n}\). Under the alternative, the standardized mean difference is approximately \(\delta/(\sigma\sqrt{2/n})\). To achieve power \(1-\beta\) for a two-sided test, this signal should be about \(z_{1-\alpha/2}+z_{1-\beta}\) or greater. Solving that inequality for \(n\) gives the planning formula:

This derivation explains why higher variance increases the target approximately in proportion to \(\sigma^2\), and why a fixed maximum can prevent the rule from reaching its intended power. It also relies on the stated normal approximation and design assumptions; simulation of the final test is the check on how well the approximation serves the specific design.

Blinded adaptation does not, by itself, prove type I error control. The re-estimation statistic is calculated from outcome data, and its relationship to the final test statistic can affect the null rejection probability. The pooled variance can also reflect a treatment effect under the alternative. Consequently, simulate the full rule under the null, including its estimator, timing, rounding, cap, and final analysis. Earlier material on calibrating type I error provides the general simulation principle; here the key point is to apply it to every adaptive branch of the SSR design.

Useful diagnostics include checking that the realized sample-size distribution agrees with the rule, that the interim estimate uses only blinded information, and that the simulated dropout and variance match each scenario. Record how often the maximum is reached, how often the target is unchanged, and whether the estimator is unstable when few endpoint observations are available. Check for implementation errors such as counting the interim participants twice, calculating dropout among the wrong population, or using outcomes that would not yet be observed at the interim.

The main alternative is a fixed sample size chosen conservatively to cover plausible nuisance values. It is operationally simpler and avoids data-dependent recruitment, but can be inefficient when the planning assumptions are optimistic. Blinded SSR can respond to observed nuisance information while preserving the blind, but it adds operational complexity and may still miss the target because of estimation noise or a cap. An unblinded re-estimation uses treatment-specific information and raises distinct inferential and operational concerns; it is not interchangeable with blinded SSR. The appropriate comparison is not “adaptive versus nonadaptive” in the abstract, but simulated operating characteristics and resource use under the same plausible scenarios.

Important limitations remain. A blinded pooled variance can be affected by treatment separation; a dropout proportion estimated from an immature interim cohort may not predict final endpoint availability; and simple inflation by \(1/(1-r)\) does not correct bias caused by informative missingness. A cap limits cost but can reduce power precisely in adverse conditions. If the endpoint distribution, variance structure, allocation, or analysis differs from the assumed model, the re-estimation rule and its evaluation must reflect those features. Simulation assesses the specified scenarios; it cannot guarantee performance under unconsidered conditions.

Common mistake

Wrong reasoning: “The sample-size update is blinded, so it cannot affect type I error; checking power under the alternative is enough.” Correction: Blinding is not a mathematical guarantee of type I error control. The re-estimation uses outcome data, and the final test is applied after a sample size has been selected using those data. Simulate the entire procedure under \(H_0\) as well as under alternatives, and report the null rejection rate with its Monte Carlo uncertainty. Also distinguish a failure to detect a difference from evidence that type I error is acceptably controlled: too few simulation replicates may simply provide an imprecise estimate.