Simulating Sample Size Re-Estimation Rules

From an Update Rule to a Simulation

A sample-size re-estimation rule is a sequence of decisions: generate interim observations, calculate an interim quantity, apply a prespecified update, and continue the trial to its revised total sample size. A simulation makes that sequence executable across many hypothetical trials. It can reveal whether the implementation follows the intended rule and show what results it produces under specified data-generating scenarios.

The aim here is to simulate the whole trial, not just to plug a variance estimate into a formula. Each replicate must preserve the interim observations, generate any additional observations required by the update, and apply the planned final analysis. The same procedure is run under a null scenario and one or more alternative scenarios. The simulated rejection rate is an empirical summary of those runs; its precision depends on how many replicates are simulated.

Specify the Trial and the Update Rule

Consider a two-arm trial with a continuous outcome, equal allocation, and a common variance. For clarity, let \(n\) denote the total number of participants per arm. Assume independent outcomes across participants, normally distributed outcomes within each arm, and a common variance \(\sigma^2\). In the control arm the mean is \(\mu\); in the treatment arm it is \(\mu+\delta\). A null scenario sets \(\delta=0\), while an alternative scenario sets \(\delta\) to a prespecified nonzero value.

Suppose \(n_0\) participants per arm contribute interim outcomes. The interim variance estimate is the pooled within-arm sample variance. Let \(\bar{x}_{C,I}\) and \(\bar{x}_{T,I}\) be the interim sample means, and let \(SSE_{C,I}\) and \(SSE_{T,I}\) be the sums of squared deviations from those respective means. Then

Centering each arm on its own interim mean matters: the treatment difference in means is not counted as within-arm variation. Under the common-variance model, \(S_I^2\) estimates \(\sigma^2\). The update rule below uses it as the variance input to an approximate planning formula. It is a particular rule for illustration, not a universal recommendation.

For a two-sided test with significance level \(\alpha\), target power \(1-\beta\) at a target difference \(\Delta\), and equal allocation, a normal-approximation planning formula gives the required total sample size per arm as

Here \(z_q\) is the \(q\)-quantile of the standard normal distribution, and \(\lceil u\rceil\) is the smallest integer greater than or equal to \(u\). The bounds \(n_{\min}\) and \(n_{\max}\) are prespecified; in particular, \(n_{\min}\) must be at least \(n_0\), so the final total cannot be smaller than the interim sample already observed. The formula targets \(\Delta\), which must be chosen before examining interim results.

Why the Planning Formula Has This Form

Under the model, the difference in sample means at a fixed sample size \(n\) has variance \(2\sigma^2/n\). If the true difference is \(\Delta>0\), its standardized mean is \(\Delta/\sqrt{2\sigma^2/n}\). A two-sided normal test rejects when the standardized statistic exceeds \(z_{1-\alpha/2}\) in the positive direction. To obtain approximately \(1-\beta\) power from that direction, require the mean to exceed the critical value by about \(z_{1-\beta}\) standard errors:

This is a normal-approximation justification, not an exact identity for the power of a finite-sample \(t\)-test. It approximates power using the positive rejection boundary and does not explicitly account for the small probability of rejection in the opposite tail. Replacing \(\sigma^2\) by \(S_I^2\) creates the stated re-estimation rule. The simulation must still apply the actual prespecified final test, rather than treating this approximate planning calculation as the final analysis.

Run Complete Replicates

For each replicate, generate interim outcomes from the scenario’s model, compute \(S_I^2\), and apply the update rule. Then generate \(n_{\mathrm{new}}-n_0\) additional outcomes per arm and combine them with the interim outcomes. Analyze all \(n_{\mathrm{new}}\) outcomes in each arm using the prespecified final procedure. For a pooled two-sample \(t\)-test, the final statistic and degrees of freedom are

The pooled test assumes independent normally distributed outcomes with a common variance. If those assumptions do not match the planned final analysis, the simulation should implement the planned analysis instead. Under the stated model, the simulated observations should be independent draws from the specified arm distributions; the interim observations must not be discarded or regenerated after the sample-size update.

Use the same design and update rule when switching from a null to an alternative scenario; change the data-generating treatment effect to the scenario value. A simulation can record more than rejection: for example, it can store the interim variance estimate and final sample size for each replicate. Those records are useful for checking whether the program applied the rule correctly. The simulation is only as informative as its implementation: a mistake repeated thousands of times remains a mistake.

Summarize Simulation Results Carefully

If \(R\) independent replicates are run in one scenario and \(K\) of them reject \(H_0\), the simulated rejection proportion is \(\hat{p}_{\mathrm{sim}}=K/R\). Its Monte Carlo standard error describes the random variation from using a finite number of simulation replicates, conditional on the simulated model and procedure:

Under a null scenario, this proportion estimates the rejection frequency of the implemented procedure under that particular null model. Under an alternative scenario, it estimates the rejection frequency under that alternative model. These are simulation estimates, not exact probabilities, and they do not establish that the model describes a future trial.

Diagnostics, Alternatives, and Limitations

Several checks help distinguish a sound simulation from a program that merely produces plausible-looking numbers. First, verify that the interim variance formula uses \(2n_0-2\) degrees of freedom and centers outcomes within each arm. Second, check that the updated \(n_{\mathrm{new}}\) is an integer within its bounds and is at least \(n_0\). Third, confirm that the final analysis includes all interim observations and only the required additional observations. Finally, inspect a few replicates manually, including cases where the update hits each bound.

For an implementation check, set the treatment effect and variance to known values and examine generated interim sample means and variances across many replicates. Their averages should be broadly consistent with the scenario parameters, allowing for simulation variation. A rejection proportion that changes substantially across independent runs may simply have large Monte Carlo error; report \(R\) and \(SE_{\mathrm{MC}}\) rather than treating small differences as meaningful. Also verify that the random-number stream and scenario settings are recorded so a run can be reproduced.

Simulation complements analytic planning. A formula is faster and easier to inspect when its assumptions and approximations are suitable; simulation can follow an explicitly coded rule when the full sequence is too complicated for a simple calculation. Neither approach repairs a misspecified model. In particular, this example assumes normal outcomes, a common variance, independent participants, complete outcomes, and the stated final test. Unequal variances, missing outcomes, or a different final analysis require a data-generating model and analysis that reflect those features. Simulation results are conditional on every such choice.

A second easy error is to report a rejection proportion without stating the scenario or number of replicates. A number such as 0.049 is not self-interpreting: the reader needs to know the data-generating treatment effect, variance, rule, final test, and \(R\). A further limitation is that one null and one alternative scenario cover only those parameter settings. A result under a single null model does not establish behavior across every plausible null setting.