Effect-Driven Sample Size Changes With Error Control

Why the sample-size rule and the test must be considered together

An interim treatment-effect estimate can suggest that a trial needs more participants than originally planned: the observed effect may be smaller than anticipated, but still promising enough to justify additional information. The difficulty is not merely choosing the second-stage sample size. Because that choice uses interim data, the final test must still have its intended Type I error under the null hypothesis.

Two useful approaches are conditional-error recalibration and an inverse-normal combination test. They protect error in different ways. Conditional-error recalibration asks how much rejection probability the original design had remaining after the observed interim data. An inverse-normal test combines stagewise evidence using weights fixed independently of the interim results. In either approach, an effect-driven sample-size rule is compatible with error control only when the relevant null calibration conditions hold.

Model and assumptions

Consider a one-sided test of \(H_0\) against \(H_a\) at significance level \(\alpha\), with two stages using distinct participants. Let \(Z_1\) be the first-stage test statistic and \(Z_2\) the second-stage statistic. The interim data include \(Z_1\), or more generally the stage-one information used to estimate the treatment effect. A rule based only on those data chooses the second-stage sample size \(n_2\), before any stage-two outcomes are observed.

The conditional distribution requirement is important. It is often justified when stage two uses new participants and a statistic standardized using its correct stage-two standard error. Independence follows because the conditional distribution of \(Z_2\) is the same standard normal distribution for every possible stage-one result and sample-size choice. If the selection rule changes the stage-two test in a way that makes its null distribution depend on interim data, this justification no longer applies without further calibration.

Under an alternative with standardized effect \(\theta\), a common model for illustration is \(Z_2\mid\text{stage one},n_2\sim N(\theta\sqrt{n_2},1)\). Thus changing \(n_2\) can change power even when the null distribution remains standard normal. The Type I error arguments below concern the null calibration; they do not guarantee that an effect-based rule gives unbiased effect estimates or optimal power.

Conditional-error recalibration

Suppose an original design rejects when \(w_1Z_1+w_2Z_2\ge c\), where \(w_1,w_2\) are fixed, \(w_1^2+w_2^2=1\), \(w_2>0\), and \(c\) is the one-sided critical value. At an interim result \(Z_1=z_1\), the original design's conditional error is its probability of ultimately rejecting, conditional on that result and under \(H_0\). Since \(Z_2\) is conditionally standard normal, this probability is

Here \(\Phi\) is the standard normal cumulative distribution function. A revised stage-two test preserves the original conditional-error guarantee if, for every interim result, its conditional probability of rejection under \(H_0\) is no greater than \(A(z_1)\). Equality uses the full remaining budget. For a standard normal \(Z_2\), the exact recalibrated rule is \(Z_2\ge(c-w_1z_1)/w_2\). The sample size may be selected using the interim effect estimate, provided the stage-two null statistic retains the stated conditional distribution.

Inverse-normal combination with fixed weights

For a one-sided stagewise test, write each stage's \(p\)-value as \(p_i=1-\Phi(Z_i)\), and its normal-score evidence as \(Y_i=\Phi^{-1}(1-p_i)=Z_i\), for \(i=1,2\). Under the stated null assumptions, each stagewise \(p\)-value is uniform under \(H_0\), and the stage-two value is independent of the stage-one data. An inverse-normal combination test rejects when

where the weights and critical value are fixed independently of the interim data. Under \(H_0\), the two normal scores are independent standard normal variables, so the weighted sum is standard normal. Its rejection probability is therefore \(1-\Phi(c)\), even if \(n_2\) is chosen from the stage-one effect estimate, as long as the conditional null calibration of the stage-two \(p\)-value remains valid.

This is an unconditional calibration argument. At a particular interim result, the probability of rejection from the combination rule need not equal the original design's conditional error. That distinction matters: a fixed-weight inverse-normal test can control the overall Type I error while spending conditional error differently from the original design. If a design also requires preservation of the original conditional-error bound, that condition must be checked separately.

Why data-dependent combination weights need calibration

If the weights are chosen after observing \(Z_1\), the standard-normal argument for a fixed weighted sum does not automatically apply. Conditional on the interim data, the first-stage contribution and the weights are fixed, while only the stage-two statistic remains random. The conditional rejection probability is determined by those selected weights and the resulting boundary. It can differ from the original conditional-error budget, and averaging over interim results need not produce the nominal Type I error.

An effect-driven sample-size change does not by itself make inverse-normal testing invalid. The key distinction is whether the sample-size choice leaves the conditional null distribution of the stage-two \(p\)-value valid, and whether the test weights remain fixed as specified. If weights are altered based on the observed effect or selected size, one must establish null calibration for that adaptive rule—for example, by enforcing the conditional-error bound. Merely renormalizing the realized weights so their squares sum to one is not such a proof.

Practical checks and limits

Before applying either approach, specify the sample-size rule, stagewise tests, weights or conditional boundary, and null distribution after every possible choice. A useful diagnostic is to calculate the conditional rejection probability as a function of the interim statistic. Under conditional-error control, compare that probability with \(A(z_1)\) over the full range of interim results, not just at a typical value. For fixed-weight inverse-normal testing, verify that the stage-two \(p\)-value is conditionally uniform under the null for every permitted \(n_2\), and that the weights were not selected using the interim data.

Simulation under the null can reveal coding errors, incorrect standard errors, or an unintended dependence between stagewise data. It does not replace a calibration argument: a simulation over a limited set of interim values can miss a problematic region or conceal a small error-rate excess. If the assumed stage-two conditional distribution fails, or the data used to choose \(n_2\) include stage-two outcomes, the arguments given here do not establish Type I error control.

The approaches answer related but distinct questions. Conditional-error recalibration explicitly caps the revised test's conditional rejection probability by the original remaining budget and can accommodate changes to the stage-two rejection rule when its conditional null distribution is correctly specified. A fixed-weight inverse-normal test is a direct combination procedure with a simple unconditional null calibration, and it permits sample-size adaptation when the stagewise \(p\)-values retain their required null properties. Neither method licenses arbitrary data-dependent weights or an unverified change of stage-two test.