Combining Futility Decisions With Sample Size Updates

Why Futility and Sample-Size Decisions Must Be Coordinated

At an interim analysis, a weak treatment-effect signal can motivate two different actions. The trial might stop because success is no longer sufficiently plausible, or it might continue with more participants because additional information could still make the result conclusive. A rule that does not distinguish these actions can spend resources on a trial with little prospect of success, or stop a trial that could benefit from a planned increase.

The decisions are related because the same interim statistic informs both. Their operating characteristics are therefore joint: the probability of stopping, increasing, or retaining the planned sample size depends on the interim result, while the probability of final rejection also depends on the data accumulated after the interim. A final test calibrated for a fixed sample size cannot simply be presumed to retain its nominal error rate after these data-dependent choices.

Model and Conditional Power

Consider a one-sided test of \(H_0:\delta\leq 0\) against \(H_a:\delta>0\), where \(\delta\) is a standardized effect per square root of the sample-size information unit. For illustration, assume a normally distributed score with known variance, equal allocation, fixed information per participant, and independent increments. Write \(Z_1\) for the standardized interim statistic after \(n_1\) information units. Under a specified effect \(\delta\), assume \(Z_1\sim N(\delta\sqrt{n_1},1)\). The final test rejects when \(Z_N\geq 1.96\), corresponding to a one-sided significance level of approximately 0.025 under a fixed sample size.

For a planned total \(N>n_1\), the independent-increments model gives \(Z_N=\sqrt{n_1/N}\,Z_1+\sqrt{(N-n_1)/N}\,Z_{\mathrm{inc}}\), where \(Z_{\mathrm{inc}}\) is the standardized statistic from the new information. Conditional on the interim data, \(Z_{\mathrm{inc}}\) is independent of \(Z_1\) and has distribution \(N(\delta\sqrt{N-n_1},1)\) under the effect assumption used for prediction.

The conditional power, denoted \(CP_N(z;\delta)\), is the probability of crossing the final boundary given the observed interim statistic \(z\), if the final total is \(N\) and the future increments follow the specified effect assumption \(\delta\). Thus, \(CP_N(z;\delta)=P(Z_N\geq1.96\mid Z_1=z)\). This is a conditional probability under a model and effect assumption, not the unconditional probability that the trial will succeed.

A decision rule needs to specify its boundaries before the interim data are examined. The following hypothetical rule illustrates one way to separate the actions. The trial has \(n_1=50\) interim information units, an original total of 100, and a maximum total of 140. It stops for futility if \(Z_1\leq0\), or if even the maximum total has conditional power below 0.40 under the chosen effect assumption. If it is not futile, it increases to 140 when \(CP_{100}<0.40\); otherwise it retains the total of 100. Consequently, an increase occurs only when \(CP_{100}<0.40\leq CP_{140}\). The value 0.40 and the effect assumption are design choices, not universal standards.

For the numerical illustration below, take \(\delta=0.20\). Conditional power increases with \(z\), so each conditional-power threshold corresponds to an interim-statistic cutoff. Solving \(CP_{140}(z;0.20)=0.40\) gives \(z\approx0.3942\); solving \(CP_{100}(z;0.20)=0.40\) gives \(z\approx1.104\). The combined rule is therefore futility for \(z<0.3942\), increase for \(0.3942\leq z<1.104\), and retain 100 for \(z\geq1.104\). The explicit \(Z_1\leq0\) futility condition is included in the rule, although the maximum-sample conditional-power condition makes the futility region wider in this numerical example.

Joint Probabilities of the Decisions

The decision regions can be translated into probabilities under any specified effect. In this example, let \(c_F=0.3942\) be the futility cutoff and \(c_I=1.104\) the increase-to-retain cutoff. Since \(Z_1\sim N(\delta\sqrt{50},1)\), the probabilities of futility, increase, and retain are respectively \(\Phi(c_F-\delta\sqrt{50})\), \(\Phi(c_I-\delta\sqrt{50})-\Phi(c_F-\delta\sqrt{50})\), and \(1-\Phi(c_I-\delta\sqrt{50})\). These are probabilities of the interim actions only; they do not by themselves describe final rejection.

Under the boundary null \(\delta=0\), the same interim cutoffs imply futility probability \(\Phi(0.3942)\approx0.6533\), increase probability \(\Phi(1.104)-\Phi(0.3942)\approx0.8652-0.6533=0.2119\), and retain probability \(1-\Phi(1.104)\approx0.1348\). These action probabilities are themselves useful operational characteristics, but neither the increase probability nor the futility probability is the type I error rate.

Overall Rejection Probability and Its Justification

To evaluate the final test, combine each continuation branch's conditional rejection probability with the probability of reaching that branch. Under the stated model, let \(f_\delta(z)\) be the normal density with mean \(\delta\sqrt{50}\) and variance 1. Futility contributes no rejection because the trial stops. The overall rejection probability is the sum of the increase-branch and retain-branch contributions:

This expression is a direct application of conditioning on the interim statistic. For example, on the increase branch, the joint event “increase and reject” has probability \(P(c_F\leq Z_1<c_I,\ Z_{140}\geq1.96)\). Integrating the conditional probability of rejection over the interim values in that branch gives \(\int_{c_F}^{c_I}P(Z_{140}\geq1.96\mid Z_1=z)f_\delta(z)\,dz\). The retain branch has the corresponding integral with \(CP_{100}\). Adding the disjoint branches gives the displayed result.

For the boundary null, evaluating \(R(0)\) by numerical integration gives the design's type I error at \(\delta=0\) under these model assumptions. To assess a composite null, examine the rejection probability over the null parameter values relevant to the model rather than treating a single calculation as a universal guarantee. The fixed-sample critical value of 1.96 alone does not establish that the adaptive rule has its intended error rate: the final sample size depends on the interim statistic. The same integral evaluated at a scientifically relevant positive \(\delta\) gives the power of the complete adaptive procedure, including the possibility of futility stopping.

Design Checks, Alternatives, and Limitations

Before implementation, verify that the action regions are mutually exclusive, cover all possible interim results, and match the protocol's intended policy. Check conditional power at the cutoffs and on either side of them; here, \(CP_{140}\) reaches 0.40 at the futility-to-increase boundary and \(CP_{100}\) reaches 0.40 at the increase-to-retain boundary. Also check that the chosen effect assumption, information scale, interim timing, and final boundary are the ones used to calculate the operating characteristics.

The independent-increments normal model is an approximation whose adequacy depends on the endpoint and analysis. The calculations above assume known variance and a fixed amount of information per participant. If these assumptions are poor, the conditional distribution and resulting probabilities need an appropriate model. The effect used to calculate conditional power is particularly consequential: an optimistic value can make an increase appear more promising than it is under less favorable effects. Report operating characteristics over a range of effects, including values near the null and the design alternative, rather than relying on one scenario.

A fixed-sample design is simpler to interpret and does not require interim action regions, but it cannot respond to an interim indication that the original sample size is inadequate. A futility-only design can limit exposure and cost but does not attempt to recover power by adding participants. An increase-only design may continue trials that have little prospect of success. Combining futility and increase decisions makes that trade-off explicit, but adds decision boundaries and requires joint operating-characteristic evaluation. Where a design uses a different final testing procedure to protect its error rate, that procedure must be included in the operating-characteristic calculation.