Nonbinding Futility Rules

Why Futility Can Be Nonbinding

At an interim analysis, the data may suggest that the treatment is unlikely to succeed. A futility rule can identify such results and recommend stopping to avoid further cost or exposure. If that rule is nonbinding, however, the trial may continue after the trigger. The key question is whether this flexibility compromises type I error control.

Conditional error answers that question by measuring how much rejection probability remains, under the null hypothesis, after the interim data are observed. In the setup below, continuing with the planned final test preserves its original error rate. Stopping and declaring no rejection removes possible rejections, so it cannot increase that rate. If the continuation test itself is changed, conditional error provides a pointwise criterion for preserving the planned bound.

Model and Conditional Error

Consider a one-sided two-stage test at a fixed information fraction $t$, where $0<t<1$. Under the null hypothesis, let $Z_1$ and $Z_2$ be independent standard normal statistics from the first and second stages. The planned final statistic and rejection rule are

Z=\sqrt{t}\,Z_1+\sqrt{1-t}\,Z_2,\qquad \text{reject }H_0\text{ if }Z>c.

Here $c$ is chosen so that the planned test has type I error $\alpha$; for example, $c=\Phi^{-1}(1-\alpha)$ for a one-sided test with a standard normal null distribution. The assumptions are that the information fraction is fixed, the stage statistics have the stated null distributions and independence, and the final statistic and boundary are prespecified. Then $Z\sim N(0,1)$ under $H_0$, so the planned rejection probability is $\alpha$.

Definition: The conditional error at interim value $Z_1=z$ is the planned test’s probability of rejecting $H_0$, conditional on that interim value, under $H_0$. For this two-stage normal test, $$ A_0(z)=P_0(Z>c\mid Z_1=z) =1-\Phi\!\left(\frac{c-\sqrt{t}\,z}{\sqrt{1-t}}\right). $$

The formula follows because, given $Z_1=z$, rejection requires $Z_2>(c-\sqrt{t}\,z)/\sqrt{1-t}$, and $Z_2$ remains standard normal. A larger interim statistic leaves a smaller gap to the final boundary: $A_0(z)$ is increasing in $z$. Specifically, differentiating gives

A_0'(z)= \phi\!\left(\frac{c-\sqrt{t}\,z}{\sqrt{1-t}}\right) \sqrt{\frac{t}{1-t}}>0.

The unconditional planned error is the average of these conditional errors across interim results:

E_0[A_0(Z_1)] =\int_{-\infty}^{\infty}A_0(z)\phi(z)\,dz =P_0(Z>c)=\alpha.

Worked Example: Conditional Error at a Futility Trigger

Suppose the planned test is one-sided with $\alpha=0.025$, so $c=1.96$, and the interim occurs at $t=0.5$. A futility rule triggers when $Z_1\leq -0.5$. Calculate the conditional error at the boundary $Z_1=-0.5$.

Substitute $z=-0.5$, $t=0.5$, and $c=1.96$:

A_0(-0.5) =1-\Phi\!\left( \frac{1.96-\sqrt{0.5}(-0.5)}{\sqrt{0.5}} \right) =1-\Phi(3.2718) \approx 0.0005342.

Thus, if the observed interim statistic is exactly $-0.5$, the planned test has about $0.0005342$ probability of eventual rejection under $H_0$, conditional on that result. This is not the probability that the null hypothesis is true; it is a conditional rejection probability under the null model. The value also illustrates that a futility-triggering result can leave some, but very little, planned type I error.

Why Stopping Cannot Increase Rejection Probability

Let a futility trigger occur when $Z_1\leq f$. If the trial stops there and no rejection is possible after stopping, its rejection event is a subset of the rejection event of the original test. In the original test, the trial could still reject after an interim result in the trigger region; the stopping rule removes those possible rejections. Therefore the rejection probability cannot increase.

This can also be expressed directly using conditional error. The planned test’s probability of rejection is $\alpha$. Stopping at or below $f$ removes the conditional rejection probability in that region, giving

P_0(\text{reject after stopping at }Z_1\leq f) =\int_f^\infty A_0(z)\phi(z)\,dz =\alpha-\int_{-\infty}^{f}A_0(z)\phi(z)\,dz \leq\alpha.

The inequality holds because both $A_0(z)$ and $\phi(z)$ are nonnegative. For any finite $f$ in this normal model, $A_0(z)>0$ for finite $z$, so stopping strictly reduces the rejection probability. The resulting test is conservative relative to the original $\alpha$, unless the trigger region has zero probability or zero conditional error.

Worked Example: A Nonbinding Rule That Is Ignored

Use the preceding design: $t=0.5$, $c=1.96$, and the futility trigger $Z_1\leq -0.5$. Suppose the monitoring committee observes $Z_1=-0.8$, which triggers the recommendation, but the trial continues and uses the planned final test.

At this observed value, the continuation probability of rejection under the null is

A_0(-0.8) =1-\Phi\!\left( \frac{1.96-\sqrt{0.5}(-0.8)}{\sqrt{0.5}} \right) =1-\Phi(3.5718) \approx 0.000177.

Continuing with the unchanged final test uses exactly this planned conditional error at $Z_1=-0.8$. Repeating that policy across possible interim outcomes—continue with the original test whether or not futility is triggered—gives the original overall rejection probability $\alpha=0.025$. The rule is nonbinding because the trigger does not force stopping; ignoring it does not alter the test.

If the committee instead stops whenever $Z_1\leq-0.5$, it sets the rejection probability to zero for those interim values. Its overall rejection probability is then $\alpha-\int_{-\infty}^{-0.5}A_0(z)\phi(z)\,dz$, which is no greater than $0.025$. The integral is positive, so this stopping policy is more conservative than continuing with the planned test.

Changing the Continuation Test

Continuing does not automatically preserve error control if the final test changes. For example, a more permissive final boundary can increase the conditional rejection probability at an interim result. Conditional error gives a sufficient preservation condition: for every interim value $z$ under consideration, the conditional rejection probability $q(z)$ of the continuation procedure must be no greater than the planned $A_0(z)$. If the trial stops, take $q(z)=0$.

The justification is a direct conditional-expectation argument. If $q(z)\leq A_0(z)$ at every $z$, then

P_0(\text{reject}) =E_0[q(Z_1)] \leq E_0[A_0(Z_1)] =\alpha.

This condition is conservative as a general requirement: a pointwise increase at some interim values does not by itself prove that the overall error exceeds $\alpha$, because other values may have lower conditional rejection probabilities. But without a separate valid calibration, a pointwise increase means the conditional-error guarantee is not established. The earlier material on conditional error preservation develops the general framework for comparing continuation tests.

Worked Example: Checking a Modified Continuation Rule

At $t=0.5$, $c=1.96$, and $Z_1=-0.5$, the planned conditional error is about $0.0005342$. Consider a modified continuation rule that rejects only if $Z_2>\Phi^{-1}(0.9996)$. Its conditional rejection probability under the null is $q(-0.5)=0.0004$.

The numerical comparison is

q(-0.5)=0.0004\leq A_0(-0.5)\approx0.0005342.

At this interim value, the modified rule satisfies the conditional-error bound. Equivalently, its second-stage boundary is approximately $\Phi^{-1}(0.9996)=3.3528$, which is more stringent than the planned conditional boundary $3.2718$. This one-value calculation does not establish validity for every possible interim result; the inequality must hold throughout the relevant range, or the overall error must be justified by another valid calibration.

Assumptions, Diagnostics, and Alternatives

The calculations above rely on the specified null distribution, independence of stage statistics, and fixed information fraction. In an actual design, verify that the statistic and information used in the conditional-error calculation match the planned design. If the stage statistics are not independent, have a different null distribution, or the information fraction changes, the displayed normal formula may not apply; use the appropriate conditional distribution rather than inserting values into an unsuitable formula.

Useful checks include confirming that $A_0(z)$ lies between zero and one, increases with $z$, and averages to $\alpha$ under the null distribution of $Z_1$. A computed conditional error that decreases with $z$, or whose numerical average is inconsistent with the planned $\alpha$, can indicate a sign, scaling, or distribution error. For example, the denominator in the conditional normal tail is $\sqrt{1-t}$, the standard deviation of the remaining stage statistic after scaling—not $1-t$.

A nonbinding futility rule differs from two main alternatives:

Approach	What happens at the trigger?	Error-control implication
Nonbinding futility	Stopping is recommended but not required.	Continuing with the planned test retains its planned error; stopping without rejection cannot increase it.
Enforced stopping	The trial must stop and cannot reject after the trigger.	With the unchanged test elsewhere, stopping removes possible rejections and is conservative relative to the plan.
Changed continuation test	The trial continues with a different final rule.	Check its conditional rejection probability against $A_0(z)$, or provide another valid calibration.

These statements concern type I error under the specified null model. They do not establish that continuing is clinically appropriate, that the trial has adequate power, or that any modified procedure is valid under a composite null with nuisance parameters. In those settings, error control must be justified across the null parameter space using the relevant conditional distributions or bounds.

Common Pitfall

A frequent error is to reason, “The interim result crossed the futility boundary, so continuing must inflate type I error.” That conclusion confuses a recommendation to stop with a change to the rejection rule. If the trial continues and applies the original final test, its conditional rejection probability remains $A_0(z)$, and averaging over interim data gives the original $\alpha$. Conversely, another tempting but incorrect statement is that stopping at a futility boundary leaves the type I error unchanged. Stopping removes possible rejections in the trigger region, so it can only leave the error unchanged or reduce it.

Key takeaway: A nonbinding futility trigger does not require stopping for the planned test to control type I error. Continuing with the unchanged test preserves its planned error; stopping with no rejection can only reduce the rejection probability. If continuation is modified, compare its conditional rejection probability with the planned conditional error or justify the overall error by another valid method.

Check Your Understanding

Answer each question using the one-sided two-stage normal model unless the question says otherwise.

For $t=0.5$, $c=1.96$, and $Z_1=0$, write the conditional error formula and calculate its approximate value.
A nonbinding rule triggers at $Z_1\leq f$. The trial continues after a trigger and uses the original final boundary. What is the overall type I error, and why?
If the trial stops at $Z_1\leq f$ and cannot reject after stopping, express its type I error as an integral and compare it with $\alpha$.
At an interim value $z$, a proposed continuation test has $q(z)>A_0(z)$. What can and cannot be concluded from this inequality alone?

Solutions. (1) $A_0(0)=1-\Phi(1.96/\sqrt{0.5})=1-\Phi(2.7719)\approx0.00279$. (2) It is $\alpha$: for every interim result the unchanged continuation test has conditional error $A_0(z)$, whose null expectation is $\alpha$. (3) It is $\int_f^\infty A_0(z)\phi(z)\,dz=\alpha-\int_{-\infty}^f A_0(z)\phi(z)\,dz\leq\alpha$. (4) The pointwise conditional-error bound fails at that value, so the pointwise preservation guarantee is not established. The inequality alone does not prove that overall type I error exceeds $\alpha$; that requires evaluating or otherwise bounding the error over all interim values.

What You'll Learn