Binding and Nonbinding Futility in Estimation

Why the Futility Rule Matters for Estimation

A futility boundary can be used to recommend stopping when interim results look unfavorable. Whether that boundary is binding is not merely a label for the boundary: it determines which data paths stop early and which continue to collect information. Since the reported estimate is based on the data collected before stopping, the rule also affects its repeated-sampling distribution and bias.

The comparison here isolates that effect in a two-look normal model. At the first look, both designs may stop for efficacy. They differ in what happens after crossing the futility boundary: the binding design stops, while the nonbinding design continues. The main result compares the unconditional bias of the estimates produced under those two specified rules. It does not assert that either estimate is unbiased in every setting.

Model and Stopping Rules

Assume a known-information normal model with two looks. Let \(I_1\) be the information at the interim look and \(I_2>I_1\) the information at the maximum look. Let \(S_k\) be the cumulative score at look \(k\), with independent increments. Specifically, \(S_1\sim N(\theta I_1,I_1)\), and \(S_2-S_1\sim N(\theta(I_2-I_1),I_2-I_1)\), independently of \(S_1\). The treatment effect \(\theta\) and its estimator are on the same scale; for example, if \(\theta\) is a mean difference, the units are those of the measured outcome.

At look \(k\), define the estimator \(\hat{\theta}_k=S_k/I_k\) and standardized statistic \(Z_k=S_k/\sqrt{I_k}\). Thus \(E_{\theta}(\hat{\theta}_k)=\theta\) if sampling always reaches look \(k\), before conditioning on any stopping event. The interim efficacy threshold is \(u\), and the futility threshold is \(l\), with \(l<u\). For clarity, assume the only efficacy stop is at look 1; trials that continue reach look 2.

This definition makes the operational comparison precise. A futility boundary called “nonbinding” does not, by itself, determine what investigators will do after crossing it. The calculation below assumes that all such trials continue. If a team sometimes stops and sometimes continues, the resulting rule is different and must be described and analyzed as such.

Deriving the Bias Difference

Write \(r=I_1/I_2\) for the fraction of total information available at the interim look, and \(\mu=\theta\sqrt{I_1}\) for the mean of \(Z_1\). Under the model, \(Z_1\sim N(\mu,1)\). The key relationship is the expected final estimate conditional on the interim data. The independent future increment has mean \(\theta(I_2-I_1)\), so

Thus, if a trial that would have stopped for futility instead continues, its conditional expected final-estimate error is \(r\) times its interim-estimate error. The difference between the interim error and that conditional expected final error is \((1-r)(\hat{\theta}_1-\theta)\).

Let \(A_L=\{Z_1\leq l\}\) be the futility-crossing event. The binding and nonbinding rules are identical outside \(A_L\), so their unconditional bias difference is the expected difference in reported estimation error on that event:

where \(I(A_L)\) is 1 when \(A_L\) occurs and 0 otherwise. Since \(\hat{\theta}_1-\theta=(Z_1-\mu)/\sqrt{I_1}\), the expectation is a lower-tail normal moment. For a standard normal variable \(W\), \(\int_{-\infty}^{a}w\phi(w)\,dw=-\phi(a)\), where \(\phi\) is the standard normal density. Taking \(a=l-\mu\) gives the required moment and hence

Because \(\phi(l-\mu)>0\) and \(0<r<1\), this difference is negative. In this model, enforcing the lower futility boundary shifts the unconditional mean of the reported estimate downward relative to continuing those trials. The size of the shift depends on the probability mass near the boundary, the information fraction, and the parameter value. It is not generally the entire bias of the binding estimator: the efficacy stopping rule also selects which estimates are reported early.

Conditional Bias and the Overall Comparison

A useful distinction is between bias among trials that stop for futility and bias across all trials. Conditional on \(Z_1\leq l\), the interim estimate is selected from the lower tail of its sampling distribution. Its conditional bias is

For the preceding numerical values, this is \(-0.2420/[10(0.1587)]=-0.1525\) outcome units. This conditional value describes the mean interim-estimate error among futility-crossing trials under the model. It is not the unconditional bias of the estimator across all trial outcomes. In a nonbinding design that always continues after crossing futility, there is no group of trials that stops for that reason under the specified rule.

Diagnostics, Alternatives, and Limitations

A reliable bias calculation starts with the stopping events actually used, not just the boundary values written in a design document. Record whether each interim outcome led to efficacy stopping, futility stopping, or continuation, and confirm which estimate would have been reported in each case. The event sets must be mutually exclusive and cover all possible outcomes. A mismatch between observed practice and the assumed rule invalidates a comparison based on that rule.

A practical diagnostic is to examine how often simulated or historical trial paths cross the futility boundary and how the reported estimate differs on those paths from the estimate that would have been available after continuation. In the model above, the predicted bias difference is zero if the futility event has essentially no probability, or if \(r\) is near 1; it is larger when appreciable probability lies near the lower boundary and substantial information remains to collect. A numerical result inconsistent with these features suggests checking the information fraction, event definition, or sign in the lower-tail calculation.

The principal alternatives differ in their stopping behavior and therefore their sampling distributions:

The third row is not a third operational futility rule. It is a reminder that choosing an estimator without accounting for the actual stopping process does not undo selection caused by that process. The derivation also has limitations: it assumes known information, normal scores, independent increments, and the exact two-look stopping rules stated above. If standard errors are estimated, stopping can occur at additional looks, or the nonbinding boundary is sometimes overridden, the formulas need not apply unchanged. In particular, “nonbinding” should not be treated as meaning that futility crossings have no effect on estimation; the effect depends on what the trial actually does.