Binding Futility Constraints and Boundary Calibration

Why Binding Futility Changes the Error Calculation

An interim futility rule can end an arm’s participation in a trial when its interim result is not promising enough. If the rule is binding, the protocol requires that the arm stop, so it cannot later reject its null hypothesis. Consequently, a later-stage efficacy boundary is reached only along paths that passed the futility rule. Familywise error must be calculated using those eligible paths, not as if every arm continued to every stage.

This distinction matters when efficacy boundaries are calibrated to a target familywise error rate (FWER). A calculation that ignores enforced futility includes later rejection opportunities that the design does not permit. It can therefore misstate the FWER of the actual design. Conversely, simply lowering boundaries because some paths stop is not a calibration method: the joint distribution of the stagewise statistics and the stopping rule determine the amount of error spent.

The derivations below use a two-stage normal model, with upper-tail efficacy tests. The same principle applies to more stages: each later-stage crossing probability must be evaluated only among the paths that have not already stopped or rejected.

Model and Binding Stopping Rule

Assume initially that all tested null hypotheses are true and that the stagewise statistics have a specified joint distribution under that null configuration. For a transparent calculation, take the canonical two-stage normal model: each stagewise statistic has a standard normal marginal distribution, the correlation between stages for an arm is \(\sqrt{t}\), and \(t\) is the interim information fraction. Then

Here the second argument of \(N\) is the variance. For multiple arms, their statistics may be correlated, for example because they share a control group. The full cross-arm and cross-stage covariance structure must therefore be specified for a joint FWER calculation. We first derive the path restriction for one arm; the multi-arm calculation then takes the probability of at least one arm’s rejection.

The assumptions behind this model include a valid null distribution for the stagewise statistics, the stated information fraction and covariance structure, prespecified boundaries and futility rule, and no unmodelled changes to the testing procedure. If the statistics or their covariance do not follow the assumed model, calculations based on it need not give the design’s true error rate.

From Eligible Paths to Familywise Error

For one arm, under its null hypothesis, the rejection probability is the probability of early efficacy plus the probability of final efficacy after passing futility and not crossing the interim efficacy boundary:

The integral follows by conditioning on \(Z_1=x\). The factor \(\phi(x)\,dx\) describes the probability density of that interim result, and the bracket is the conditional probability that the final statistic crosses \(c_2\). The limits \(f\) and \(c_1\) are essential: results at or below \(f\) are stopped, while results at or above \(c_1\) have already been counted as early rejection.

For \(M\) arms, let \(R_i\) denote arm \(i\)’s rejection event as defined above. The FWER under a specified null configuration is

Under the complete null, a direct calculation integrates over the joint interim distribution and, for each interim vector, over the conditional distribution of the final-stage statistics for arms that continued. Arms with \(Z_{i1}\le f\) have no later efficacy opportunity. Arms with \(Z_{i1}\ge c_1\) already reject, so such paths are excluded from the no-rejection probability. This is the multi-arm version of restricting the one-arm integral to eligible paths.

Calibration means choosing boundary values so that the calculated FWER equals, or does not exceed, the prespecified \(\alpha\). The integrals are often evaluated numerically. A candidate boundary should be checked by recomputing the joint rejection probability under the assumed covariance model, not just by checking each arm separately.

Boundary Calibration for Multiple Arms

A simple calibration illustration uses two independent arms with the same null rejection probability \(q\). Independence is an explicit simplifying assumption here; it is not generally appropriate when arms share a control group. For two independent arms, the probability that neither rejects is \((1-q)^2\), so the FWER is

To target \(\alpha=0.05\), the required per-arm probability is \(q=1-\sqrt{1-0.05}\approx0.02532\). The next example shows how to choose a final boundary under the same stopping rule.

Binding Versus Nonbinding Futility

With a nonbinding futility rule, the trial may stop for futility in practice, but the type I error calculation does not rely on that stop being enforced. For the single-arm setup with \(c_1=\infty\) and \(c_2=2.0\), the nonbinding calculation treats every path as eligible at stage 2. Since the final statistic has a standard normal marginal distribution, its rejection probability is \(1-\Phi(2.0)\approx0.02275\). Under binding futility, it is approximately \(0.0207\).

For two independent arms in this illustration, the corresponding FWERs are approximately \(1-(1-0.02275)^2=0.0450\) under the nonbinding calculation and \(1-(1-0.0207)^2=0.0410\) under enforced binding futility. The difference is modest here, but it is design- and boundary-dependent. A nonbinding calculation can be useful when a futility recommendation might not be followed; it is not the probability calculation for a design that enforces binding futility.

For a MAMS design, the comparison must also account for dependence among arms. Under a shared-control design, for example, one arm’s statistic may be correlated with another’s. Multiplying per-arm nonrejection probabilities as though the arms were independent would then give the wrong FWER. Earlier material on multivariate-normal calibration and recursive probability calculation provides the relevant joint-probability tools; the key adjustment here is to remove stopped arms’ later rejection paths.

Diagnostics, Limitations, and a Common Mistake

Before relying on a calibrated FWER, check that the calculation reflects the actual decision rule and model:

• Verify that the futility rule is truly binding in the protocol and that the continuation region is \(f<Z_{i1}<c_1\).
• Check that the interim and final boundaries use the same statistic scale and tail direction assumed in the calculation.
• Confirm the information fraction and the within-arm and cross-arm covariance structure used to obtain conditional distributions.
• Recalculate the union probability using the joint distribution when the arms are dependent; per-arm error rates alone do not determine FWER.
• Check whether control of FWER is required only under the complete null or under every configuration of true and false null hypotheses. Calibration under the complete null does not by itself establish strong control for every configuration; that conclusion requires appropriate additional justification for the design.

A practical diagnostic is to compare the numerical result with limiting cases. If \(f\) is moved far below any plausible interim statistic, the binding calculation should approach the calculation without futility stopping. If \(f\) is raised, fewer paths should remain eligible for later rejection. Also inspect numerical integration or simulation error, especially when boundaries are extreme or many arms are correlated.

Common mistake: “The final statistic has a standard normal distribution, so its final-boundary tail probability is the arm’s rejection probability, even when futility is binding.” The marginal distribution statement is correct, but the conclusion ignores the rule. Under binding futility, final rejection requires both continuation and a final crossing; its probability is \(P(Z_1>f,\ Z_2\ge c_2)\) when there is no interim efficacy boundary, not simply \(P(Z_2\ge c_2)\). For example, in the calculation above these probabilities are about \(0.0207\) and \(0.02275\), respectively.

Binding-futility calibration is limited by the accuracy of the assumed joint distribution and by the prespecified nature of the stopping and testing rules. If information fractions, analysis times, allocation, or the statistic’s distribution differ materially from the assumptions, the computed FWER may not describe the implemented design. The central lesson is not that binding futility always yields a particular reduction in error, but that its effect must be calculated on the paths the design actually allows.