Futility Crossing Probability Versus Overall Beta

Two Probabilities With Different Meanings

A futility boundary describes a possible interim decision: the trial may stop because the evidence is not promising enough to continue. Under the design alternative, one can calculate how often the test statistic crosses that boundary. That probability is useful for describing the futility rule, but it is not, by itself, the probability that the trial fails to reject the null hypothesis.

Overall type II error concerns the final decision under a specified alternative. It includes every path on which the procedure does not reject \(H_0\). For a design that enforces futility stopping, those paths include early futility stops and trials that continue to the final analysis but do not cross the efficacy boundary. The distinction matters when assessing whether a design achieves its target beta.

The design alternative must be stated because both probabilities depend on the distribution of the test statistics. In this tutorial, consider a one-sided group sequential test with looks \(k=1,\ldots,K\), efficacy boundaries \(u_k\), and futility boundaries \(l_k\) at interim looks \(k<K\). At look \(k\), efficacy is declared if \(Z_k\ge u_k\); an enforced futility stop occurs if \(Z_k\le l_k\). Otherwise, the trial continues. At the final look, rejection occurs if \(Z_K\ge u_K\).

For calculations under \(\theta_1\), assume the test statistics follow the design's specified joint distribution. In the usual canonical normal model, \(Z_k\) has a normal distribution under the alternative, and the statistics across looks are correlated because each analysis uses accumulated data. The path probabilities below must be calculated from that joint distribution, or from a simulation that faithfully reproduces it. They are not generally obtained by treating the looks as independent.

Partitioning the Paths

For a binding futility rule, the non-rejection paths fall into two disjoint groups. First, the test may stop for futility before any efficacy rejection. Second, the test may pass all interim futility checks and reach the final look, but fail to cross the final efficacy boundary. Therefore,

The first term is the futility-crossing probability when crossing means the first stopping event is futility. The second term is also part of beta, but it is not a futility crossing. Thus, unless the second term is zero, overall beta is larger than the futility-crossing probability.

With multiple interim looks, calculate the first term using mutually exclusive first-crossing events. A trial that first crosses the futility boundary at look 2 must not also be counted as a look 1 futility crossing. If \(F_k\) denotes the event of a first futility stop at look \(k\), then

This sum describes the probability of stopping for futility, not the probability of every possible non-rejection. The remaining beta contribution comes from paths that continue to the final analysis and do not reject.

Several Interim Looks and Incremental Probabilities

When there are several interim looks, a cumulative crossing probability through look \(k\) includes crossings at earlier looks. An incremental first-crossing probability refers only to trials whose first futility crossing is at look \(k\). To form the probability of any futility stop, add the incremental probabilities, not cumulative probabilities.

Why Enforcing the Rule Changes the Interpretation

A binding futility boundary is part of the decision procedure: once crossed, the trial stops and cannot later reject. With a nonbinding rule, the boundary may be advisory. If it is not enforced, crossing it does not itself determine the final decision; the efficacy rule continues to govern the test. Consequently, a crossing probability can still be reported for a nonbinding boundary, but it should not be treated as the probability that the procedure fails to reject.

The following path comparison makes the distinction explicit. Imagine hypothetical outcomes under the design alternative classified by what would happen if the test continued after crossing futility. Of all paths, 0.12 cross futility; among these, 0.04 would later reject if allowed to continue and 0.08 would not. Among the 0.88 paths that do not cross futility, 0.70 reject and 0.18 do not. These categories sum to one.

This distinction is consistent with the purpose of nonbinding futility: an advisory recommendation can be ignored without changing the efficacy test's formal rejection rule. If the recommendation is routinely enforced in practice, however, the implemented procedure behaves like a binding rule for power and beta, and its operating characteristics should be evaluated accordingly.

Derivation, Diagnostics, and Limitations

The path decomposition follows directly from the definition of type II error. Under an enforced rule, consider a trial that does not reject. It either stops for futility at exactly one interim look, or it reaches the final look without a previous efficacy rejection or futility stop and has \(Z_K<u_K\). These events are mutually exclusive and cover all non-rejection paths. Additivity of probability therefore gives the beta expression above. Paths with an earlier efficacy rejection are excluded because they are not type II errors.

When checking a calculation, verify that the stopping paths form an exhaustive, non-overlapping partition under the stated rule. Useful checks include:

• The first-futility probabilities are disjoint and are not cumulative values added together.
• Early efficacy rejection, futility stopping, final rejection, and final non-rejection account for all paths, with total probability one up to numerical error.
• The final non-rejection probability is conditional on reaching the final look without an earlier stop; it must be weighted by the probability of reaching that state if reported conditionally.
• All probabilities are evaluated under the stated design alternative and the correct joint distribution across looks.
• The reported beta corresponds to the operational rule: an advisory boundary and an enforced boundary need not have the same power.

A useful comparison is between the crossing probability, overall beta, and power. For a specified alternative and decision rule, power is the probability of rejecting \(H_0\), and \(\beta(\theta_1)=1-\text{power}(\theta_1)\). Futility-crossing probability is instead one component of the non-rejection probability when stopping is enforced. It is not generally equal to \(1-\text{power}\), nor is it interchangeable with a target beta used to plan the design.

These quantities have limitations. They are tied to the chosen alternative, boundary values, information times, and whether the futility recommendation is followed. A crossing probability at one alternative does not describe performance at every effect size. Also, a probability calculated from an approximate joint normal model can differ from the probability under the actual design if the model, information pattern, or implementation differs. Simulation can provide a useful check, but it must reproduce the stopping rule and classify first crossings correctly.

Common Mistake

Incorrect reasoning: “The futility boundary is crossed with probability 0.12, so the design's type II error is 0.12.”

Correction: That conclusion omits paths that continue to the final analysis and fail to reject. Under an enforced rule, add the probability of those final non-rejections to the probability of futility stopping. In the first example, beta is \(0.12+0.08=0.20\), not 0.12. Always state whether the reported quantity is a crossing probability or overall beta, and specify whether futility is enforced.