Tutorials › Group Sequential Methods and Sample Size Re-Estimation › Computing Futility Stopping Probabilities

Computing Boundaries and Power · Tutorial 246 of 699

Computing Futility Stopping Probabilities

Learn to update the survivor distribution with a futility boundary and calculate, interpret, and check stopping probabilities across sequential looks.

Advanced 10 min read

What You'll Learn

  • Define efficacy stopping, futility stopping, and continuation regions at each look.
  • Derive the Gaussian conditional transition used to update the survivor density.
  • Calculate look-specific and cumulative stopping probabilities from that density.
  • Check recursive probabilities using mass balance and marginal-tail bounds.
  • Recognize when a futility rule is binding and understand how that changes operating probabilities.

Why a Lower Boundary Changes the Recursion

At an interim analysis, a group sequential design may stop not only when the evidence is sufficiently strong for efficacy, but also when it is sufficiently weak to make continuation unattractive. The lower boundary therefore removes paths from the population that can reach later looks. If those paths remain in the calculation, later efficacy and futility probabilities will be overstated. If they are removed twice, probabilities will be understated.

We use the canonical Gaussian information model. At look \(k\), let \(I_k\) be cumulative information, with \(0<I_1<\cdots<I_K\), and let \(Z_k\) be the standardized test statistic. Under drift \(\theta\), the cumulative score has mean \(\theta I_k\) and independent Gaussian increments, so \(Z_k\sim N(\theta\sqrt{I_k},1)\). The parameter \(\theta\) is the standardized effect per square root of information; its value is fixed for the probability calculation. Boundaries are fixed in advance, and the stated probabilities are conditional on this model and the specified \(\theta\).

Definition: At look \(k\), let \(a_k\) be a lower futility boundary and \(b_k\) an upper efficacy boundary, where \(a_k<b_k\). A path stops for futility at look \(k\) if it has reached that look and \(Z_k\le a_k\); it stops for efficacy if it has reached that look and \(Z_k\ge b_k\). It continues past an interim look if \(a_k<Z_k<b_k\). The survivor density at look \(k\) is the unnormalized density of \(Z_k\) among paths that have reached look \(k\) and are in its continuation region.

The word binding matters. A binding futility boundary is part of the stopping rule: paths below it are removed from all later calculations. A nonbinding futility boundary is a recommendation to stop that does not constrain the formal efficacy test if the study continues. The recursion below computes the actual operating probabilities for the rule being modeled; a boundary should be included as a stopping region only if crossing it truly stops the trial in that rule.

Conditional Transition and Survivor Update

Write \(f_k(z)\) for the survivor density at look \(k\), and set \(f_0\) aside because the first-look statistic has its own marginal distribution. The Gaussian increment model gives, for \(k\ge2\),

$$ Z_k\mid Z_{k-1}=x\sim N\left(\sqrt{\frac{I_{k-1}}{I_k}}x+ \frac{\theta(I_k-I_{k-1})}{\sqrt{I_k}}, \frac{I_k-I_{k-1}}{I_k}\right). $$

The second argument of \(N\) is the variance. This transition follows by writing the score at look \(k\) as the score at look \(k-1\) plus an independent increment of mean \(\theta(I_k-I_{k-1})\) and variance \(I_k-I_{k-1}\), then dividing by \(\sqrt{I_k}\). It is the Gaussian transition used in the earlier material on recursive integration.

Let \(q_k(y\mid x)\) denote the conditional normal density in that formula. At the first look, the unnormalized survivor density is the marginal normal density restricted to the continuation region. At each subsequent interim look, the update is

$$ f_k(y)=\mathbf{1}\{a_k<y<b_k\} \int_{a_{k-1}}^{b_{k-1}} f_{k-1}(x)q_k(y\mid x)\,dx. $$

The integration range contains only paths that survived look \(k-1\); the indicator then retains only paths that also continue at look \(k\). The probability of reaching look \(k\) is \(C_{k-1}=\int f_{k-1}(x)\,dx\). The look-specific probabilities are obtained by integrating the transition probabilities over this same incoming survivor density:

$$ F_k=\int_{a_{k-1}}^{b_{k-1}} f_{k-1}(x) P(Z_k\le a_k\mid Z_{k-1}=x)\,dx,\qquad E_k=\int_{a_{k-1}}^{b_{k-1}} f_{k-1}(x) P(Z_k\ge b_k\mid Z_{k-1}=x)\,dx. $$

Here \(F_k\) and \(E_k\) are probabilities of stopping specifically at look \(k\), not probabilities conditional on having reached it. At an interim look, \(C_k=\int f_k(y)\,dy=C_{k-1}-F_k-E_k\). At the final planned look, the remaining paths are terminal non-crossings: they have not crossed either boundary by the end of the design, rather than continuing to another planned look.

Worked Example: Two Looks With a Futility Boundary

Consider a hypothetical two-look design with \(I_1=1\), \(I_2=2\), and \(\theta=0.3\). At look 1, use boundaries \(a_1=-1\) and \(b_1=2\). At look 2, use \(a_2=-1.5\) and \(b_2=2\). The calculation below uses a deliberately coarse three-bin midpoint approximation to the survivor distribution at look 1; it illustrates the recursion, so its look-2 probabilities are approximate, not high-precision design results.

At look 1, \(Z_1\sim N(0.3,1)\). Thus \(F_1=\Phi(-1.3)\approx0.0968\), \(E_1=1-\Phi(1.7)\approx0.0446\), and the probability reaching look 2 is

$$ C_1=\Phi(1.7)-\Phi(-1.3)\approx0.9554-0.0968=0.8586. $$

For the transition, \(\sqrt{I_1/I_2}=1/\sqrt{2}\), the conditional variance is \(1/2\), and the conditional mean at a value \(Z_1=x\) is \(x/\sqrt{2}+0.3/\sqrt{2}\). Divide the surviving interval \((-1,2)\) into three unit-width bins and use midpoints \(x=-0.5,0.5,1.5\). The marginal densities at those midpoints are approximately \(0.2897,0.3910,0.1942\). Normalizing these three weights gives approximate survivor proportions \(0.3311,0.4469,0.2220\). Normalization makes their total equal to the known incoming survivor mass \(C_1\).

At the three midpoints, the conditional means are approximately \(-0.1414,0.5657,1.273\), each with standard deviation \(\sqrt{0.5}=0.7071\). The conditional probabilities of crossing the lower boundary \(-1.5\) are approximately \(0.02734,0.00174,0.000044\); those of crossing the upper boundary \(2\) are approximately \(0.00123,0.0213,0.1520\). Weighting each set by the normalized survivor proportions gives

$$ F_2\approx0.8586[(0.3311)(0.02734)+(0.4469)(0.00174)+(0.2220)(0.000044)] \approx0.00845, $$
$$ E_2\approx0.8586[(0.3311)(0.00123)+(0.4469)(0.0213)+(0.2220)(0.1520)] \approx0.0375. $$

The remaining probability at look 2 is approximately \(C_2=0.8586-0.00845-0.0375=0.8127\). This is terminal non-crossing probability in this two-look design. The approximation respects the incoming-mass balance, and its efficacy probability is below the marginal upper-tail probability at look 2, as it must be. A finer integration grid would be needed for precise design calculations.

Mass Balance and a Useful Probability Bound

The recursion has two immediate checks. First, each arriving path must stop for futility, stop for efficacy, or remain in the continuation region. Hence, at an interim look, \(F_k+E_k+C_k=C_{k-1}\), up to numerical integration error. Across the whole design, the sum of all look-specific stopping probabilities plus the terminal non-crossing probability must equal one.

Second, look-specific efficacy stopping at look \(k\) is a subset of the event \(Z_k\ge b_k\). Consequently,

$$ E_k\le P(Z_k\ge b_k)=1-\Phi(b_k-\theta\sqrt{I_k}). $$

The inequality is usually strict when earlier stopping boundaries remove some paths that otherwise could have reached the efficacy region. A computed \(E_k\) larger than this marginal tail probability signals an error in the calculation, its inputs, or its interpretation.

Worked Example: Checking the Look-2 Efficacy Probability

For the two-look design above, the marginal look-2 statistic has distribution \(Z_2\sim N(0.3\sqrt{2},1)\). Therefore the probability of exceeding the upper boundary \(2\), without regard to earlier stopping, is

$$ P(Z_2\ge2)=1-\Phi(2-0.3\sqrt{2}) =1-\Phi(1.576)\approx0.0575. $$

The approximate recursive value \(E_2\approx0.0375\) is below this bound. That is plausible: the marginal tail includes paths that may already have stopped at look 1, whereas \(E_2\) counts only paths that survived to look 2 and then crossed its upper boundary. A value such as \(0.0600\) would fail this check and could not be the look-2 efficacy stopping probability under these assumptions.

Worked Example: Auditing the Whole Two-Look Calculation

Use the rounded probabilities from the first example: \(F_1=0.0968\), \(E_1=0.0446\), \(F_2\approx0.00845\), \(E_2\approx0.0375\), and terminal non-crossing probability \(0.8127\). The total is

$$ 0.0968+0.0446+0.00845+0.0375+0.8127 =1.00005\approx1.000. $$

The small discrepancy from one is due to rounding. The probability of ever stopping for futility is \(F_1+F_2\approx0.1053\), and the probability of ever stopping for efficacy is \(E_1+E_2\approx0.0821\). Together with terminal non-crossing, these cumulative categories account for the full probability mass. They are operating probabilities under the specified drift and boundaries, not conditional probabilities among only the paths that reached a particular look.

Numerical Practice, Comparisons, and Failure Modes

In computation, the same survivor update can be evaluated by numerical integration over the previous survivor region. The main alternative is direct evaluation of joint Gaussian probabilities for the crossing regions. The recursive approach is naturally organized by look and exposes the survivor mass at every stage; a direct joint calculation can provide an independent check for a small number of looks. Both approaches rely on the same Gaussian model and correctly specified stopping regions. Neither repairs a misspecified information schedule or an incorrectly coded boundary.

For each look, inspect whether the arriving mass is nonnegative, whether \(F_k+E_k\le C_{k-1}\), and whether the mass balance holds to the intended numerical tolerance. Also compare \(E_k\) with the marginal-tail bound above. Large changes in the calculated probabilities when the integration grid is refined suggest inadequate numerical resolution, especially near a boundary. A failure to account for the lower tail, or an update that integrates over all possible previous \(Z\) values instead of only survivors, is a more fundamental region-definition error.

The method has limits. The Gaussian transition assumes the canonical independent-increment model and the specified information values. Boundaries must be applied using the statistic and scale for which they were designed. The resulting probabilities can change substantially with \(\theta\); a calculation under one drift does not describe every alternative. Finally, a terminal non-crossing probability is not an interim continuation probability: at the maximum planned look, there is no further planned analysis to continue to.

  • Wrong: “The lower boundary only changes the label assigned to a final statistic.” Correction: A binding futility crossing removes that path from every later look.
  • Wrong: “The look-2 efficacy probability is the marginal probability \(P(Z_2\ge b_2)\).” Correction: That marginal probability includes paths stopped earlier; recursive \(E_2\) includes only paths that arrived at look 2.
  • Wrong: “Any leftover mass at the final look is interim continuation.” Correction: It is terminal non-crossing probability for the planned design.

Check Your Understanding

Answer each question using the model and definitions above. The short solutions follow the questions.

  1. For \(I_{k-1}=1\), \(I_k=2\), \(\theta=0.3\), and \(Z_{k-1}=0.5\), what are the conditional mean and variance of \(Z_k\)?
  2. Under \(\theta=0\), what is the probability of stopping for futility at a single look with lower boundary \(a=-1\)?
  3. Why must the look-2 efficacy stopping probability in the worked design be no greater than about \(0.0575\)?
  4. If \(C_{k-1}=0.70\), \(F_k=0.08\), and \(E_k=0.12\), what is the interim continuation probability \(C_k\)?

Solutions. (1) The mean is \(0.5/\sqrt{2}+0.3/\sqrt{2}=0.8/\sqrt{2}\approx0.5657\), and the variance is \((2-1)/2=0.5\). (2) \(P(Z\le-1)=\Phi(-1)\approx0.1587\). (3) Efficacy stopping at look 2 is a subset of \(Z_2\ge2\); under the stated drift, the latter probability is \(1-\Phi(2-0.3\sqrt{2})\approx0.0575\). (4) Mass balance gives \(C_k=0.70-0.08-0.12=0.50\).

Clinical Biostats