Worked Example Of A Two-Look Futility Design

Why Two-Look Futility Limits Must Be Calculated Jointly

A lower futility boundary is a rule for stopping when the accumulating evidence is sufficiently unfavorable relative to a prespecified design alternative. With two looks, the second boundary cannot generally be calculated as if the second statistic were the only statistic observed: the trial reaches look two only if it has not already crossed the first boundary. The design calculation must therefore account for the joint distribution across looks.

This worked example develops that calculation for a one-sided design in which larger standardized statistics favor the treatment. It focuses on the futility limits, not on deriving an efficacy boundary. The distinction between the probability of a futility crossing under a design alternative and the overall type II error is important: they are not generally the same quantity.

Model and assumptions: At look $k$, let $Z_k$ be the standardized statistic, with positive values favoring the treatment. Under the specified design alternative, assume $(Z_1,Z_2)$ is jointly normal, each statistic has variance 1, and $$ E(Z_k)=\mu_k=\delta\sqrt{I_k}, \qquad \operatorname{Corr}(Z_1,Z_2)=\sqrt{\frac{I_1}{I_2}}, $$ where $\delta>0$ is the standardized effect per square root of information and $I_1<I_2$ are the information levels. This canonical model follows from normal independent increments and is often used as a large-sample approximation. The design alternative, information levels, look order, and futility-spending schedule are prespecified.

Write the lower futility limits as $f_1$ and $f_2$. A crossing at look $k$ means $Z_k\le f_k$. For the purpose of calculating the planned futility crossings, the trial stops at the first such crossing. Let $\beta_F(t_k)$ denote the cumulative probability, under the design alternative, of having stopped for futility by look $k$, where $t_k=I_k/I_2$. This notation labels the spending target; it does not assert that this probability equals the overall type II error.

Two-look spending equations: Choose $f_1$ to satisfy $$ P(Z_1\le f_1\mid \text{design alternative})=\beta_F(t_1). $$ Then choose $f_2$ so that the probability of crossing at look two after not crossing at look one is the remaining spending: $$ P(Z_1>f_1,\ Z_2\le f_2\mid \text{design alternative}) =\beta_F(1)-\beta_F(t_1). $$

Deriving the Second-Look Equation

Under the model, the conditional distribution of $Z_2$ given $Z_1=z$ is normal with mean $\mu_2+\rho(z-\mu_1)$ and variance $1-\rho^2$, where $\rho=\sqrt{I_1/I_2}$. Thus the probability of a second-look crossing, among paths that continued past look one, is

P(Z_1>f_1,\ Z_2\le f_2) = \int_{f_1}^{\infty} \phi(z-\mu_1) \Phi\!\left( \frac{f_2-\mu_2-\rho(z-\mu_1)} {\sqrt{1-\rho^2}} \right)\,dz.

Here $\phi$ and $\Phi$ are the standard normal density and cumulative distribution function. The integrand is nonnegative, and increasing $f_2$ increases the conditional probability inside the integral. The integral therefore increases with $f_2$. Its limiting value is 0 as $f_2$ tends to negative infinity and $P(Z_1>f_1)$ as $f_2$ tends to positive infinity. Consequently, when the requested remaining spending is between those limits, there is a unique solution. In practice, numerical quadrature evaluates the integral and a one-dimensional root-finding method locates $f_2$.

Set the first limit.
Use the first cumulative spending probability and the marginal distribution of $Z_1$ to solve for $f_1$.

Compute the remaining spending.
Subtract the probability already spent at look one from the final cumulative futility-spending target.

Solve the continuation probability.
Evaluate the joint-probability integral over $z>f_1$ and find the $f_2$ that gives the remaining spending.

Verify the design.
Substitute the unrounded root into the integral and check that the attained probability matches the target to the chosen numerical tolerance.

Worked Example: Calculating Both Limits From a Spending Schedule

Consider a hypothetical one-sided trial with information levels $I_1=50$ and $I_2=100$. Under the design alternative, take $\delta=0.2602$. Then

\mu_1=0.2602\sqrt{50}\approx1.840,\qquad \mu_2=0.2602\sqrt{100}=2.602,\qquad \rho=\sqrt{50/100}=0.7071.

Assume the cumulative futility-spending schedule assigns $\beta_F(0.5)=0.0239$ by the first look and $\beta_F(1)=0.1239$ by the final look. The additional spending reserved for a crossing at look two, conditional on having continued, is therefore $0.1239-0.0239=0.1000$.

At look one, $Z_1\sim N(1.840,1)$. Set the first boundary to $f_1=-0.140$. The first crossing probability is

P(Z_1\le -0.140) = \Phi(-0.140-1.840) = \Phi(-1.980) \approx0.0239.

Thus the first limit meets the first spending target. For the second look, solve

\int_{-0.140}^{\infty} \phi(z-1.840) \Phi\!\left( \frac{f_2-2.602-0.7071(z-1.840)} {\sqrt{1-0.7071^2}} \right)\,dz =0.1000.

Numerical quadrature followed by root-finding gives $f_2\approx1.416$ (about $1.42$). Substitution of the unrounded numerical root into the integral gives an attained probability of approximately $0.1000$. The cumulative probability of a futility crossing by the end is therefore about $0.0239+0.1000=0.1239$, as specified.

The second threshold is higher than the first in this example, but that feature should not be treated as a general rule. The thresholds are determined by the spending schedule, the design alternative, and the dependence between looks. This calculation concerns futility crossings under the alternative; it does not by itself calculate the chance of failing to reject $H_0$.

Reading the Result and Checking the Calculation

The first equation uses a marginal probability because look one is the first opportunity to stop. The second uses a joint probability because its event includes both survival past look one and a crossing at look two. A marginal calculation such as $P(Z_2\le f_2)=0.1000$ would spend probability on paths that had already crossed the first boundary. It does not, in general, produce the specified 0.1000 increment in first-crossing probability.

A useful numerical check is to add the first-look crossing probability and the second-look continuation-crossing probability. In the example, the total is $0.0239+0.1000=0.1239$. This addition is valid because the two first-crossing events are disjoint: the second event explicitly requires $Z_1>f_1$. Also check that the remaining spending is nonnegative and does not exceed $P(Z_1>f_1)$; otherwise no finite $f_2$ can satisfy the equation.

Rounding is a separate operational issue. The unrounded roots should be used to verify the planned probabilities. If boundaries are rounded for a monitoring plan or software implementation, recompute the crossing probabilities at the rounded limits and document any resulting difference. A rounded value may be adequate for reporting but need not attain the target exactly.

Worked Example: Translating Spending Into a Conditional Probability

Using the first-look result above, the probability under the design alternative of reaching look two is

$$ P(Z_1>f_1)=1-0.0239=0.9761. $$

The planned probability of crossing for futility at look two, conditional on reaching that look, is therefore

P(Z_2\le f_2\mid Z_1>f_1) = \frac{P(Z_1>f_1,\ Z_2\le f_2)} {P(Z_1>f_1)} \approx\frac{0.1000}{0.9761} \approx0.1024.

So about 10.24% of paths that reach the second look cross its futility limit under this alternative. This is not a marginal tail probability for $Z_2$: conditioning on survival changes the distribution being considered. In particular, one should not simply find a normal quantile with lower-tail probability 0.1024 and use it as $f_2$.

Binding and Nonbinding Decisions

The crossing calculation specifies a planned stopping event; it does not alone determine what happens operationally when a lower limit is crossed. A binding futility rule is enforced: crossing it ends the trial, so later evidence is not used for a trial decision. A nonbinding futility rule is a recommendation that may be ignored. If it is ignored, monitoring can continue and a later efficacy crossing may still occur. Whether an efficacy design controls its error rate when a nonbinding recommendation is ignored depends on how that efficacy design was calibrated; the two operating procedures should not be treated as identical.

Worked Example: The Same Path Under Binding and Nonbinding Rules

Suppose, hypothetically, a trial has the futility limits $f_1=-0.140$ and $f_2=1.416$, and its observed first-look statistic is $Z_1=-0.200$. Since $-0.200\le-0.140$, the first-look futility criterion is met. Imagine also that a separate, hypothetical efficacy rule would declare efficacy at look two if $Z_2\ge2.000$, and that the statistic along this illustrative path would be $Z_2=2.100$ if monitoring continued.

Binding rule. The trial stops for futility at look one. The later statistic is not used to declare efficacy because the stopping rule has ended monitoring.

Nonbinding rule. The futility recommendation may be ignored. If the trial continues, $2.100\ge2.000$, so the separate hypothetical efficacy criterion is crossed at look two. This illustrates that a nonbinding futility crossing does not logically prevent a later efficacy crossing. It does not establish that the hypothetical efficacy rule controls its error rate under every monitoring policy.

Common Mistakes, Diagnostics, and Limitations

Using a marginal tail at look two. The second spending increment is $P(Z_1>f_1,Z_2\le f_2)$, not merely $P(Z_2\le f_2)$. The correction is to integrate only over paths that passed the first limit.
Calling the spending target the overall type II error. The schedule controls specified futility-crossing probabilities under the chosen alternative. Overall type II error also depends on the efficacy rule and the full decision procedure.
Changing the design alternative without recalculating. The means $\mu_1,\mu_2$ change with the alternative, which changes the limits that attain a given schedule. The resulting boundaries are design-specific.
Ignoring a numerical inconsistency. If quadrature at the proposed $f_2$ does not reproduce the remaining spending, do not report the limit as a solution. Check the means, correlation, integration range, and whether the calculation uses the unrounded root.

The normal canonical model is an approximation when the underlying statistics are not exactly normal or the information process is irregular. Dependence that differs from the independent-increments model, unplanned changes to look timing, or a change in the design alternative can invalidate the planned calculation. The design should therefore state its assumptions and use the same analysis model in planning and verification.

Key takeaway: Set the first futility limit using its marginal crossing probability. Set the second by solving the joint probability of continuing past the first limit and crossing the second. Then distinguish that planned first-crossing probability from the operational consequences of enforcing or ignoring the rule.

Check Your Understanding

Answer each question using the two-look setup and definitions in this tutorial. Short solutions follow.

Why does the second-look spending equation include the condition $Z_1>f_1$?
If the first-look crossing probability is 0.03 and the cumulative final futility-spending target is 0.15, what probability must be assigned to a first crossing at look two?
In the model above, what is the correlation between looks when $I_1/I_2=0.36$?
Suppose $Z_1\le f_1$ but a hypothetical later statistic would exceed an efficacy threshold. What happens under a binding futility rule, and what could happen under a nonbinding rule?
Why is a futility-crossing spending target not automatically the overall type II error?

Solutions. (1) The second-look event must exclude paths that already stopped at look one; otherwise those paths would be counted twice. (2) The increment is $0.15-0.03=0.12$. (3) $\rho=\sqrt{0.36}=0.600$. (4) A binding rule stops the trial at look one, so the later statistic is not used; a nonbinding recommendation can be ignored, allowing continued monitoring and a possible later efficacy crossing. (5) Overall type II error depends on the complete decision rule, including the efficacy criterion, while futility spending specifies crossing probabilities under the design alternative.