Worked Example With Three Unequally Timed Looks

Why Unequally Timed Looks Require Recursive Calculation

A three-look design can allocate its information unevenly: the first analysis might occur after relatively little information, the second after a larger increment, and the final analysis at the planned maximum. Under the design alternative, a futility rule stops when the observed evidence is too weak to justify continuing. The boundary at each look must be calibrated to the chance of stopping at that look, given that the trial has not already stopped. Unequal timing affects the dependence among the interim statistics, so marginal normal probabilities alone do not determine the boundaries.

The calculation below concerns lower futility boundaries only. It assumes a prespecified alternative and an information schedule, and it treats the stopping rule as binding: once a boundary is crossed, subsequent analyses are not conducted. Efficacy testing and its error spending are separate design components.

Definition: Let $0<t_1<t_2<t_3=1$ be the information fractions at three analyses, and let $Z_i$ be the standardized test statistic at look $i$. Under the canonical group sequential model, $$ (Z_1,Z_2,Z_3)\sim N\left( \begin{pmatrix} \delta\sqrt{t_1}\\ \delta\sqrt{t_2}\\ \delta\sqrt{t_3} \end{pmatrix}, \begin{pmatrix} 1 & \sqrt{t_1/t_2} & \sqrt{t_1/t_3}\\ \sqrt{t_1/t_2} & 1 & \sqrt{t_2/t_3}\\ \sqrt{t_1/t_3} & \sqrt{t_2/t_3} & 1 \end{pmatrix} \right) $$ under the design alternative. Here $\delta$ is the standardized effect at full information. The futility rule stops at look $i$ when $Z_i\leq l_i$, provided the trial reached that look.

This model assumes that information is correctly represented by the fractions $t_i$, the statistic has the stated joint normal distribution (exactly or as a suitable approximation), and the alternative used for calibration is fixed in advance. In particular, the correlation between looks $i$ and $j$ is $\sqrt{t_i/t_j}$ for $i<j$. A different analysis statistic, changing information schedule, or materially non-normal sampling distribution can invalidate the calculations.

Incremental Spending and the Boundary Equations

Let $b_i$ denote the target probability of first stopping for futility at look $i$, under the design alternative. Then the cumulative beta spending through look $i$ is $B_i=\sum_{j=1}^{i}b_j$. This is a probability under the specified alternative and the binding stopping rule; it is not the probability of stopping at look $i$ without regard to earlier stops.

For the first look, calibration is a one-dimensional normal quantile. At look 2, the boundary is chosen by integrating over the possible $Z_1$ values that survive the first boundary. At look 3, both earlier survival conditions must be imposed. Write $f_i$ for the joint normal density of the first $i$ statistics under the design alternative. The recursive equations are:

\begin{aligned} P(Z_1\leq l_1)&=b_1,\\ \int_{l_1}^{\infty}\int_{-\infty}^{l_2} f_2(z_1,z_2)\,dz_2\,dz_1&=b_2,\\ \int_{l_1}^{\infty}\int_{l_2}^{\infty}\int_{-\infty}^{l_3} f_3(z_1,z_2,z_3)\,dz_3\,dz_2\,dz_1&=b_3. \end{aligned}

These equations define the first-stop probabilities, not the cumulative probabilities. The second equation includes $Z_1>l_1$, and the third includes both $Z_1>l_1$ and $Z_2>l_2$. Since each event grows as its current lower boundary rises, the left side is nondecreasing in that boundary. Before root finding, require $0<b_1<1$ at the first look and, at each later look $i$, $0<b_i<P(Z_1>l_1,\ldots,Z_{i-1}>l_{i-1})$. For finite boundaries in the continuous normal model, the event probability approaches zero and the probability of reaching look $i$ only as the current boundary tends to negative and positive infinity, respectively; endpoint targets must therefore be handled separately. When the target is within the attainable range, a numerical root finder can solve each equation in sequence, using the already-calibrated earlier boundaries.

Justification: The first-stop event at look 2 is $\{Z_1>l_1,\ Z_2\leq l_2\}$; at look 3 it is $\{Z_1>l_1,\ Z_2>l_2,\ Z_3\leq l_3\}$. These events are mutually exclusive. Thus their probabilities add: $$ P(\text{futility by look }i)=\sum_{j=1}^{i}P(\text{first futility stop at }j)=B_i. $$ This identity explains why a target cumulative spending schedule must be converted to increments before solving the recursive equations.

Worked Three-Look Calculation

Consider a hypothetical design with unequal information fractions $t_1=0.25$, $t_2=0.60$, and $t_3=1.00$, calibrated under a prespecified design alternative with standardized effect $\delta$. The cumulative beta-spending targets are $B_1=0.05$, $B_2=0.15$, and $B_3=0.30$, so the incremental targets are $b_1=0.05$, $b_2=0.10$, and $b_3=0.15$. The numerical boundaries depend on $\delta$; since no numerical value is assigned here, the calculations below specify the equations but do not give numerical boundary or probability values. The first-look boundary is obtained from the alternative distribution at $t_1$; the next two require the recursive integrals above.

Worked Example: Solving the Second-Look Boundary

After finding $l_1$ from $P(Z_1\leq l_1)=0.05$, solve $$ \int_{l_1}^{\infty}\int_{-\infty}^{l_2} f_2(z_1,z_2)\,dz_2\,dz_1=0.10. $$

For any specified $\delta$, solve the displayed integral using the corresponding $l_1$ and joint normal density $f_2$. The root is not obtained by solving $P(Z_2\leq l_2)=0.10$: that marginal probability includes paths that would already have stopped at look 1. The calculation retains only paths with $Z_1>l_1$.

After specifying $\delta$, check the resulting root by substituting its unrounded value back into the integral; the probability of first stopping at look 2 should be $0.10$, up to numerical integration tolerance. The cumulative probability of stopping by look 2 is consequently $0.05+0.10=0.15$, not $0.10$.

Worked Example: Solving the Third-Look Boundary

Continue with the same hypothetical plan. The third boundary must satisfy $$ \int_{l_1}^{\infty}\int_{l_2}^{\infty}\int_{-\infty}^{l_3} f_3(z_1,z_2,z_3)\,dz_3\,dz_2\,dz_1=0.15. $$

For any specified $\delta$, use the unrounded first- and second-look boundaries in this integral and solve for $l_3$. Check the resulting root by substitution: under the stated survival conditions its first-stop probability should be $0.15$, up to numerical integration tolerance. Raising the boundary increases the event $\{Z_3\leq l_3\}$ among paths that reached look 3.

The cumulative futility-stop probability by the final look is $0.05+0.10+0.15=0.30$. Equivalently, the probability of reaching the final analysis is $1-0.30=0.70$, under this alternative and binding rule. This is a probability of reaching the analysis, not a guarantee that the final test will reject.

Final Non-Rejection and the Operating Characteristics

To assess final non-rejection, the final decision rule must also be specified. For illustration, suppose the design uses a one-sided final efficacy critical value $c_3$, so that a reached trial rejects at the final analysis only when $Z_3\geq c_3$. The probability of final non-rejection is the probability of reaching look 3 and then not crossing this efficacy limit:

P(Z_1>l_1,\ Z_2>l_2,\ Z_3<c_3).

This probability includes both paths that stop for futility at look 3 and paths that reach the final analysis but do not reject. If $c_3>l_3$, these disjoint components can be written as:

P(\text{final non-rejection})= P(Z_1>l_1,Z_2>l_2,Z_3\leq l_3) + P(Z_1>l_1,Z_2>l_2,l_3<Z_3<c_3).

The first term is the third-look futility increment, $b_3=0.15$. The second term is the probability of continuing through futility monitoring but failing the final efficacy test. It must be evaluated from the same joint model; it cannot be inferred from $B_3$ alone. In the absence of a specified efficacy limit, the design determines the probability of reaching the final analysis and stopping for futility, but it does not determine a final efficacy non-rejection probability.

Worked Example: Separating Futility From Final Non-Rejection

Suppose the three-look calculation above is paired with a prespecified final critical value $c_3$, with $l_3<c_3$. The probability of final futility stopping is the calibrated third increment, $0.15$. The additional final non-rejection probability among trials that pass the third futility boundary is $$ P(Z_1>l_1,\ Z_2>l_2,\ l_3<Z_3<c_3). $$

This last quantity requires integration over the joint distribution and earlier survival region. For example, it can be computed as the difference between the same three-dimensional integral with upper limits $1.96$ and $1.45$ on $z_3$. Adding that result to $0.15$ gives the total probability of final non-rejection under the alternative. The numerical value depends on the alternative mean vector and correlations, so it must not be replaced by $0.15$ or by $1-B_2=0.85$.

Implementation, Diagnostics, and Alternatives

A reliable calculation follows the recursion strictly. Use the same alternative distribution and unrounded earlier boundaries at every step. Numerical integration may use conditional normal distributions, adaptive quadrature, or validated multivariate-normal probability software. Simulation can provide an independent check, but it is less precise for small probability increments unless the number of simulated paths is sufficiently large.

Specify the model and targets.
Record the information fractions, design alternative, binding stopping rule, and cumulative beta targets. Convert cumulative targets to increments $b_i=B_i-B_{i-1}$, with $B_0=0$.

Find the first boundary.
Solve the marginal first-look equation. Check that the probability under the design alternative equals $b_1$.

Find later boundaries recursively.
At each look, integrate only over paths satisfying all earlier survival conditions, then solve for the boundary that gives the current incremental target.

Verify and report.
Re-evaluate each first-stop probability using unrounded boundaries; sum the increments to check cumulative spending, then calculate reach and final-decision probabilities separately.

Useful diagnostics include checking that every increment is nonnegative, that cumulative spending is nondecreasing, and that each substituted boundary reproduces its target within the numerical tolerance. If a computed probability is materially off, check for a missing earlier survival condition, a reversed inequality, an incorrect correlation, or use of a rounded boundary during later integration. A simulation check should estimate first-stop frequencies by look, not count every path below a marginal boundary as a new stop.

Recursive quadrature is the direct approach for a small number of looks and a specified joint normal model. Multivariate-normal probability routines can be more convenient, while simulation is flexible for more complicated models but introduces Monte Carlo error. None of these methods repairs a misspecified alternative or an incorrect information schedule. Furthermore, beta spending is defined relative to the chosen alternative: changing that alternative changes the boundaries and their operating characteristics. Binding and nonbinding futility rules also differ in their operational consequences; the calculations here assume the rule is enforced.

Common Mistakes

Using marginal probabilities at later looks. Solving $P(Z_2\leq l_2)=b_2$ counts paths that already stopped at look 1. Integrate over $Z_1>l_1$ instead.
Confusing increments and cumulative spending. If $B_2=0.15$ and $B_1=0.05$, the second-look target is $b_2=0.10$, not $0.15$.
Ignoring dependence across looks. The statistics are correlated under the canonical model. Treating them as independent changes first-stop probabilities and boundaries.
Using rounded boundaries in later steps. A rounded value can noticeably alter a small probability increment. Retain full numerical precision for integration and round only when reporting.
Calling futility spending the final non-rejection probability. Futility-stop probability omits trials that reach the final analysis but do not reject. The final decision rule is needed to calculate total non-rejection.

Key takeaway: With unequal information fractions, calculate each boundary from the probability of a first futility stop at that look, conditional through the event definition on surviving every earlier boundary. Verify incremental probabilities first; cumulative spending and final non-rejection are distinct summaries.

Check Your Understanding

Answer each question using the binding futility rule and the design alternative unless stated otherwise.

Cumulative spending is $B_1=0.04$, $B_2=0.13$, and $B_3=0.25$. What are the three incremental targets?
Why does the second-look integral begin at $z_1=l_1$ and extend to infinity?
A calculation reports a second-look first-stop probability of $0.09$ and a first-look probability of $0.04$. What is the cumulative futility-stop probability by look 2?
At the final look, $l_3<c_3$. Which two types of paths contribute to final non-rejection?
Why should the unrounded value of $l_2$ be used when calculating $l_3$?

Solutions. (1) $b_1=0.04$, $b_2=0.13-0.04=0.09$, and $b_3=0.25-0.13=0.12$. (2) The trial reaches look 2 only if $Z_1>l_1$; values at or below $l_1$ have already stopped for futility. (3) The cumulative probability is $0.04+0.09=0.13$. (4) Paths stopping for futility at look 3, with $Z_3\leq l_3$, and paths passing futility but failing efficacy, with $l_3<Z_3<c_3$, provided they survived both earlier boundaries. (5) The third-look probability conditions on $Z_2>l_2$; rounding changes that survival region and can shift the calibrated third increment.