Updating Boundaries at Later Looks

Why a Later Boundary Must Account for Earlier Looks

At a later interim analysis, some of the nominal type I error has already been spent: under $H_0$, there was a chance of crossing an efficacy boundary at each earlier look. The next boundary therefore cannot be chosen as if the current test were the first test. It must be calibrated against the probability of crossing at any look through the current one.

This tutorial considers upper efficacy boundaries for canonical test statistics under the null. The spending function supplies a cumulative target at each information fraction; boundary updating converts that target into a cutoff while accounting for the joint distribution of the statistics. Choosing the spending function is a separate design decision.

Definition: Let $Z_k$ be the canonical test statistic at look $k$, with information $I_k$, and let $b_k$ be its upper efficacy boundary. The cumulative null crossing probability through look $j$ is $$ A_j(b_1,\ldots,b_j)=P_0\left(\bigcup_{k=1}^{j}\{Z_k\ge b_k\}\right). $$ If the cumulative spending target at information fraction $t_j=I_j/I_{\max}$ is $\alpha(t_j)$, the updated boundary $b_j$ is chosen to satisfy $A_j=\alpha(t_j)$, with the earlier boundaries held fixed.

The canonical null model assumes standardized statistics with mean zero, variance one, and joint normal increments. For looks ordered by increasing information, their covariance is $\operatorname{Cov}_0(Z_i,Z_j)=\sqrt{I_i/I_j}$ when $i<j$. Thus successive statistics are dependent. The update also assumes that the information fractions and the analysis order are specified, that the null distribution is adequately represented by the canonical model, and that the crossing event being calibrated is the one stated. Adding futility stopping or changing the test statistic changes the event or its distribution and requires the corresponding calibration.

The Later-Look Calibration Equation

Suppose the boundaries through look $j-1$ have already been set so that $A_{j-1}=\alpha(t_{j-1})$. A crossing by look $j$ occurs either earlier, or for the first time at look $j$. These events are disjoint, so the new boundary must make the probability of a first crossing at look $j$ equal to the additional amount of cumulative spending:

P_0\left(Z_1<b_1,\ldots,Z_{j-1}<b_{j-1},\,Z_j\ge b_j\right) =\alpha(t_j)-\alpha(t_{j-1}).

The strict inequalities describe not having crossed earlier. For continuous statistics, replacing them with non-strict inequalities does not change the probability. This equation is the central update: the right side is the incremental spending, while the left side is a joint probability under the correlated null distribution.

In the canonical model, the process has a useful conditional representation. Given the statistic at the immediately preceding look, $Z_j\mid Z_{j-1}=z$ is normal with mean $\sqrt{I_{j-1}/I_j}\,z$ and variance $1-I_{j-1}/I_j$. The canonical process is Markov in information time, so this conditional distribution can be used along with the joint distribution of the earlier statistics. For two looks, the calibration equation is the one-dimensional integral

\int_{-\infty}^{b_1} \left[ 1-\Phi\left( \frac{b_2-\sqrt{I_1/I_2}\,z} {\sqrt{1-I_1/I_2}} \right) \right]\phi(z)\,dz =\alpha(t_2)-\alpha(t_1),

where $\phi$ and $\Phi$ are the standard normal density and distribution functions. For more looks, the integral is over all earlier statistics that have not crossed their respective boundaries. Numerical multivariate-normal integration is commonly used.

Why the Numerical Root Is Well Defined

With previous boundaries fixed, increasing $b_j$ can only reduce the chance of a first crossing at look $j$. Under a continuous normal model, that probability changes continuously with $b_j$. As $b_j$ tends to infinity, the chance of a new crossing tends to zero; as it tends to negative infinity, it tends to the probability of not having crossed earlier. Therefore, if the requested incremental spending lies strictly between these limits, a boundary can be found by a one-dimensional root search. It is unique when the probability decreases strictly over the relevant range.

The target must also be compatible with the earlier calibration. Cumulative spending should be nondecreasing: $\alpha(t_j)\ge\alpha(t_{j-1})$. If a proposed target is smaller than the probability already spent, no later cutoff can undo those earlier crossings. In a valid spending design, the incremental target is nonnegative and no greater than the probability available among paths that have not yet crossed.

Worked Example: Updating a Boundary at the Second Look

Consider a hypothetical two-look design with $I_1/I_2=0.5$. Suppose the first boundary is $b_1=2.576$, corresponding to approximately $0.005$ upper-tail probability under $H_0$, and the cumulative target at the second look is $0.025$. The additional spending is $\,0.025-0.005=0.020$.

Under the canonical model, the correlation between the two statistics is $\sqrt{0.5}=0.7071$. The second boundary solves

\int_{-\infty}^{2.576} \left[ 1-\Phi\left( \frac{b_2-0.7071z}{0.7071} \right) \right]\phi(z)\,dz=0.020.

The integral is evaluated numerically and the difference between its value and $0.020$ is set to zero by root-finding. A numerical solution is approximately $b_2=2.00$ (about $2.00$ to two decimal places). At this cutoff, the probability of a first crossing at look 2 is approximately $0.020$; adding the $0.005$ probability already spent at look 1 gives a cumulative crossing probability of approximately $0.025$. The calculation uses the conditional distribution given $Z_1$, not an independent-test approximation.

Updating Across Several Looks

For three or more looks, the same principle applies at every update. Each new equation includes all earlier noncrossing conditions. In practice, a program evaluates the probability of being below each earlier boundary and crossing the candidate boundary at the current look, then adjusts the candidate until the incremental target is met.

Record the design inputs.
List the information fractions, cumulative spending targets, and previously calibrated boundaries. Check that information increases and cumulative targets do not decrease.

Calculate the increment.
At look $j$, compute $\Delta_j=\alpha(t_j)-\alpha(t_{j-1})$, taking the spending before the first look as zero.

Evaluate the first-crossing probability.
For a trial value of $b_j$, calculate the null probability that all earlier statistics remain below their boundaries and $Z_j\ge b_j$, using the canonical joint distribution.

Solve and verify.
Use a bracketed root-finding method to match that probability to $\Delta_j$. Recalculate the cumulative crossing probability through look $j$ as a check.

Worked Example: Three Looks With Unequal Information Increments

Suppose a hypothetical design has information fractions $t_1=0.20$, $t_2=0.60$, and $t_3=1.00$, with cumulative targets $0.002$, $0.012$, and $0.025$. The spending increments are \[ \Delta_1=0.002,\qquad \Delta_2=0.012-0.002=0.010,\qquad \Delta_3=0.025-0.012=0.013. \] The first upper boundary is the standard normal quantile with upper-tail probability $0.002$, approximately $b_1=2.878$.

At look 2, the correlation with $Z_1$ is $\sqrt{0.20/0.60}=0.5774$. The boundary is obtained by solving

P_0(Z_1<2.878,\ Z_2\ge b_2)=0.010.

At look 3, the update must account for both earlier looks. The relevant equation is

P_0(Z_1<2.878,\ Z_2<b_2,\ Z_3\ge b_3)=0.013.

Here $\operatorname{Corr}_0(Z_2,Z_3)=\sqrt{0.60}=0.7746$, and $\operatorname{Corr}_0(Z_1,Z_3)=\sqrt{0.20}=0.4472$. A numerical routine first solves for $b_2$, then holds $b_1$ and $b_2$ fixed while solving for $b_3$. It does not assign a fresh $0.025$ tail probability at look 3: that would ignore the $0.012$ cumulative probability already spent. The checks are that the first-crossing probabilities are approximately $0.002$, $0.010$, and $0.013$, and their sum is $0.025$.

Diagnostics, Alternatives, and Limitations

A useful diagnostic is to reconstruct cumulative crossing probabilities after calibration. At each look $j$, verify that $P_0(\text{cross by }j)$ agrees with $\alpha(t_j)$ to the numerical tolerance. Also inspect the residual in the root equation, the integration tolerance, the root-search interval, and the information fractions used in the covariance model. A boundary that is unexpectedly low or high can signal an incorrect target, a mistaken information fraction, or a calculation that treated dependent statistics as independent.

The main alternatives are not interchangeable with this update. A fixed classical cutoff applies a specified critical value at each look, and its overall crossing probability is determined by that rule; it does not automatically match an arbitrary cumulative spending curve. Allocating incremental alpha as if each look were an independent test also fails here, because the canonical statistics are correlated. Numerical integration is generally preferable when available; simulation can provide a check or an approximation, but its Monte Carlo error should be reported and may be material for small tail probabilities.

This calibration is only as reliable as its model and inputs. The canonical normal approximation may be inadequate for some finite-sample or data-dependent settings. Repeatedly changing the analysis schedule without using a method valid for the actual timing can also invalidate the intended error control. If the test includes a lower efficacy boundary, a futility rule, or a two-sided crossing event, the event probability must include those features; the upper-boundary equation shown here is not by itself a calibration for them.

Common Mistake

Pitfall: “The cumulative target at look 2 is $0.025$, so use a boundary with a $0.025$ upper-tail probability at look 2.” This treats the second look as if there had been no earlier opportunity to reject. The correction is to solve for the probability of a first crossing at look 2 equal to $\alpha(t_2)-\alpha(t_1)$, conditional in the joint calculation on not having crossed at look 1. Then verify that the total crossing probability through look 2 equals $0.025$.

Key takeaway: A later boundary is determined by cumulative probability, not by a marginal tail area at the current look. Hold earlier boundaries fixed, calculate the correlated null probability of crossing for the first time now, and solve for the boundary that matches the additional spending.

Check Your Understanding

Try each question, then compare your reasoning with the solutions below.

If cumulative spending rises from $0.004$ at look 1 to $0.015$ at look 2, what first-crossing probability must the second boundary produce?
Why is the marginal probability $P_0(Z_2\ge b_2)$ generally not the probability spent at look 2?
For information fractions $t_1=0.25$ and $t_2=1$, what is the null correlation of the two canonical statistics?
A proposed cumulative target falls from $0.010$ at one look to $0.008$ at the next. Can a new boundary achieve that target? Explain.
After calibrating a boundary, what probability should be checked to verify the cumulative target?

Solutions. (1) The increment is $0.015-0.004=0.011$, so the probability of a first crossing at look 2 must be $0.011$. (2) The marginal probability includes paths that already crossed at look 1; the look-2 increment is the joint probability of no earlier crossing and a crossing now. (3) The correlation is $\sqrt{0.25/1}=0.5$. (4) No. A later boundary cannot remove the $0.010$ crossing probability already incurred, so the cumulative target must not decrease. (5) Check $P_0(\text{cross at any look through the current one})$; it should equal the current cumulative spending target within numerical tolerance.