Boundary Calibration at Irregular Look Times

Why Irregular Look Times Change the Calibration

An efficacy boundary is calibrated against the probability of crossing it under $H_0$, not simply against the alpha assigned to an isolated test. At an interim analysis, some type I error may already have been spent by earlier opportunities to stop. The boundary at the current look must therefore be chosen so that the cumulative probability of crossing by that look matches the spending function evaluated at the information actually available.

When looks are equally spaced, a planned schedule can make the information fractions look like a regular sequence. In practice, however, information may accumulate unevenly. A look at information fraction $0.60$ is not a look at $0.50$, even if it occupies the same numbered position in a schedule that was initially expected to have equally spaced analyses. The spending target must use $0.60$. The boundary also depends on how the current statistic is associated with the statistics at earlier looks.

This tutorial considers a one-sided upper efficacy boundary. The same central idea applies more generally, but two-sided designs require their specified allocation of type I error to the two tails.

Model and Calibration Condition

Definition: Let $t_1<t_2<\cdots<t_K=1$ be the information fractions at the planned looks, where $t_k$ is the information available at look $k$ divided by the information planned for the final analysis. Under the canonical group-sequential null model, the standardized test statistics satisfy $Z_k\sim N(0,1)$, with $$ \operatorname{Corr}(Z_i,Z_j)=\sqrt{\frac{t_i}{t_j}}, \qquad i<j. $$ For an upper efficacy boundary $b_k$, a crossing occurs at look $k$ if $Z_k\ge b_k$, provided no earlier boundary has been crossed. If $A(t)$ is the cumulative one-sided alpha-spending function, calibration targets $$ P_{H_0}\left(\bigcup_{j=1}^{k}\{Z_j\ge b_j\}\right)=A(t_k). $$

The canonical model is appropriate when the test statistics have the stated joint null distribution, the information fractions are correctly specified, and the analysis plan uses the same spending function and sidedness used in calibration. In common large-sample settings, the model follows from approximately normal score or likelihood-based statistics with information increasing over time. The correlation formula reflects shared accumulated data: statistics at later looks contain information from earlier looks as well as new information.

A spending function is nondecreasing, has $A(0)=0$, and reaches the design’s total one-sided type I error $\alpha$ at $t=1$. The spending increment assigned to look $k$ is $A(t_k)-A(t_{k-1})$, taking $A(t_0)=0$. This increment is a target for new crossing probability, not generally the marginal tail probability $P(Z_k\ge b_k)$. Some paths reaching the upper tail at look $k$ have already crossed a previous boundary and must not be counted a second time.

From Spending Targets to Boundaries

At the first look, there is no earlier boundary, so calibration is direct: choose $b_1$ such that $P(Z_1\ge b_1)=A(t_1)$. At a later look, the calibration equation must exclude paths that crossed earlier. For a two-look design, it is

P(Z_1<b_1,\ Z_2\ge b_2)=A(t_2)-A(t_1).

Under the canonical model, put $\rho=\sqrt{t_1/t_2}$. Conditional on $Z_2=z$, $Z_1$ has a normal distribution with mean $\rho z$ and variance $1-\rho^2$. Therefore the two-look probability can be written

P(Z_1<b_1,\ Z_2\ge b_2) =\int_{b_2}^{\infty}\phi(z)\, \Phi\left(\frac{b_1-\rho z}{\sqrt{1-\rho^2}}\right)\,dz,

where $\phi$ and $\Phi$ are the standard normal density and cumulative distribution function. This expression decreases as $b_2$ increases. A numerical root search can therefore find the boundary whose integral equals the spending increment. The calculation uses both the actual information fractions and the earlier boundary.

Worked Example: Calibrating a Boundary at an Irregular Second Look

Suppose a hypothetical one-sided design has total type I error $\alpha=0.025$, with a first look at $t_1=0.25$ and a second look at $t_2=0.60$. Use the spending function $A(t)=1-\Phi(1.959964/\sqrt{t})$ for $0<t\le1$. Calibrate the first two boundaries under the canonical null model.

At the first look, $$ A(0.25)=1-\Phi(1.959964/\sqrt{0.25}) =1-\Phi(3.919928)\approx0.000044. $$ Thus $b_1=3.919928$, because the first-look upper-tail probability is the entire cumulative crossing probability at that look.

At the actual second-look fraction, $$ A(0.60)=1-\Phi(1.959964/\sqrt{0.60}) \approx0.00569821. $$ The target probability of a new crossing at look 2 is consequently $$ A(0.60)-A(0.25)\approx0.00569821-0.00004428 =0.00565393. $$ The correlation is $\rho=\sqrt{0.25/0.60}\approx0.6455$. Substitute the values into the two-look calibration equation: $$ \int_{b_2}^{\infty}\phi(z)\, \Phi\left(\frac{3.919928-0.6455z}{\sqrt{1-0.6455^2}}\right)\,dz =0.00565393. $$ Numerical integration and root-finding give $b_2\approx2.5314$, or $2.531$ to three decimal places. This boundary targets cumulative null crossing probability $A(0.60)$, not marginal probability $A(0.60)$ at look 2 alone. Values are rounded for display; the root should be obtained using unrounded spending and correlation values.

The calculation assumes the information fractions are $0.25$ and $0.60$, and that the joint null distribution follows the canonical correlation model. Given those assumptions, the target is an upper-boundary design with cumulative type I error about $0.005698$ through the second look.

A Short Justification of the Calibration Equation

For two looks, a crossing by look 2 is the union of the event $Z_1\ge b_1$ and the event $Z_1<b_1,\ Z_2\ge b_2$. These events are disjoint. Hence

P(\text{cross by look 2}) =P(Z_1\ge b_1)+P(Z_1<b_1,\ Z_2\ge b_2).

The first term is calibrated to $A(t_1)$. Requiring the left side to equal $A(t_2)$ gives $P(Z_1<b_1,\ Z_2\ge b_2)=A(t_2)-A(t_1)$, as used above. This also explains why the spending increment is not simply a marginal normal tail: the second-look event is restricted to paths that did not cross at look 1.

Worked Example: How the Actual Fraction Determines Spending

Continue with the spending function $A(t)=1-\Phi(1.959964/\sqrt{t})$. Compare an interim look at $t=0.60$ with a look at $t=0.50$, assuming the same earlier look at $t=0.25$.

At $t=0.60$, the spending is approximately $0.00569821$; at $t=0.25$, it is approximately $0.00004428$. The target increment from $0.25$ to $0.60$ is therefore $0.00565393$.

At $t=0.50$, $$ A(0.50)=1-\Phi(1.959964/\sqrt{0.50})\approx0.00278729, $$ so the increment from $0.25$ to $0.50$ is approximately $0.00278729-0.00004428=0.00274301$. Using $0.50$ in place of the actual $0.60$ would set a substantially smaller target for this look. The final boundary must be calibrated to the target for the actual fraction, with the joint distribution and earlier boundaries taken into account.

Worked Example: Splitting an Irregular Schedule into Spending Increments

Suppose the same hypothetical design has looks at $t=0.25,\ 0.50,\ 0.60,$ and $1.00$. Calculate the cumulative spending targets and the increments at these times.

The cumulative targets are approximately $A(0.25)=0.00004428$, $A(0.50)=0.00278729$, $A(0.60)=0.00569821$, and $A(1)=0.025$. Subtract consecutive cumulative targets to obtain increments:

At $t=0.25$: $0.00004428$.
At $t=0.50$: $0.00278729-0.00004428=0.00274301$.
At $t=0.60$: $0.00569821-0.00278729=0.00291092$.
At $t=1.00$: $0.025-0.00569821=0.01930179$.

These are targets for the successive new-crossing probabilities, not direct marginal-tail specifications for $Z_k$. Each boundary after the first must still account for the joint distribution of the current statistic and all earlier statistics. The increments sum to $0.025$, as required by the final cumulative spending target.

Diagnostics, Alternatives, and Limitations

Several checks help detect a calibration that is going wrong. First, verify that the supplied information fractions are increasing and lie between 0 and 1, with the final planned analysis at 1. Second, check that $A(t_k)$ is nondecreasing and that the increments are nonnegative and sum to the intended total alpha. Third, verify that the boundary at each look produces the intended cumulative crossing probability under the assumed joint null distribution. If a numerical solver returns a boundary that does not change monotonically with its target, check the probability equation, integration limits, and boundary direction.

A common error is to treat each increment $A(t_k)-A(t_{k-1})$ as the marginal tail probability $P(Z_k\ge b_k)$. That ignores paths that already crossed an earlier boundary. The correction is to set the probability of crossing at look $k$ after surviving all earlier looks equal to the increment. Another error is to use nominal equally spaced fractions when the actual information fractions are irregular. This evaluates the spending curve at the wrong times and can misallocate the cumulative error budget.

Fixed critical values are an alternative, but they do not generally implement a chosen spending function at irregular looks. A boundary design based only on a final test also does not control the probability of crossing at any of several planned looks at the specified cumulative levels. Spending-function calibration is useful precisely because it expresses error control at the actual information times. Its limitation is that the resulting boundaries depend on the assumed information fractions and joint distribution. If those assumptions are inaccurate, the calibrated crossing probabilities may not match the intended targets.

When information is estimated from event counts or other trial data, distinguish a genuinely updated information fraction from a look that has merely been delayed. Do not silently substitute a planned fraction for an observed one. If actual information times or the statistical model depart from the calibration assumptions, the design’s operating characteristics require reassessment. The calculations here specify calibration under the canonical model; they do not establish robustness to arbitrary changes in information or analysis procedures.

Key takeaway: Evaluate the spending function at each actual information fraction. Then calibrate each boundary so that cumulative null crossing probability equals cumulative spending, accounting for all previous boundaries and their dependence with the current statistic.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A design has $A(t_1)=0.003$ and $A(t_2)=0.012$. What is the target new-crossing probability at look 2?
Why is the look-2 target in the previous question generally not $P(Z_2\ge b_2)$?
Under the canonical model, what is $\operatorname{Corr}(Z_1,Z_2)$ when $t_1=0.36$ and $t_2=0.81$?
For the spending function in the worked examples, the increment from $0.25$ to $0.50$ is approximately $0.00274301$. What is wrong with using this increment for a look that actually occurs at $t=0.60$?

Solutions. (1) The target is $A(t_2)-A(t_1)=0.012-0.003=0.009$. (2) The marginal tail includes paths that crossed at an earlier look; the target is the probability of crossing at look 2 among paths that survived earlier boundaries. (3) The correlation is $\sqrt{0.36/0.81}=\sqrt{4/9}=2/3\approx0.667$. (4) The increment is evaluated at the wrong information fraction. At $t=0.60$, the increment from $0.25$ is approximately $0.00565393$, not $0.00274301$; the boundary must be calibrated using the actual fraction and the earlier boundary.