Validity Under Flexible Look Timing

Why Flexible Look Timing Needs a Validity Argument

An interim analysis may occur earlier or later than planned because recruitment, event accumulation, or data processing proceeds at a different rate. A spending-function design accommodates such changes by assigning cumulative type I error spending to the information time at which each analysis actually occurs. The boundaries must then be recalculated for those actual times; the originally planned numerical cutoffs do not automatically remain appropriate.

This flexibility is conditional, not unlimited. The calculation relies on a specified null distribution for the sequence of test statistics and on a monitoring schedule that does not select analysis times in response to unblinded evidence. When those conditions hold, the schedule need not match the planned calendar dates or information fractions. When they fail, evaluating the same spending function at the observed times does not by itself establish type I error control.

The introductory material on alpha spending functions defines cumulative spending and its increments. Here the focus is how that budget remains valid when the actual information times differ from the planned ones.

Setup and Assumptions

Definition: Let \(I_{\max}\) be the planned maximum information and \(I_k\) the information available at interim look \(k\). The information time is \(t_k=I_k/I_{\max}\). A cumulative alpha-spending function \(A(t)\) specifies the type I error budget spent by information time \(t\), with \(A(0)=0\), \(A(1)=\alpha\), and nondecreasing spending. At look \(k\), the cumulative target is \(A(t_k)\); the incremental target is \(A(t_k)-A(t_{k-1})\), with \(t_0=0\).

For a one-sided upper efficacy test, let \(Z_k\) be the standardized test statistic at look \(k\), and let \(b_k\) be its efficacy boundary. The design rejects \(H_0\) if any observed statistic reaches or exceeds its boundary. A common null model assumes \(E_0(Z_k)=0\), \(\operatorname{Var}_0(Z_k)=1\), and, for \(j<k\),

\operatorname{Corr}_0(Z_j,Z_k)=\sqrt{\frac{I_j}{I_k}}.

This joint distribution follows in standard group sequential models when the score or estimator has independent increments across disjoint information intervals and the information scale is correctly specified. It is not enough for each \(Z_k\) to be marginally standard normal: the dependence between looks determines the probability of crossing a later boundary after earlier boundaries were not crossed.

The monitoring assumptions are equally important. The information times must be fixed in advance or determined by factors external to the unblinded treatment-effect evidence, such as operational delays or blinded event-count updates. The test statistics and information values must follow the null model used to calibrate the boundaries. The analysis plan must also specify the spending function, the test direction, and how boundaries will be updated before the unblinded statistic at a new look is examined.

Under these assumptions, actual information times can differ from planned times. They must still be ordered, and the cumulative spending targets cannot decrease. A schedule with missed or delayed analyses can be accommodated by using the actual information times and recalibrating against all analyses that were actually performed.

Why Recalibration Preserves the Budget

At look \(k\), the boundary is chosen so that the null probability of having crossed at least one boundary by that look equals the cumulative spending target:

P_0\left(\bigcup_{j=1}^{k}\{Z_j\ge b_j\}\right)=A(t_k).

Because the previous looks have already used probability \(A(t_{k-1})\), the unconditional probability of a first crossing at look \(k\) must equal the increment \(A(t_k)-A(t_{k-1})\). Conditional on not having crossed earlier, this probability is \(\{A(t_k)-A(t_{k-1})\}/\{1-A(t_{k-1})\}\). More precisely, the boundary solves

P_0\left(Z_j<b_j\ \text{for }j<k,\ Z_k\ge b_k\right) =A(t_k)-A(t_{k-1}).

The left side depends on the joint null distribution and the boundaries already set. Thus an incremental alpha allocation is a target probability, not generally a standalone one-look significance level that can be converted directly into \(b_k\) with a normal quantile.

Determine the actual information time.
Calculate \(t_k\) from the information available before examining the unblinded result.

Update the cumulative target.
Evaluate \(A(t_k)\), and subtract the previous cumulative target to find the increment.

Recompute the boundary.
Use the null joint distribution of the statistics at the actual and previous looks to solve the crossing-probability equation.

Apply the prespecified rule.
Compare the current statistic with the recalculated boundary, retaining the earlier stopping rule and documenting the actual information time.

The justification is by construction. At each look the null probability of a first crossing is set to the increase in cumulative spending. The events “first crossing at look \(k\)” are mutually exclusive. Therefore, after \(K\) looks, the probability of any crossing is the sum of those increments:

P_0(\text{any crossing by }K) =\sum_{k=1}^{K}\{A(t_k)-A(t_{k-1})\} =A(t_K).

If the final analysis occurs at \(t_K=1\), the total is \(A(1)=\alpha\). If monitoring stops earlier, the cumulative type I error used is \(A(t_K)\), which is at most \(\alpha\). This argument depends on solving the boundary equation under the correct joint null distribution for the actual schedule.

Worked Example: Correcting a Two-Look Boundary at the Actual Time

Suppose a hypothetical one-sided efficacy design has \(\alpha=0.05\), and the first actual look occurs at \(t_1=0.5\). The spending function assigns \(A(0.5)=0.0125\). The second actual look is at \(t_2=1\), where \(A(1)=0.05\). Find the two boundaries under the canonical null model.

At the first look, \(Z_1\sim N(0,1)\), so the first boundary satisfies \(P_0(Z_1\ge b_1)=0.0125\). Hence

b_1=\Phi^{-1}(1-0.0125)=\Phi^{-1}(0.9875)\approx 2.241.

The information ratio is \(I_1/I_2=0.5\), so \(\operatorname{Corr}_0(Z_1,Z_2)=\sqrt{0.5}\approx0.7071\). The second look must add \(0.05-0.0125=0.0375\) to the cumulative crossing probability. Its boundary therefore solves

P_0(Z_1<2.241,\ Z_2\ge b_2)=0.0375, \qquad \operatorname{Corr}_0(Z_1,Z_2)=0.7071.

Numerical evaluation of this bivariate-normal probability gives \(b_2\approx1.700\). At that boundary, the probability of crossing at look 2 after not crossing at look 1 is approximately \(0.0375\); adding the first-look probability gives \(0.0125+0.0375=0.05\). The boundary is not obtained by applying a one-look quantile to the incremental spend alone, because the probability calculation excludes trajectories that already crossed at look 1.

Worked Examples of Revised Timing

Worked Example: An Earlier First Look

Consider a hypothetical one-sided design with \(\alpha=0.05\) and a linear spending rule \(A(t)=0.05t\). The planned first look was at \(t=0.5\), but operational delays and accrual mean the actual first look occurs at \(t=0.4\). There have been no prior analyses.

The actual cumulative spend is \(A(0.4)=0.05(0.4)=0.0200\). Because this is the first look, its boundary is the standard-normal upper quantile with tail probability \(0.0200\):

b_1=\Phi^{-1}(1-0.0200)=\Phi^{-1}(0.9800)\approx2.054.

For comparison, at the originally planned time \(t=0.5\), the target would have been \(A(0.5)=0.0250\), giving \(\Phi^{-1}(0.9750)\approx1.960\). Using the planned boundary of \(1.960\) at the actual \(t=0.4\) would spend \(0.0250\), not the \(0.0200\) allocated by this spending rule at that time. Recalibration therefore matters even when the first-look calculation is a simple normal quantile.

Worked Example: A Random Operational Schedule Independent of Outcomes

Suppose an external data-processing system determines, before treatment results are examined, whether a hypothetical trial will have its sole interim analysis at \(t=0.4\) or \(t=0.6\). The probabilities are \(0.6\) and \(0.4\), respectively. Use \(A(t)=0.05t\), and assume the null statistic at the selected look is standard normal.

If the schedule selects \(t=0.4\), the boundary is \(\Phi^{-1}(0.98)\approx2.054\), and the conditional null rejection probability is \(0.0200\). If it selects \(t=0.6\), the spend is \(A(0.6)=0.0300\), so the boundary is \(\Phi^{-1}(0.97)\approx1.881\), and the conditional null rejection probability is \(0.0300\).

Since the schedule is selected independently of the outcomes, the unconditional null rejection probability is the weighted average:

$$ 0.6(0.0200)+0.4(0.0300)=0.0240. $$

The result is below \(0.05\), as expected because neither schedule reaches the final information time. Each schedule uses its own cumulative spending target, and averaging over an external random schedule does not increase the error beyond the largest allowed cumulative spend.

Diagnostics, Alternatives, and Limitations

Before relying on flexible timing, check that the time supplied to the spending function is the intended information fraction, not simply a calendar-time fraction. Also verify that the actual schedule and prior analyses are recorded correctly, that the boundary solver uses the dependence among looks, and that any nuisance-parameter estimates used to calculate information do not invalidate the assumed null distribution. A material departure from the assumed test-statistic distribution can make the nominal crossing probability inaccurate.

The main alternative is a design with fixed planned looks and boundaries calibrated for that exact schedule. It can be simpler to implement when the schedule is reliably known, but an unplanned change in timing may make its original calibration inappropriate. A spending-function design instead updates the cumulative target at each actual information time and recalculates the boundary. It does not remove the need for a model or a prespecified analysis rule, and it does not guarantee that power or expected sample size will be unchanged when timing shifts.

A particularly serious failure occurs when the time of the next look is chosen after inspecting unblinded treatment differences—for example, when investigators decide to analyze as soon as the observed effect looks promising. The resulting analysis time is associated with the test-statistic path. The fixed-time null crossing calculation no longer justifies the error claim, and simply evaluating \(A(t)\) at that outcome-selected time need not control type I error. Such adaptive monitoring requires an analysis specifically calibrated for the selection mechanism; it cannot be treated as ordinary operational flexibility.

Likewise, repeatedly checking the test statistic between scheduled analyses is not the same as conducting a finite set of group sequential looks. A finite-look spending calculation only accounts for the looks included in its joint probability calculation. If the actual monitoring is continuous or includes extra unplanned looks, those opportunities to cross must be incorporated in a suitable design. Spending-function validity is therefore a property of the complete monitoring procedure, not just the formula for \(A(t)\).

Common Mistake

Common mistake: “The new look is earlier than planned, but the old boundary is still valid because the total alpha has not changed.” The correction is that the boundary is tied to the cumulative spend at the actual information time and to the joint distribution of the statistics at all conducted looks. Recalculate the boundary before using the statistic. If timing was selected using unblinded results, this recalculation alone is not a validity guarantee.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

For a one-sided design with \(\alpha=0.05\) and \(A(t)=0.05t\), what is the first-look spend and approximate boundary at \(t=0.2\)?
Why can the second-look boundary not generally be calculated by applying a one-sample normal quantile to the incremental alpha spend?
State one monitoring assumption that allows a revised information time to be used in the ordinary spending-function calculation.
If a study stops at \(t=0.7\), what is its cumulative type I error target under \(A(t)=0.05t\)?
Why is choosing the next analysis time after inspecting unblinded treatment differences potentially invalidating?

Solutions. (1) \(A(0.2)=0.05(0.2)=0.0100\); the first boundary is \(\Phi^{-1}(0.99)\approx2.326\). (2) The second-look increment applies only to paths that did not cross earlier, so its boundary must account for the joint distribution and previous boundaries. (3) For example, the revised time is determined by external operational factors or blinded information, not by unblinded treatment-effect evidence. (4) \(A(0.7)=0.0350\), so the cumulative target is \(0.0350\). (5) The selected time is then associated with the observed test-statistic path, so the fixed-time null crossing calculation need not describe the actual monitoring procedure.