Recalibrating After a Delayed Look

Why a Delayed Look Requires Recalibration

An interim analysis may occur later than planned because information accrues more slowly than expected. The test statistic at that look then has a different correlation with earlier statistics, and the spending function assigns a different cumulative type I error target. A boundary calibrated for the planned information fraction therefore need not control the intended error at the actual look.

The key distinction is between cumulative spending and a look’s marginal tail probability. A cumulative spending target is the probability, under \(H_0\), of having crossed an efficacy boundary at this look or any earlier look. The boundary at a delayed look must be calibrated to reach that cumulative target after accounting for the crossings already possible at earlier looks.

Definition: Let \(t_j\) be the actual information fraction at look \(j\), and let \(\alpha(t)\) be a prespecified cumulative alpha-spending function, with \(\alpha(0)=0\) and \(\alpha(1)=\alpha\). For an upper one-sided efficacy test, the boundary \(c_j\) is calibrated so that, under \(H_0\), the probability of crossing any boundary through look \(j\) is \(\alpha(t_j)\). If the look is delayed, use its actual information fraction in \(\alpha(t_j)\) and in the joint distribution of the look statistics.

The Model and the Recalibration Equation

Assume the canonical group-sequential model under \(H_0\): the standardized test statistics \(Z(t)\) have marginal distribution \(N(0,1)\), and for \(s\leq t\), their correlation is \(\sqrt{s/t}\). This covariance structure follows when the underlying score process has independent increments and information is measured on the relevant analysis scale. The information times are treated as known for calibration, the test is upper-tailed, and each earlier boundary has already been fixed and calibrated. These assumptions make the sequence jointly normal; if the analysis statistic or information process does not satisfy the canonical model, the calibration must use an appropriate model instead.

Suppose the delayed look is look \(j\), with actual information fraction \(t_j\). The cumulative target is \(\alpha(t_j)\); the target at the previous look is \(\alpha(t_{j-1})\). The required probability of a first crossing at look \(j\), after no earlier crossing, is therefore the difference between these cumulative targets. For two looks, the boundary solves the following equation:

P\{Z(t_1)<c_1,\ Z(t_2)\geq c_2\} = \alpha(t_2)-\alpha(t_1), \qquad \operatorname{Corr}\{Z(t_1),Z(t_2)\} = \sqrt{\frac{t_1}{t_2}}.

The probability on the left is not generally \(P\{Z(t_2)\geq c_2\}\). Some paths that exceed \(c_2\) have already crossed \(c_1\) and belong to the earlier spending. For more than two looks, the same principle applies: choose the new boundary so that the probability of a first crossing at that look equals the increase in cumulative spending, given the existing boundaries and the joint distribution.

Why the Boundary Equation Has This Form

For two looks, write \(Z_1=Z(t_1)\), \(Z_2=Z(t_2)\), and \(\rho=\sqrt{t_1/t_2}\). Under the canonical model, \(Z_2\mid Z_1=x\) is normal with mean \(\rho x\) and variance \(1-\rho^2\). Conditioning on the first statistic gives the probability of a first crossing at look 2:

\begin{aligned} P(Z_1<c_1,\ Z_2\geq c_2) &= \int_{-\infty}^{c_1} P(Z_2\geq c_2\mid Z_1=x)\,\phi(x)\,dx\\ &= \int_{-\infty}^{c_1} \phi(x)\, \overline{\Phi}\!\left( \frac{c_2-\rho x}{\sqrt{1-\rho^2}} \right)\,dx. \end{aligned}

Here \(\phi\) is the standard normal density and \(\overline{\Phi}(u)=1-\Phi(u)\) is its upper-tail probability. Once the actual information fraction is known, this one-dimensional integral can be evaluated numerically and the value of \(c_2\) found by solving for the required spending increment. The integral decreases as \(c_2\) increases, so a valid target increment between zero and the probability of reaching look 2 without a previous crossing identifies a boundary.

Worked Examples

Worked Example: Recalibrating a Delayed Second Look

Suppose a hypothetical one-sided trial has a planned first look at information fraction \(t_1=0.50\), and that look has already spent \(0.0100\) of type I error. The observed second look is delayed to \(t_2=0.75\). The prespecified spending function, evaluated at the actual times, gives cumulative targets \(\alpha(0.50)=0.0100\) and \(\alpha(0.75)=0.0150\). Find the second boundary, retaining the first boundary.

At the first look, with no earlier look to account for, the boundary is the standard normal upper quantile:

c_1=\Phi^{-1}(1-0.0100)=2.326.

The actual correlation is \(\rho=\sqrt{0.50/0.75}=\sqrt{2/3}\approx0.8165\), and the conditional standard deviation is \(\sqrt{1-\rho^2}=\sqrt{1/3}\approx0.5774\). The additional crossing probability required at the delayed look is \(0.0150-0.0100=0.0050\). Thus solve:

\int_{-\infty}^{2.326} \phi(x)\, \overline{\Phi}\!\left( \frac{c_2-0.8165x}{0.5774} \right)\,dx =0.0050.

Numerical evaluation gives \(c_2\approx2.38\) (about \(2.38\) to the precision warranted here). With this boundary, the probability of a first crossing at look 2 is approximately \(0.0050\), so the cumulative crossing probability through look 2 is \(0.0100+0.0050=0.0150\), as targeted. This is a hypothetical calibration calculation under the stated canonical null model.

Worked Example: A Delayed First Look

In a separate hypothetical design, the first analysis is delayed from its planned information fraction to an actual fraction of \(0.60\). There are no earlier analyses, and the spending function gives a cumulative target of \(0.0120\) at \(t=0.60\). Find the efficacy boundary.

Because this is the first look, its cumulative crossing probability is just the marginal upper-tail probability. Hence:

P\{Z(0.60)\geq c_1\}=0.0120, \qquad c_1=\Phi^{-1}(1-0.0120)=\Phi^{-1}(0.9880)\approx2.257.

The boundary is about \(2.257\) on the standard normal \(Z\)-scale. No earlier crossing probability needs to be subtracted. The actual information fraction matters because it determines the spending target; here, the supplied value \(0.0120\) is the function evaluated at \(0.60\), not the planned fraction.

Worked Example: Checking a Candidate Boundary

Return to the hypothetical two-look design in the first example. A candidate boundary of \(c_2=2.40\) might appear plausible if one looks only at the marginal tail \(P\{Z(t_2)\geq2.40\}\approx0.00820\). Check whether it spends the required additional \(0.0050\).

Using the actual correlation and retaining \(c_1=2.326\), evaluate the first-crossing integral:

\int_{-\infty}^{2.326} \phi(x)\, \overline{\Phi}\!\left( \frac{2.40-0.8165x}{0.5774} \right)\,dx \approx0.0046.

The cumulative crossing probability is therefore approximately \(0.0100+0.0046=0.0146\), below the target \(0.0150\). The candidate boundary is too high for the required spending. Reducing it to about \(2.38\) raises the first-crossing probability to approximately \(0.0050\) and attains the target. The marginal tail \(0.00820\) is not the spending increment because it includes paths that crossed the earlier boundary.

Practical Checks and Common Failure Modes

A delayed look is not handled by simply carrying over the planned boundary or by assigning the entire cumulative target to the new look. Check the calibration in the following order:

Record the actual information fraction.
Use the information available at the observed analysis time. Confirm that it is on the same information scale used to define the spending function.

Evaluate cumulative spending.
Compute \(\alpha(t_j)\) at the actual fraction and compare it with the target already assigned at the preceding look. For a nondecreasing spending function, this increment should not be negative.

Update the dependence structure.
Under the canonical model, use correlations based on actual information times, such as \(\sqrt{t_{j-1}/t_j}\), rather than correlations implied by the planned schedule.

Solve and verify.
Find the boundary that gives the required first-crossing probability, then recompute cumulative crossing probability through the look. It should agree with \(\alpha(t_j)\), up to numerical tolerance.

A frequent error is to set \(P\{Z(t_j)\geq c_j\}=\alpha(t_j)-\alpha(t_{j-1})\). That calculation ignores the dependence on earlier statistics and counts paths that already crossed an earlier boundary as if they were new crossings. The correction is to calculate the probability of crossing at look \(j\) and not crossing any earlier boundary, using the joint null distribution.

Another error is to interpolate a boundary between planned looks and apply it at the delayed look. Interpolation can be useful for describing a planned schedule, but it does not generally solve the recalibration equation at a new actual time. Likewise, using the planned information time in the correlation while evaluating spending at the actual time mixes two different designs.

This recalibration assumes the earlier look and its boundary remain as specified, the delay changes the timing rather than the test statistic definition, and the stated information fraction adequately represents the null covariance. If the information estimate is unreliable, the canonical model is inappropriate, or the analysis procedure itself changes, the calculated boundary may not deliver the intended error control. A calibration check is also important when numerical integration or root solving is used: verify the achieved cumulative crossing probability, not just that the solver returned a number.

How This Recalibration Differs From Alternatives

Approach	What it does	Why it differs
Recalibrate at the actual information time	Evaluates spending at the observed fraction and solves using the actual joint distribution.	Targets the intended cumulative crossing probability under the stated model.
Reuse the planned boundary	Applies the boundary calibrated for the planned schedule.	The spending target and correlation may both differ at the delayed time.
Use a marginal normal quantile	Sets the tail probability at the current look equal to a chosen alpha amount.	Does not account for paths that already crossed earlier boundaries.
Interpolate planned boundaries	Estimates a boundary between scheduled information times.	Interpolation alone does not ensure the cumulative error target at the actual look.

The spending-function approach is useful because it specifies cumulative error as a function of information time, rather than requiring all analyses to occur exactly on a fixed schedule. It does not make timing irrelevant: the actual fraction must be used when evaluating the target and the joint distribution. Nor does it itself authorize additional analyses or determine how to handle a changed analysis plan; those are distinct design questions.

Key takeaway: At a delayed look, evaluate the cumulative spending function at the actual information fraction, retain the earlier calibrated boundaries, and solve for the new boundary using the probability of a first crossing under the actual joint null distribution.

Check Your Understanding

Use the definitions and assumptions in this tutorial. Solutions follow each question.

A first look has cumulative spending \(0.008\). At a delayed second look the target is \(0.013\). What is the required probability of a first crossing at the second look? Solution: \(0.013-0.008=0.005\).
Why is \(P\{Z(t_2)\geq c_2\}\) generally not the second look’s added spending? Solution: It includes paths that crossed the first boundary already. Added spending is \(P\{Z(t_1)<c_1,\ Z(t_2)\geq c_2\}\) in a two-look design.
Under the canonical model, calculate the correlation for information fractions \(t_1=0.40\) and \(t_2=0.64\). Solution: \(\sqrt{0.40/0.64}=\sqrt{0.625}\approx0.7906\).
A calibration uses the actual delayed fraction in the spending function but the planned fraction in the correlation. What is wrong? Solution: The spending target and dependence structure refer to inconsistent information times. Both must use the actual schedule under the canonical model.
If the new boundary gives the correct first-crossing increment, what further check should be made? Solution: Add that increment to the earlier cumulative crossing probability and confirm the result matches \(\alpha(t_j)\), within numerical tolerance.