Sensitivity to Information-Time Misestimation

Why Information-Time Accuracy Matters

A group-sequential design sets boundaries at a sequence of analyses. The information fraction at a look determines how much cumulative type I error an alpha-spending function allocates by that time. If the fraction is misestimated, the spending target used in planning can differ from the target associated with the realized information time. For multiple looks, the error can also change the dependence among the interim test statistics, so recalibration may change more than the spending target.

The distinction is important: a discrepancy between two spending targets is not, by itself, the realized type I error of the design. Realized error depends on the boundaries actually used and the joint null distribution of the test statistics at the actual information times.

Definition: Let \(I_k\) be the information at look \(k\), and let \(I_{\max}\) be the planned maximum information. The information fraction is \(t_k=I_k/I_{\max}\), with \(0<t_1<\cdots<t_K=1\). An information-time misestimation occurs when the fraction used for planning or calibration, \(\tilde t_k\), differs from the realized fraction \(t_k\). An alpha-spending function \(A(t)\) specifies the planned cumulative one-sided type I error expenditure by information fraction \(t\), with \(A(0)=0\) and \(A(1)=\alpha\).

For the sensitivity calculations below, assume the canonical group-sequential model under \(H_0\): each \(Z_k\) has a standard normal marginal distribution and, for \(i<j\), \(\operatorname{Corr}(Z_i,Z_j)=\sqrt{t_i/t_j}\). This model presumes that the test statistics and information times are correctly represented by the canonical joint normal approximation. We consider an upper efficacy boundary, so crossing means \(Z_k\ge b_k\). In a one-look calculation, the boundary for a one-sided tail target \(a\) is \(\Phi^{-1}(1-a)\).

How an Error Changes the Spending Target

Suppose a design uses \(\tilde t\) when it evaluates the spending function, but the relevant information fraction is \(t\). The target discrepancy is \(A(\tilde t)-A(t)\). For a small error \(\delta=\tilde t-t\), a first-order Taylor approximation gives \(A(\tilde t)-A(t)\approx A'(t)\delta\), provided \(A\) is differentiable near \(t\). The sign depends on the slope of the chosen spending function: for an increasing function, using a larger fraction gives a larger target.

Worked Example: Change in a Spending Target

Consider the hypothetical one-sided spending function \(A(t)=0.025t^{1.5}\). Suppose the actual fraction is \(t=0.50\), but the calculation uses \(\tilde t=0.55\). The target at the assumed fraction is

A(0.55)=0.025(0.55)^{1.5}\approx0.0101973.

At the actual fraction, the target is

A(0.50)=0.025(0.50)^{1.5}\approx0.00883883.

The assumed-fraction target exceeds the actual-fraction target by \(0.0101973-0.00883883=0.00135844\), or about 0.136 percentage points. Relative to the actual-fraction target, the assumed-fraction target is higher by \(0.00135844/0.00883883\approx0.1537\), about 15.4%. Equivalently, relative to the assumed-fraction target, the actual target is about 13.3% lower. This comparison concerns spending targets, not the realized crossing probability of a multi-look design.

Boundary Sensitivity at a Single Look

At a single look, recalibrating the boundary to a target \(A(t)\) gives \(b(t)=\Phi^{-1}\{1-A(t)\}\). Differentiating both sides of \(\Phi(b(t))=1-A(t)\) yields

\phi(b(t))b'(t)=-A'(t), \qquad b'(t)=-\frac{A'(t)}{\phi(b(t))}.

Because the standard normal density \(\phi(b)\) is small for a high boundary, even a modest change in the tail target can produce a noticeable boundary change. This derivative is a local approximation, not a substitute for recalculating the quantile when the information-time error is large.

Worked Example: Recalibrating a One-Look Boundary

Use the same hypothetical spending function \(A(t)=0.025t^{1.5}\). If the assumed fraction is \(0.50\), the one-look boundary is

b(0.50)=\Phi^{-1}(1-0.00883883)\approx2.372.

If the fraction used for recalibration is \(0.55\), then

b(0.55)=\Phi^{-1}(1-0.0101973)\approx2.319.

The direct recalculation gives a change of about \(2.319-2.372=-0.053\). For a local check, \(A'(t)=0.025(1.5)\sqrt{t}\), so \(A'(0.50)\approx0.02652\). Using the rounded boundary density near \(2.375\), \(\phi(2.375)\approx0.0238\), gives the approximate slope \(b'(0.50)\approx-0.02652/0.0238\approx-1.115\). For a change of \(0.05\), the first-order approximation is \((-1.115)(0.05)\approx-0.0558\). This is close to, but not identical to, the directly recalculated change of about \(-0.053\); the derivative is local and the inputs are rounded.

At a single look the null crossing probability at a fixed boundary is its normal upper-tail probability, regardless of the information fraction. Thus, if the boundary is left unchanged, changing \(t\) does not change that marginal probability. The boundary changes above arise because the target used for recalibration changes.

Multi-Look Recalibration and Its Justification

With multiple looks, the cumulative null crossing probability is \(P_0(\text{at least one }Z_k\ge b_k)\). It is not generally equal to the last-look marginal tail probability, because an earlier crossing also contributes. Further, the canonical correlation between looks depends on the information fractions. A timing error can therefore change both the targets to be matched and the joint probabilities used to match them.

For two looks with \(t_1<t_2\), write \(\rho=\sqrt{t_1/t_2}\). Under the canonical model, \((Z_1,Z_2)\) is bivariate standard normal with correlation \(\rho\). If \(b_1\) is fixed, a second boundary \(b_2\) that attains cumulative target \(A(t_2)\) must satisfy

P_0(Z_1\ge b_1\ \text{or}\ Z_2\ge b_2) =1-\Phi_2(b_1,b_2;\rho) =A(t_2),

where \(\Phi_2(b_1,b_2;\rho)=P_0(Z_1<b_1,Z_2<b_2)\). This equation, rather than a one-look quantile applied to the cumulative target, defines the second boundary. In a design that matches cumulative spending at each look, the first boundary is calibrated to the first-look target; at each subsequent look, the new boundary is chosen so the joint cumulative crossing probability matches the target at that look.

The sensitivity of a later boundary can be described implicitly. Let \(G(b_2,t_1)\) denote the two-look crossing probability with \(b_1\) and \(t_2\) held fixed, where the dependence on \(t_1\) enters through \(\rho=\sqrt{t_1/t_2}\). If recalibration requires \(G(b_2,t_1)=A(t_2)\), differentiating gives

\frac{\partial G}{\partial b_2}\frac{db_2}{dt_1} +\frac{\partial G}{\partial t_1}=0, \qquad \frac{db_2}{dt_1} =-\frac{\partial G/\partial t_1}{\partial G/\partial b_2}.

This states precisely why a multi-look boundary sensitivity cannot be inferred from the spending-function slope alone: it also depends on how the joint crossing probability changes with the correlation. The denominator is nonzero when changing \(b_2\) changes the crossing probability locally, as it does for a finite boundary in a nondegenerate normal model.

Worked Example: A Timing Error Changes the Two-Look Dependence

Suppose a hypothetical two-look design has \(t_2=1\), and its first look is assumed to occur at \(t_1=0.50\). The canonical correlation is \(\rho=\sqrt{0.50/1}\approx0.7071\). If the realized first-look fraction is instead \(0.55\), then \(\rho=\sqrt{0.55/1}\approx0.7416\). The correlation has increased by about \(0.0345\).

For fixed boundaries, the appropriate cumulative null error at the realized timing is \(1-\Phi_2(b_1,b_2;0.7416)\), not the value computed using \(0.7071\). If the design is recalibrated, \(b_2\) must solve \(1-\Phi_2(b_1,b_2;0.7416)=A(1)\), with the first boundary treated according to the design’s recalibration rule. The numerical value of \(b_2\) depends on \(b_1\) and the specified spending targets; there is no universal correction based only on the 0.05 information-fraction error. This example shows the calculation required and why using the altered correlation is essential.

Practical Sensitivity Analysis and Failure Modes

A useful analysis separates three quantities: the change in the spending target, the change in recalibrated boundary values, and the cumulative type I error under the boundaries actually implemented. Report which of these is being compared. A target can change while a fixed single-look boundary’s marginal null crossing probability does not; at multiple looks, a changed correlation can alter cumulative crossing probability even if the boundaries are held fixed.

For an actual design, evaluate the spending function at both nominal and plausible realized fractions. Then recalibrate the boundary sequence under each timing scenario using the same crossing-probability definition and the corresponding joint null distribution. Finally, evaluate the boundaries used in practice under the realized timing. This last calculation answers whether the implemented design still controls the intended cumulative type I error; matching a nominal target during planning alone does not establish that.

Local approximation failure: The derivative approximation assumes a small change and a smooth spending function. For a larger timing error, recompute \(A(t)\) and the quantile or boundary sequence directly.
Confusing target and error rate: A spending target is a calibration goal. The realized cumulative error is the probability of crossing any implemented boundary under the null.
Ignoring dependence: Reusing a multi-look boundary sequence after changing the information times while keeping the old correlations can misstate cumulative error.
Changing several inputs at once: If both the information fractions and the spending-function specification change, their effects are confounded. Vary one at a time for diagnosis, then evaluate the combined scenario.

A fixed nominal schedule is operationally simple, while recalibration to updated information times can better align boundaries with the intended spending targets. Recalibration requires reliable information estimates and a valid joint model; it is not automatically safer if the information estimate itself is biased or the canonical approximation is unsuitable. Direct numerical evaluation of crossing probabilities is preferable to interpreting a local derivative as a guaranteed error bound.

Key takeaway: Information-time error affects the spending target through \(A(t)\), and it can affect multi-look calibration again through the joint distribution of the test statistics. Quantify both effects, and distinguish them from the realized type I error of the boundaries actually used.

Check Your Understanding

Answer each question, then compare with the solutions below.

For \(A(t)=0.025t^{1.5}\), does using a larger information fraction increase or decrease the spending target?
Why does a one-look boundary recalibrated to a larger target generally move downward?
In a two-look design with \(t_2\) fixed, what part of the canonical joint distribution changes when \(t_1\) changes?
Is the difference between \(A(\tilde t)\) and \(A(t)\) necessarily the change in realized cumulative type I error?

Solutions. (1) It increases, since \(A'(t)=0.0375\sqrt{t}>0\) for \(t>0\). (2) A larger upper-tail probability requires a smaller normal quantile: \(b=\Phi^{-1}(1-a)\) decreases as \(a\) increases. (3) The correlation \(\sqrt{t_1/t_2}\) changes, so the joint crossing probability changes even if the boundaries do not. (4) No. That difference compares spending targets; realized cumulative error is calculated from the implemented boundaries and the joint null distribution at the actual information times.