Sequential Efficacy Boundaries In Z-Statistic Designs

Why efficacy boundaries are needed

In a group sequential trial, the treatment effect is examined at several planned times rather than only once at the end. Repeatedly looking at the accumulating results and stopping whenever an ordinary single-analysis test is significant can change the chance of a false positive. A sequential efficacy rule therefore specifies, before the results are examined, both when the analyses occur and how strong the evidence must be at each one.

This tutorial develops the basic rule for a one-sided Z-statistic design. Its purpose is to identify the planned looks, calculate the statistic at each look, and determine whether the trial crosses an efficacy boundary for the first time. It does not specify how to calibrate a particular boundary shape; the critical values are treated as design inputs.

Planned looks and one-sided Z statistics

Information measures the precision available for estimating the treatment effect; it is not necessarily the number of participants enrolled. For example, with a continuous outcome and a known variance, information can be represented as the reciprocal of the variance of the effect estimate. In other settings, the relationship between sample size, events, and information depends on the design and analysis. The look schedule should be stated in terms of the planned information targets, with the corresponding operational timing defined in the protocol.

Let \(\theta\) denote the treatment effect, with positive values defined in advance as beneficial. At look \(k\), let \(\hat{\theta}_k\) be its estimate and \(SE_k\) its standard error. The one-sided hypotheses are \(H_0:\theta\leq 0\) and \(H_a:\theta>0\). The boundary between these hypotheses is \(\theta=0\).

For a canonical information-based model, the standard error is approximately \(1/\sqrt{I_k}\). If \(U(I_k)\) is the cumulative score at information \(I_k\), then \(Z_k=U(I_k)/\sqrt{I_k}\). Under the null boundary \(\theta=0\), the score has mean zero and variance \(I_k\), so \(Z_k\) is approximately standard normal at each look. Under an alternative effect \(\theta\), the model gives an approximate mean of \(\theta\sqrt{I_k}\).

The standard group sequential approximation also assumes that score increments from non-overlapping information intervals are independent and approximately normal. The resulting Z statistics at different looks are not independent. For \(i<j\), the canonical joint model gives:

The middle equality uses the independent-increment model: the shared portion of the two cumulative scores is the score through information \(I_i\), with variance \(I_i\). Since each Z statistic has variance approximately one under the null, the covariance here is also the correlation. This dependence is why a sequence of looks cannot be treated as a sequence of unrelated single-analysis tests.

Critical values and the first-crossing rule

For each planned look, the design specifies a critical value \(c_k\). In a one-sided efficacy design, the trial crosses the efficacy boundary at look \(k\) if \(Z_k\geq c_k\). A design may use different values at different looks. The values are part of the overall testing plan; they are not selected after seeing the observed statistics.

The “first” in first crossing matters. After a boundary is crossed, later statistics are not used to decide whether the trial should have stopped earlier. The rule is based on the ordered sequence of planned looks, not just on whether any statistic in a retrospectively assembled list happens to be large.

How to apply the rule reliably

Assumptions, diagnostics, and limitations

The rule is meaningful only when its design assumptions and implementation are credible. At minimum, the planned information levels, analysis times, effect direction, statistic, and critical values must be specified coherently. The Z approximation should be adequate for the endpoint and estimator at the looks being used. The standard error must reflect the intended analysis, including relevant features such as clustering, missing outcomes, or estimated variance when they affect precision. The canonical correlation model is an approximation, not a guarantee for every design.

Useful checks include comparing actual information at each analysis with the planned target; confirming that the effect estimate, standard error, and critical value all correspond to the same look; checking that higher values of \(Z_k\) really favor the intended beneficial direction; and verifying that analyses were not added, omitted, or shifted without an approved design-specific procedure. If the trial’s data structure or analysis differs materially from the assumptions used to set the rule, the nominal boundary comparison may no longer have its intended operating characteristics.

A boundary value is a threshold for a particular look, not a standalone statement about the entire sequence. In particular, interpreting each \(c_k\) as if it were an ordinary single-analysis cutoff does not account for the planned repeated opportunities to cross. The joint error-rate implications of multiple looks require design-level calibration; the boundary values must be considered as a sequence. The next step in the course’s treatment of classical boundaries is to examine that familywise error issue.

A further limitation is that an efficacy boundary answers a narrow question: whether the statistic has crossed the pre-specified threshold in the beneficial direction. It does not by itself establish clinical importance, resolve safety concerns, or authorize an unplanned analysis. Nor does failure to cross an efficacy boundary imply that the treatment is ineffective. Those conclusions require additional evidence and the trial’s full decision framework.

Common mistake: treating looks as independent tests

A related error is to stop at the first positive statistic, regardless of its size. The efficacy rule requires \(Z_k\geq c_k\), not merely \(Z_k>0\). Another is to reverse the sign convention mid-analysis: if negative effects favor the intervention under the original definition, the statistic and one-sided alternative must be aligned before applying a positive-direction boundary.