Why Two-Sided Inference Must Rank Both Signs
When a group sequential trial can stop early, the observed result includes both a statistic and the look at which the trial stopped. A two-sided p-value must account for both parts. In a symmetric design, evidence in either direction can trigger an efficacy stop, so a large positive statistic and a large negative statistic at the same look are equally extreme. The p-value must also account for outcomes that would have stopped at an earlier look.
An ordinary two-sided p-value computed from the observed statistic alone does not generally make this adjustment. For example, if the trial stopped at its final look, the ordinary calculation treats the statistic as though that look had been selected in advance. A stagewise p-value instead uses the null distribution of the stopped outcome, including the chance of stopping earlier.
This ordering is a particular way to measure evidence, not the only possible one. Its interpretation depends on the declared design and ordering. Here, the symmetric boundaries and the absolute-value score make the two signs equivalent at each look.
Model and Calculation Rule
Assume a planned design with \(K\) looks at information levels \(I_1<I_2<\cdots<I_K\). Under the null hypothesis \(H_0\), let \(Z_k\) be the standardized test statistic at look \(k\). We use the canonical group sequential null model: the vector of statistics is jointly normal, each \(Z_k\) has mean zero and variance one, and for \(i<j\),
The stopping rule considered here is an efficacy-only rule with symmetric boundaries. At each nonfinal look \(k\), the trial stops if \(|Z_k|\ge b_k\), where \(b_k>0\). If it has not stopped before look \(K\), it stops at look \(K\) regardless of the statistic. Write \(J\) for the stopping look. Thus \(J=j\) means that all earlier looks continued and the trial stopped at look \(j\).
The model presumes that the look schedule and boundaries were specified for the design, and that the normal approximation and information-based correlation structure are appropriate. The calculation must use the actual stopping rule: omitting a planned look or changing a boundary changes the null distribution of the stopped outcome.
The first term counts every outcome stopped at an earlier look, in either direction. The second counts outcomes that reach the observed look and have an absolute statistic at least as large as the observed one. Outcomes stopping later are not counted as more extreme under this ordering. At the final look, the second term includes the final-look two-sided tail among trials that reached that look.
For \(j<K\), an observed stopped outcome satisfies \(|z|\ge b_j\), and the event in the second term requires continuation through all earlier looks plus \(|Z_j|\ge r\). At the final look, continuation through earlier looks is required, but the terminal statistic need not cross a boundary for the trial to stop. These distinctions are important when translating the rule into an integration region.
Why the Probability Is a Valid P-Value
A p-value is valid under \(H_0\) if it is superuniform: for every \(a\) between zero and one, \(P_{H_0}(p_{\mathrm{st}}\le a)\le a\). The stagewise construction has this property because it assigns each stopped outcome the null probability in its upper tail under a fixed ordering.
To see the argument, consider all stopped outcomes ordered from most to least extreme. For an outcome \(x\), let \(q(x)\) be the null probability of outcomes at least as extreme as \(x\). The set of outcomes for which \(q(x)\le a\) is an initial segment of this ordering: if an outcome is in that set, every outcome ranked more extreme is also in it. The null probability of that segment cannot exceed \(a\), because \(q(x)\) is the probability of the full upper tail at its least extreme boundary. In this continuous-statistic setting, ties at a fixed look have probability zero; if there were probability mass at tied values, counting the whole tie would remain conservative. Hence \(P_{H_0}(q(X)\le a)\le a\).
This justification is about calibration under the specified null model and stopping rule. It does not say that the p-value is the probability that \(H_0\) is true, or that the observed effect has a particular probability of being real.
Worked Example: A First-Look Two-Sided Result
Suppose a hypothetical two-look trial uses a symmetric first-look boundary \(b_1=2.4\). It stops at the first look if \(|Z_1|\ge2.4\). The trial stops at the final look if it continues. Under \(H_0\), \(Z_1\sim N(0,1)\). Imagine the first-look statistic is \(Z_1=2.60\), so the trial stops at look 1.
Because this is the first look, there are no earlier stopping outcomes to add. The p-value is the probability under \(H_0\) of either sign being at least as extreme as 2.60:
The observed value exceeds the boundary, since \(2.60\ge2.4\), so stopping at look 1 is consistent with the rule. The two-sided stagewise p-value is about 0.00932. It measures the null probability of a first-look statistic at least this far from zero in either direction; it does not condition on a particular sign.
Worked Symmetric-Boundary Example Across Two Looks
The distinction between an ordinary tail probability and a stagewise p-value is clearest when the trial reaches its final look. Suppose a hypothetical trial has two looks with \(I_1/I_2=0.25\), so the null correlation is \(\rho=\sqrt{0.25}=0.5\). It uses symmetric efficacy boundaries \(b_1=2.4\) and \(b_2=2.0\), and stops at look 2 if it has not stopped at look 1. The canonical null model gives \(Z_1,Z_2\) standard normal with correlation 0.5.
Worked Example: A Negative Statistic at the Final Look
Suppose the trial continues at look 1 and stops at look 2 with \(Z_2=-2.20\). This crosses the final symmetric boundary because \(|-2.20|=2.20\ge2.0\). The observed absolute statistic is \(r=2.20\). The stagewise p-value counts every first-look stop, positive or negative, and then the two-sided look-2 tail among trials that continued through look 1:
The first probability is
For the second probability, condition on \(Z_1=x\). Under the stated bivariate normal model, \(Z_2\mid Z_1=x\sim N(0.5x,0.75)\), because the conditional variance is \(1-\rho^2=0.75\). Therefore,
The integral includes both tails of \(Z_2\), while restricting \(Z_1\) to the continuation region. Adding the two parts gives
The stagewise p-value is approximately 0.0414. In contrast, the ordinary two-sided tail at the final look is \(2\{1-\Phi(2.20)\}\approx0.0278\). That ordinary calculation omits the probability of stopping at look 1, so it is not the stagewise p-value for this stopped outcome. The stagewise result includes both directions at both looks, as required by the ordering.
What Symmetry Contributes
With symmetric boundaries, changing the sign of every statistic leaves the stopping rule unchanged. The null model is also symmetric about zero. Consequently, outcomes \((j,z)\) and \((j,-z)\) have the same stagewise rank and the same p-value. The absolute value \(r=|z|\) is sufficient for the within-look part of this particular ordering.
This is why a two-sided calculation is not obtained by simply doubling a one-sided stagewise p-value in every design. Such doubling is justified only when the two directional tail events have the needed symmetry and the ordering and stopping rule treat the signs symmetrically. Here those conditions hold by construction. When boundaries or stopping rules differ by direction, absolute-value ranking need not represent the design’s evidence ordering; a different construction is required.
Worked Example: Reversing the Sign at the First Look
Return to the first-look boundary \(b_1=2.4\), and suppose a hypothetical trial stops at look 1 with \(Z_1=-2.60\). The observed result has the same magnitude as the positive result in the earlier calculation. The stopping condition holds because \(|-2.60|=2.60\ge2.4\), and the two-sided stagewise p-value is
Using only the lower tail would give \(P(Z_1\le-2.60)=0.004661\), which is a one-sided tail probability, not the requested two-sided p-value. The other direction contributes the same probability under the symmetric null model, yielding 0.009322 in total.
Diagnostics, Alternatives, and Limitations
Before reporting a two-sided stagewise p-value, check the construction against the design and the observed outcome. In particular, verify that the stopping look is correctly identified, that all earlier continuation conditions are included, and that both signs are included in each two-sided tail. For a final-look observation, make sure the probability of all earlier stops is added. A useful numerical check is to compare the integrated stagewise tail with the probability obtained by an independent implementation of the same boundary regions.
- Using only the observed look. Computing \(2\{1-\Phi(|z|)\}\) ignores the earlier stopping opportunities. Use the stopped-outcome ordering and its null probabilities instead.
- Counting only one direction. A positive-tail calculation misses negative evidence when the design and question are two-sided. Include both tails at each look.
- Using the wrong continuation region. The look-\(j\) probability must require continuation at every earlier look. Otherwise, paths that would already have stopped are counted again.
- Assuming every outcome at a look is a boundary crossing. At the final look, the trial stops even if the statistic is between the boundaries. The ordering still ranks terminal outcomes by absolute statistic.
The stagewise p-value is unconditional over stopping looks under the specified null model: it includes the chance of stopping earlier rather than conditioning on the observed look. An ordinary fixed-look two-sided p-value answers a different calculation and may be misleading after a data-dependent stop. The stagewise value also depends on the chosen ordering; another ordering of stopped outcomes can give a different, still potentially valid, p-value.
The method’s limitations follow from those choices. Its calibration relies on the assumed joint null distribution and the actual design’s information levels and stopping rule. Model misspecification, unrecorded changes to the look schedule, or inaccurate numerical integration can affect the result. The symmetric absolute-value ordering is not automatically appropriate when the design is not symmetric. A reported value should therefore be understood as the tail probability for this specified ordering and design, not as a universal summary of sequential evidence.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In a two-look symmetric design, what parts of the stopped-outcome event contribute to the stagewise p-value for an observation at look 2?
- Why is the ordinary two-sided normal tail generally not the stagewise p-value for a final-look observation?
- Under the two-look model above, what is the null correlation when \(I_1/I_2=0.36\)?
- At the first look, suppose \(Z_1=-2.00\) and the stopping boundary is \(b_1=2.4\). Does the trial stop? If the observation were used as a first-look result, what is its two-sided tail probability under the standard normal null?
- Why can absolute-value ordering fail to match the design when the positive and negative boundaries are not symmetric?
Solutions. (1) Count the probability of stopping at look 1 in either direction, then add the probability of reaching look 2 with \(|Z_2|\) at least as large as the observed absolute statistic. (2) The ordinary tail uses only the final statistic’s marginal distribution and omits the probability of earlier stops under the sequential rule. (3) The correlation is \(\sqrt{0.36}=0.6\). (4) No: \(|-2.00|=2.00<2.4\), so the trial continues under that boundary rule. The first-look two-sided tail probability is \(2\{1-\Phi(2.00)\}\approx2(0.02275)=0.0455\). (5) If the boundaries differ by sign, sign reversal can change whether or when stopping occurs. Ranking solely by \(|Z_k|\) may therefore rank outcomes inconsistently with the actual stopping rule.