Calculating a One-Sided Stagewise P-Value

Why the Stopping Look Matters

In a group sequential trial, the test may stop as soon as an interim result crosses a prespecified efficacy boundary. The observed result therefore includes both a test statistic and the look at which the trial stopped. A one-sided stagewise p-value assesses how much null probability is assigned to stopped outcomes at least as favorable as that observed result, under a specified ordering of those outcomes.

The calculation below uses two planned looks and fixed upper efficacy boundaries. It focuses on the stagewise ordering in which stopping earlier is more favorable evidence than stopping later, and, within a given look, larger values of the one-sided test statistic are more favorable. This ordering must be specified before calculating the p-value; it is not a rule that follows automatically from the observed data.

Definition: Under the one-sided stagewise ordering used here, an outcome stopping at an earlier look ranks at least as favorably as an outcome stopping at a later look. Among outcomes stopping at the same look, a larger value of the one-sided statistic \(Z_k\) ranks as more favorable evidence. The stagewise p-value is the null probability of the set of stopped outcomes that rank at least as favorably as the observed outcome.

The ordering is a way to rank stopped outcomes, while the p-value is a probability calculated under the null distribution. In particular, the p-value is not the probability that the null hypothesis is true. The validity result for this type of p-value is that, under the null and the stated ordering and design, its distribution is superuniform: \(P(p\leq a)\leq a\) for every \(0\leq a\leq1\).

Model and Calculation for Two Looks

Suppose the one-sided hypotheses are \(H_0:\theta=0\) against \(H_a:\theta>0\), and larger \(Z\)-statistics support \(H_a\). Let \(Z_1\) and \(Z_2\) be the statistics at the interim and final looks. Under \(H_0\), assume the canonical joint normal model

(Z_1,Z_2)\sim N\left( \begin{pmatrix}0\\0\end{pmatrix}, \begin{pmatrix}1&\rho\\\rho&1\end{pmatrix} \right), \qquad \rho=\sqrt{\frac{I_1}{I_2}},

where \(I_1<I_2\) are the information levels at the two looks. This model assumes the look schedule, analysis, and boundaries were fixed as planned, and that the standardized statistics have the stated null distribution. In applications the normal model may be exact or an approximation, depending on the endpoint and analysis.

Let \(c_1\) and \(c_2\) be the fixed upper efficacy boundaries: stop and reject \(H_0\) at look 1 if \(Z_1\geq c_1\); otherwise continue and stop at look 2, rejecting if \(Z_2\geq c_2\). If the observed outcome stops at look 2 with \(Z_2=z\), the outcomes at least as favorable under the stated ordering are (i) all outcomes stopping at look 1, and (ii) outcomes that continue to look 2 and have \(Z_2\geq z\). Thus

p_{\mathrm{stage}}(2,z) = P_0(Z_1\geq c_1) + P_0(Z_1<c_1,\ Z_2\geq z).

The subscript 0 indicates that probabilities are calculated under \(H_0\). For calculation, the conditional distribution is

Z_2\mid Z_1=x\sim N\left(\rho x,\,1-\rho^2\right),

so the second probability can be written as a one-dimensional integral:

P_0(Z_1<c_1,\ Z_2\geq z) = \int_{-\infty}^{c_1} \phi(x)\, \Phi\left(\frac{\rho x-z}{\sqrt{1-\rho^2}}\right)\,dx.

Here \(\phi\) and \(\Phi\) are the standard normal density and cumulative distribution function. This integral accounts for the dependence between the two looks; multiplying marginal probabilities would not do so.

Worked Example: Stopping at the Interim Look

Suppose the planned interim boundary is \(c_1=2.40\), and a hypothetical trial stops at look 1 with \(Z_1=2.65\). Under \(H_0\), assume \(Z_1\sim N(0,1)\), and use the one-sided stagewise ordering defined above.

Because the observed outcome stops at the first look, outcomes at least as favorable must also stop at the first look and have \(Z_1\geq2.65\). The boundary is satisfied because \(2.65\geq2.40\). Therefore

p_{\mathrm{stage}}(1,2.65) =P_0(Z_1\geq2.65) =1-\Phi(2.65) \approx 0.0040.

The p-value is about 0.0040. Under the null model, approximately 0.40% of trials would produce a first-look statistic at least as large as 2.65. This probability refers to repeated trials under \(H_0\), not to the probability that \(H_0\) is true.

A Final-Look Calculation

At a final-look stop, the earlier-stop outcomes must be included because they rank at least as favorably as the observed final-look result under this ordering. The remaining probability is for trials that did not cross the interim boundary but reached the final look with a statistic at least as large as the observed one.

Worked Example: A Final-Look Stagewise P-Value

Consider a hypothetical two-look design with information levels \(I_1=0.50\) and \(I_2=1.00\), and fixed boundaries \(c_1=2.40\) and \(c_2=2.00\). A hypothetical trial does not cross the interim boundary and stops at the final look with \(Z_2=2.35\). The model assumptions are the canonical joint normal null model above, a fixed look schedule, and the planned rule to continue only when \(Z_1<2.40\). The final boundary is crossed because \(2.35\geq2.00\).

First calculate the null correlation and conditional standard deviation:

\rho=\sqrt{\frac{0.50}{1.00}}=0.7071, \qquad \sqrt{1-\rho^2}=\sqrt{0.50}=0.7071.

The probability of an earlier stop is

P_0(Z_1\geq2.40)=1-\Phi(2.40)\approx0.00820.

For trials that continue, use the conditional integral with \(z=2.35\):

\begin{aligned} P_0(Z_1<2.40,\ Z_2\geq2.35) &=\int_{-\infty}^{2.40} \phi(x)\, \Phi\left(\frac{0.7071x-2.35}{0.7071}\right)\,dx\\ &\approx0.00706. \end{aligned}

Numerical evaluation of this one-dimensional normal integral gives the second probability. Adding the two parts gives

p_{\mathrm{stage}}(2,2.35) \approx0.00820+0.00706=0.01526.

Thus the one-sided stagewise p-value is approximately 0.0153. Under the null model, about 1.53% of trials would stop at look 1 or would continue and reach a final statistic of at least 2.35. The early-stop probability is included because of the specified stagewise ordering, not because the hypothetical trial itself stopped early.

Worked Example: A More Extreme Final Statistic

Keep the same hypothetical design and suppose instead that the final statistic is \(Z_2=2.60\), with no interim crossing. This is a more favorable final-look outcome than \(Z_2=2.35\). The early-stop probability is unchanged at \(0.00820\). The final-look contribution is now

\begin{aligned} P_0(Z_1<2.40,\ Z_2\geq2.60) &=\int_{-\infty}^{2.40} \phi(x)\, \Phi\left(\frac{0.7071x-2.60}{0.7071}\right)\,dx\\ &\approx0.00317. \end{aligned}

Consequently,

p_{\mathrm{stage}}(2,2.60) \approx0.00820+0.00317=0.01137.

The p-value, about 0.0114, is smaller than the value for \(Z_2=2.35\), as it should be: the set of outcomes at least as favorable as 2.60 is contained within the set at least as favorable as 2.35. The unchanged early-stop contribution explains why the p-value is not just the upper tail probability at the final look.

Why This Is Not the Ordinary Final-Look P-Value

For \(Z_2=2.35\), the ordinary one-sided standard normal tail at the final look is \(1-\Phi(2.35)\approx0.00939\). That calculation treats the final statistic as if it were the only result relevant to the analysis. It does not account for the rule that the trial could have stopped at the interim look, or for the ordering that places those earlier stopped outcomes ahead of a final-look outcome. Under the stagewise ordering here, the correct null probability is about 0.0153.

This comparison is specific to the ordering and design used. Other p-value constructions can use different orderings or condition on different information. The important point for this calculation is to use the planned stopping rule and the joint null distribution, rather than substitute a fixed-sample formula that ignores them.

Diagnostics, Assumptions, and Common Errors

Check the stopping rule. In the example, the final-look probability is restricted to \(Z_1<2.40\). Integrating over all \(Z_1\) would include paths that should already have stopped.
Check the dependence. With information fraction \(0.50\), the canonical correlation is \(0.7071\), not zero. Treating the looks as independent changes the probability of reaching the final statistic.
Check that boundaries and timing match the analysis. If the actual design used a different look schedule, boundary, or stopping rule, the probability calculation must use that design. Unplanned changes can invalidate the stated null reference distribution.
Check numerical integration. The conditional integrand is nonnegative and the resulting probability must lie between zero and one. Compare numerical precision across tighter integration tolerances or an independent bivariate-normal calculation when the result is consequential.

Common mistake: “The observed final statistic is 2.35, so the p-value is \(1-\Phi(2.35)=0.00939\).” This uses a fixed-final-look tail and omits the earlier stopped outcomes required by the stagewise ordering. For the stated design, add the null probability of stopping at look 1 to the null probability of continuing and obtaining \(Z_2\geq2.35\), giving approximately 0.0153.

The calculation also has limits. It relies on the assumed null distribution and on the specified stopping and ordering rules. If the normal model is only approximate, the p-value inherits that approximation. A stagewise p-value is not automatically a conditional probability given the observed stopping look, and it is not interchangeable with every other sequential p-value definition.

Key takeaway: For a final-look result \(z\) under the one-sided stagewise ordering used here, count null outcomes that stopped earlier and outcomes that continued to the observed look with statistic at least \(z\). Use the design's joint null distribution to evaluate that probability.

Check Your Understanding

Answer each question using the two-look setting and ordering defined above.

For a final-look observation \(Z_2=z\), which two kinds of outcomes contribute to the stagewise p-value?
If the observed final statistic increases from 2.35 to 2.60 while the design remains fixed, should the stagewise p-value increase or decrease?
Why is it incorrect to calculate the final-look contribution as \(P_0(Z_2\geq z)\) without restricting to paths that continued?
In the worked design, what is the null correlation between the two statistics, and what information fraction determines it?

Solutions. (1) All outcomes stopping at look 1, plus outcomes with \(Z_1<c_1\) and \(Z_2\geq z\). (2) It should decrease: the set of outcomes at least as favorable becomes smaller. The worked values are approximately 0.0153 and 0.0114. (3) It includes paths that crossed the interim boundary and should have stopped before the final look. (4) The correlation is \(\sqrt{I_1/I_2}=\sqrt{0.50}=0.7071\); the information fraction is \(I_1/I_2=0.50\).