Reconstructing a P-Value From a Boundary Crossing

From an Observed Crossing to a Null Outcome Set

When a group sequential trial stops at an efficacy boundary, the reported statistic is not the result of a single fixed-look test: the stopping rule has already determined which outcomes could have been observed. To reconstruct a stagewise p-value, specify that rule, order the possible stopped outcomes by their stagewise statistics, and find the null probability of outcomes at least as extreme as the observation.

This tutorial focuses on a one-sided upper-tail ordering. Suppose the observed trial crosses an upper efficacy boundary at look $k$, with observed standardized statistic $z_{\mathrm{obs}}$. The p-value is based on the probability, under $H_0$, of stopping at any planned look with a stopped statistic at least $z_{\mathrm{obs}}$. It is not generally just the upper-tail probability at look $k$, nor is it necessarily the probability of crossing the planned efficacy boundary by look $k$.

Definition: Let $T$ be the stopping look, and let $Z_j$ be the standardized test statistic at look $j$. Under the specified stopping rule, the ordered null outcome set for an observed value $z_{\mathrm{obs}}$ is the set of stopped outcomes $(j,Z_j)$ satisfying $Z_j \ge z_{\mathrm{obs}}$. The one-sided stagewise p-value is $$ p_{\mathrm{stage}}(z_{\mathrm{obs}})=P_0(Z_T\ge z_{\mathrm{obs}}) =\sum_{j=1}^{K}P_0(T=j,\ Z_j\ge z_{\mathrm{obs}}), $$ where $K$ is the final planned look and $P_0$ denotes probability under $H_0$.

The terms in the sum refer to distinct events: an outcome can stop at only one look. Each term must include the condition that the trial actually stops at that look. For example, a look-2 outcome belongs in the look-2 term only if the trial continued past look 1. If the design has futility stopping as well as efficacy stopping, outcomes stopped for futility are included only when their stopped statistic meets the ordering criterion.

For a continuous statistic, the distinction between $>$ and $\ge$ at a single point does not change the probability. Using $\ge$ makes the ordering explicit and also gives the appropriate inclusive rule when the statistic is discrete or reported with ties. The p-value is a null probability; it is not the probability that $H_0$ is true.

Model and Calculation

For the examples, assume a fixed design with planned information levels $I_1<\cdots<I_K$. Under the canonical group sequential model under $H_0$, the standardized statistics have a joint multivariate normal distribution with mean zero, unit variances, and

\operatorname{Corr}_0(Z_i,Z_j)=\sqrt{\frac{I_i}{I_j}},\qquad i<j.

This model reflects independent information increments: statistics from later looks include the earlier accumulated information, so they are correlated. Assume the information schedule and stopping boundaries are fixed in advance, the observed statistic is computed using the design’s specified statistic and information, and the model adequately describes its null distribution. If the design uses a different specified joint null distribution, that distribution—not the canonical normal model—must be used for the probability calculation.

For a design with upper efficacy boundaries $u_j$ and lower futility boundaries $l_j$, one possible rule is to stop at look $j<K$ if $Z_j\ge u_j$ or $Z_j\le l_j$, and otherwise continue. At the final look, stop regardless. Thus, for $j<K$, the event $T=j$ includes the boundary-crossing conditions at look $j$ and the continuation conditions at all earlier looks. Those conditions are part of the probability calculation; they cannot be discarded when forming the ordered set.

Calculation principle: First write the event $T=j,\ Z_j\ge z_{\mathrm{obs}}$ for each possible stopping look. Then evaluate each event under the null distribution and add the probabilities. The event definition comes from the ordering and stopping rule; numerical integration is only a way to evaluate its probability.

Worked Reconstructions

Worked Example: A Single Final Look

Consider a hypothetical one-look design with $Z_1\sim N(0,1)$ under $H_0$. The upper efficacy boundary is 1.645, and the observed statistic is $z_{\mathrm{obs}}=1.70$. The observation crosses the boundary. Identify the ordered set and calculate its p-value.

There is only one possible stopping look, so the ordered null outcome set is $Z_1\ge1.70$. The boundary determines whether the observed trial stops, but the ordering threshold is the observed statistic, not the boundary:

p_{\mathrm{stage}}=P_0(Z_1\ge1.70) =1-\Phi(1.70) \approx 0.0446.

The p-value is about 0.0446. Under this one-look design it equals the usual one-sided fixed-look tail probability, because there are no other stopped outcomes to include. That equality need not hold when there are multiple looks.

Worked Example: Two Looks With Zero Boundaries

In a hypothetical two-look design, let $I_1/I_2=0.25$. At look 1, stop for efficacy if $Z_1\ge0$; otherwise continue to look 2, where the trial stops. Suppose the observed final-look statistic is $z_{\mathrm{obs}}=0$. Under $H_0$, the canonical model gives $\operatorname{Corr}(Z_1,Z_2)=\sqrt{0.25}=0.5$. Calculate the stagewise p-value.

The ordered set consists of a look-1 stop with $Z_1\ge0$, or a look-2 stop with $Z_2\ge0$ after continuing, which requires $Z_1<0$. Therefore,

p_{\mathrm{stage}} =P_0(Z_1\ge0)+P_0(Z_1<0,\ Z_2\ge0) =1-P_0(Z_1<0,\ Z_2<0).

For a standard bivariate normal pair with correlation $\rho$, the probability that both variables are negative is $1/4+\arcsin(\rho)/(2\pi)$. Substituting $\rho=0.5$, for which $\arcsin(0.5)=\pi/6$, gives

p_{\mathrm{stage}} =1-\left(\frac14+\frac{\pi/6}{2\pi}\right) =1-\left(\frac14+\frac1{12}\right) =\frac23 \approx0.6667.

The nominal upper-tail probability at the final look alone is $P_0(Z_2\ge0)=0.5$, not $0.6667$. The difference arises because the ordered set also includes look-1 stopped outcomes with $Z_1\ge0$. The example uses a boundary value of zero for transparent arithmetic; the same event-partitioning logic applies with nonzero boundaries.

Worked Example: A Final-Look Crossing Above Its Boundary

Consider a hypothetical two-look design with $I_1/I_2=0.25$, upper efficacy boundaries $u_1=1.5$ and $u_2=1.8$, and no interim futility stop. An observed trial continues at look 1 and stops for efficacy at look 2 with $z_{\mathrm{obs}}=2.0$. Calculate the stagewise p-value under the canonical null model.

The correlation is $0.5$. An ordered outcome stopped at look 1 must have $Z_1\ge2.0$; this already implies crossing the look-1 boundary of 1.5. An ordered outcome stopped at look 2 must have $Z_1<1.5$ and $Z_2\ge2.0$. Thus,

p_{\mathrm{stage}} =P_0(Z_1\ge2.0)+P_0(Z_1<1.5,\ Z_2\ge2.0).

The first probability is $1-\Phi(2.0)\approx0.02275$. Under the bivariate normal model, $Z_2\mid Z_1=x\sim N(0.5x,0.75)$, so the second probability is

\int_{-\infty}^{1.5}\phi(x) \left[1-\Phi\left(\frac{2.0-0.5x}{\sqrt{0.75}}\right)\right]\,dx \approx0.0146.

Consequently,

p_{\mathrm{stage}}\approx0.02275+0.0146=0.03735\approx0.0374.

The calculation includes both qualifying early stops and qualifying final stops, with the continuation requirement applied to the latter. In contrast, the fixed-look upper-tail probability $1-\Phi(2.0)\approx0.02275$ omits the additional qualifying look-1 outcomes.

Why the Event Definition Matters

The derivation is a direct consequence of two facts. First, the ordering declares an outcome at least as extreme when its stopped statistic is at least $z_{\mathrm{obs}}$. Second, the stopping time takes exactly one value from $1,\ldots,K$. Therefore, the event $Z_T\ge z_{\mathrm{obs}}$ can be partitioned into disjoint stopping-look events:

\{Z_T\ge z_{\mathrm{obs}}\} =\bigcup_{j=1}^{K}\{T=j,\ Z_j\ge z_{\mathrm{obs}}\}, \qquad P_0(Z_T\ge z_{\mathrm{obs}}) =\sum_{j=1}^{K}P_0(T=j,\ Z_j\ge z_{\mathrm{obs}}).

This partition also explains why the rule must be reconstructed before calculating. If look 1 has an upper boundary $u_1$, the look-2 term includes $Z_1<u_1$ when continuation is required. Counting the unrestricted event $Z_2\ge z_{\mathrm{obs}}$ would include paths that actually stopped at look 1, counting some paths twice or assigning them to the wrong stopped outcome.

When the observation is exactly at an upper boundary $u_k$, set $z_{\mathrm{obs}}=u_k$ in the ordering. Earlier and later stopped outcomes are still assessed by their own $Z_j$ values against that threshold. The resulting stagewise p-value is not automatically equal to the design’s cumulative alpha-spending probability through look $k$: those quantities refer to different events unless the ordering set and the boundary-crossing set coincide.

Diagnostics, Alternatives, and Limitations

Before trusting a reconstructed value, check the following:

Stopping logic: Confirm whether each look stops for efficacy, futility, or neither, and encode continuation conditions for every later-look event.
Observed statistic: Use the statistic and scale specified by the design. A raw treatment-effect estimate cannot replace a standardized $Z_j$ in this ordering without a justified conversion.
Information and dependence: Verify the information levels and null covariance. Treating correlated looks as independent generally gives incorrect probabilities.
Ordering: Confirm that the requested p-value uses the stagewise ordering. A different ordering can define a different ordered null outcome set for the same observation.
Boundary crossing: Check that the recorded crossing look agrees with the stated stopping rule. A path crossing an earlier boundary should not be analyzed as if it continued.

A fixed-look p-value uses only the marginal null tail at the observed look. A conditional p-value conditions on the stopping look, changing the probability reference set. A likelihood-ratio ordering ranks stopped outcomes using their likelihood evidence rather than simply comparing their stagewise $Z_j$ values. These are different inferential conventions, not computational shortcuts for the same quantity. The ordering must be stated before its probability is calculated.

The calculation also depends on the design being correctly specified and on its null distribution being adequate. Unplanned changes to look timing, information, stopping boundaries, or the analysis statistic can invalidate a calculation based on the planned design. For complex boundaries or many looks, evaluating the joint probabilities may require numerical methods; an accurately written event definition remains essential regardless of the computation used.

Common Mistake

Common mistake: “The trial stopped at look 2, so the p-value is just $P_0(Z_2\ge z_{\mathrm{obs}})$.” This ignores qualifying outcomes that would have stopped earlier and may also include look-2 paths that did not actually reach look 2. The correction is to sum $P_0(T=j,\ Z_j\ge z_{\mathrm{obs}})$ over all planned stopping looks, applying the stopping rule in every term.

Check Your Understanding

For each question, identify the ordered set before evaluating its probability. Solutions follow each problem.

A one-look design has $Z_1\sim N(0,1)$ under $H_0$, and the observed statistic is 1.50. What is the one-sided stagewise p-value? Solution: The ordered set is $Z_1\ge1.50$, so $p=1-\Phi(1.50)\approx0.0668$.
In a two-look design, look 1 stops for efficacy at $Z_1\ge1.2$, and otherwise the trial stops at look 2. The observed final statistic is 1.4. Write the ordered null outcome set. Solution: It is the union of $\{T=1,Z_1\ge1.4\}$ and $\{T=2,Z_2\ge1.4\}$. Since reaching look 2 requires $Z_1<1.2$, the second event is $\{Z_1<1.2,Z_2\ge1.4\}$.
Why is $P_0(Z_2\ge z_{\mathrm{obs}})$ not generally the stagewise p-value when the observed trial stops at look 2? Solution: It omits qualifying stopped outcomes at earlier looks and does not restrict the event to paths that continued to look 2. The stagewise value sums the correctly restricted stopping-look probabilities.
For two looks with $I_1/I_2=0.36$, what is the canonical null correlation between $Z_1$ and $Z_2$? Solution: $\operatorname{Corr}_0(Z_1,Z_2)=\sqrt{0.36}=0.6$. The looks are not independent, so a joint calculation must use this dependence.