Stagewise P-Values With Asymmetric Boundaries

Why Asymmetric Boundaries Require an Explicit Ordering

When an interim design has different upper and lower stopping boundaries, the stopping rule is not symmetric: the trial may stop for efficacy at one value and for the corresponding lower-side evidence at a different magnitude. A two-sided p-value still needs a rule for comparing an observed upper-side outcome with possible lower-side outcomes. Without that rule, “at least as extreme” is ambiguous, and different conventions can produce different p-values.

This tutorial uses an absolute-statistic ordering: outcomes on opposite sides are compared by the absolute value of the standardized statistic at the look where stopping occurs. Thus, an upper-side value $Z_k=c$ and a lower-side value $Z_k=-c$ have equal extremeness, even if the design’s upper and lower boundaries differ. The stopping boundaries determine which outcomes are possible; they do not rescale the ordering.

Definition: Under the absolute-statistic stagewise ordering, an observed stopped outcome with statistic $Z_K=z$ has two-sided stagewise p-value $$ p_{\mathrm{st}}(z,K)=P_{H_0}\!\left(|Z_J|\ge |z| \text{ at the actual stopping look } J\right). $$ The probability is calculated under the null distribution and the prespecified stopping rule. It includes qualifying stopped outcomes at every look, not only outcomes at the observed look.

This convention is one precise way to extend two-sided stagewise ordering to asymmetric boundaries. It is not the only conceivable ordering. The convention must be stated because a boundary-scaled ordering, for example, would compare opposite-side outcomes differently and would generally yield a different p-value.

Model and Probability Regions

For a two-look illustration, let $Z_1$ and $Z_2$ be the standardized test statistics at information fractions $t_1$ and $t_2=1$. Under $H_0$, assume the canonical joint normal model $$ (Z_1,Z_2)\sim N\!\left( \begin{pmatrix}0\\0\end{pmatrix}, \begin{pmatrix}1&\rho\\\rho&1\end{pmatrix} \right), \qquad \rho=\sqrt{t_1}. $$

The model assumes the planned information times and analysis statistic are correctly specified, the null means are zero, and the joint distribution is the canonical normal distribution. At look 1, stop if $Z_1\ge a_1$ or $Z_1\le -b_1$, where $a_1>0$ and $b_1>0$. Otherwise continue to look 2, where the trial ends. Write the continuation region as $C=\{-b_1<Z_1<a_1\}$. The two early stopping regions and the final continuation region are disjoint, so their probabilities can be added without double-counting.

For an observed absolute statistic $c=|z|$, the qualifying early upper and lower outcomes have regions $Z_1\ge \max(a_1,c)$ and $Z_1\le-\max(b_1,c)$. The qualifying final outcomes have $Z_1\in C$ and either $Z_2\ge c$ or $Z_2\le-c$. Therefore

p_{\mathrm{st}}(c)= P\{Z_1\ge\max(a_1,c)\} +P\{Z_1\le-\max(b_1,c)\} +P\{Z_1\in C,\ Z_2\ge c\} +P\{Z_1\in C,\ Z_2\le-c\}.

The early terms use the marginal standard normal distribution of $Z_1$. The final terms must account for the correlation between the looks and the fact that a final-look outcome is observed only if the trial continued. They are not generally the ordinary marginal tail probabilities of $Z_2$.

Calculating Final-Look Contributions

Under the canonical model, $Z_1\mid Z_2=y$ is normal with mean $\rho y$ and variance $1-\rho^2$. Let $\phi$ and $\Phi$ denote the standard normal density and distribution functions. Conditioning on $Z_2$ gives the final upper contribution

P(Z_1\in C,\ Z_2\ge c) =\int_c^\infty \phi(y) \left[ \Phi\!\left(\frac{a_1-\rho y}{\sqrt{1-\rho^2}}\right) -\Phi\!\left(\frac{-b_1-\rho y}{\sqrt{1-\rho^2}}\right) \right]dy.

The lower contribution is

P(Z_1\in C,\ Z_2\le-c) =\int_{-\infty}^{-c} \phi(y) \left[ \Phi\!\left(\frac{a_1-\rho y}{\sqrt{1-\rho^2}}\right) -\Phi\!\left(\frac{-b_1-\rho y}{\sqrt{1-\rho^2}}\right) \right]dy.

These expressions integrate the probability of continuing at look 1 conditional on each final statistic. The conditional continuation probability differs in the upper and lower tails when the boundaries are asymmetric. It is therefore unsafe to assume the two final contributions are equal, even though the ordering pairs $+c$ and $-c$ at equal absolute values.

Worked Example: Final-Look Outcome at an Absolute Statistic of 2.00

Suppose the hypothetical design has $t_1=0.50$, $a_1=2.25$, and $b_1=2.20$. The observed trial continues at look 1 and ends with $Z_2=2.00$. Use the absolute-statistic ordering to calculate its two-sided stagewise p-value.

Here $\rho=\sqrt{0.50}=0.7071$, and $c=2.00$. Because $c$ is below both early stopping magnitudes, all early stops qualify. Their probabilities are

P(Z_1\ge2.25)=\Phi(-2.25)\approx0.0122,\qquad P(Z_1\le-2.20)=\Phi(-2.20)\approx0.0139.

For the final contributions, $\sqrt{1-\rho^2}=0.7071$. Substitution in the conditional-normal integrals gives, by numerical integration, approximately $0.018$ for $Z_2\ge2.00$ after continuation and $0.017$ for $Z_2\le-2.00$ after continuation. These are continuation-restricted probabilities, not the marginal probabilities $P(Z_2\ge2.00)$ and $P(Z_2\le-2.00)$.

Adding the four disjoint regions gives

p_{\mathrm{st}}(2.00)\approx0.0122+0.0139+0.018+0.017=0.0611.

Under the stated ordering and null model, the observed final statistic has a two-sided stagewise p-value of about 0.061. This is the null probability of a stopped outcome with an absolute statistic at least 2.00, not the probability that $H_0$ is true.

Worked Example: A More Extreme Final-Look Outcome

Keep the same hypothetical design, but suppose the observed final statistic is $Z_2=-2.50$, after continuation. The direction is lower, but the stated ordering compares it with upper-side outcomes having absolute statistic at least 2.50 as well.

The early upper and lower regions now begin at absolute statistic 2.50, which exceeds both stopping thresholds. Thus each early contribution is $\Phi(-2.50)\approx0.00621$. Numerical integration of the conditional-normal expressions gives final continuation contributions of approximately $0.0039$ in the upper tail and $0.0038$ in the lower tail. Consequently,

p_{\mathrm{st}}(2.50)\approx0.00621+0.00621+0.0039+0.0038=0.0201.

The result is two-sided despite the observed statistic being negative: both final tails and both early sides enter the calculation. The upper and lower final terms are close here, but they are not set equal by the ordering; each is obtained from its own continuation-restricted probability.

Why the Tail Sum Is a Valid P-Value

Let $R=|Z_J|$, where $J$ is the look at which the trial actually stops. The p-value above is the null survival probability $P_{H_0}(R\ge R_{\mathrm{obs}})$, calculated using the distribution induced by the stopping rule. This provides a short justification for validity.

Order the stopped outcomes.
Assign each outcome the score $R=|Z_J|$. Larger scores are at least as extreme under the declared convention, regardless of side or look.

Use the null distribution of that score.
Calculate its upper-tail probability under $H_0$, including the actual stopping rule and all looks.

Obtain superuniformity.
For a continuous null distribution of $R$, the tail probability $P_{H_0}(R\ge R_{\mathrm{obs}})$ is uniform under $H_0$; with ties, using the inclusive tail makes it conservative. In either case, $P_{H_0}(p_{\mathrm{st}}\le\alpha)\le\alpha$.

The argument depends on using the distribution of the statistic at the actual stopping look. Replacing it with the fixed-look distribution of $Z_2$, or ignoring the early stopping regions, changes the reference distribution and does not establish validity for the sequential procedure.

Worked Example: What the Opposite-Side Convention Changes

To isolate the ordering rule, consider a separate, explicitly hypothetical discrete null distribution; it is not an integration result from the canonical normal model. Suppose its stopped outcomes have the following probability masses:

Stopped-outcome category	Absolute statistic	Null probability
Look 1, upper side	2.50	0.010
Look 1, lower side	2.50	0.012
Look 1, upper side	2.80	0.006
Look 1, lower side	2.80	0.007
Final look after continuation	2.50	0.015
Final look after continuation	2.80	0.008
All remaining outcomes	Less than 2.50	0.942

The probabilities sum to $0.010+0.012+0.006+0.007+0.015+0.008+0.942=1.000$. For an observed upper-side outcome with absolute statistic 2.50, the declared convention counts every outcome with absolute statistic at least 2.50, including the lower-side mass at 2.50. The p-value is therefore

p_{\mathrm{st}}=0.010+0.012+0.006+0.007+0.015+0.008=0.058.

If the lower-side outcomes were omitted merely because the observed result was upper-side, the calculation would instead be one-sided and would not answer the stated two-sided question. In a continuous model exact ties have probability zero; in a discrete setting, including ties is the conservative convention used here.

Diagnostics, Alternatives, and Limitations

A reliable calculation can be checked structurally before trusting its numerical value. The early upper, early lower, final upper, and final lower regions must be disjoint. Their probabilities cannot sum to more than one. At a threshold below both early boundaries, the early contributions must equal the probabilities of crossing those boundaries; above both boundaries, they are the two standard normal tails beyond the threshold. As $c$ increases, every qualifying region shrinks, so the p-value cannot increase.

For numerical integration, evaluate upper and lower final contributions separately, verify the conditional mean $\rho y$ and variance $1-\rho^2$, and check that the continuation interval uses the correct asymmetric endpoints $-b_1$ and $a_1$. A common implementation error is to use $P(|Z_2|\ge c)$ for the final terms. That incorrectly counts trials that stopped at look 1 and ignores the dependence between $Z_1$ and $Z_2$.

The absolute-statistic convention is straightforward to explain and pairs equal-magnitude outcomes across sides. Other orderings, such as scaling each side by its own boundary or ordering by a likelihood ratio, answer different questions about extremeness. Their p-values need not agree. The ordinary fixed-look two-sided normal p-value is also an alternative only when its reference distribution matches the analysis; it is not a substitute for incorporating the sequential stopping rule.

The result depends on the prespecified boundaries, look schedule, statistic, null model, and ordering convention. Model misspecification, unplanned changes to the analysis, or inaccurate numerical integration can undermine the reported calibration. A p-value also does not measure effect magnitude or clinical importance. Report the ordering convention and the analysis that produced the p-value so readers can interpret what was counted as equally or more extreme.

Common Mistakes

Pairing sides by their boundary values without saying so. The boundary-scaled rule is a different ordering. Under the convention here, compare opposite sides by the same absolute statistic.
Using only the observed side. A two-sided p-value includes qualifying outcomes on both sides, even when the boundaries are asymmetric.
Using the final marginal tail alone. Final outcomes are observed only after continuation, so use the joint null distribution restricted to the continuation region.
Adding early and final probabilities that overlap. Define the stopped-outcome regions first; only disjoint regions may be summed.
Calling the result the probability that the null is true. It is a tail probability under $H_0$, computed according to the specified stopped-outcome ordering.

Key takeaway: With asymmetric boundaries, state how opposite sides are ordered. Under absolute-statistic ordering, calculate the null probability of all stopped outcomes whose absolute statistic meets or exceeds the observed value, accounting for continuation at each later look.

Check Your Understanding

Answer each question using the absolute-statistic ordering defined above.

If the upper and lower early boundaries are 2.3 and -2.1, respectively, how is an observed upper-side statistic of 2.6 compared with a lower-side outcome?
For an observed final statistic of -2.4, which early-stop and final-look regions contribute to its two-sided p-value?
Why is $P(|Z_2|\ge2.4)$ generally not the final-look contribution after early stopping?
In the hypothetical discrete example, what p-value results for an observed outcome with absolute statistic 2.80?

Solutions. (1) Compare by absolute value: lower-side outcomes with statistic at most $-2.6$ are equally or more extreme; the unequal boundaries do not change this pairing. (2) Include early outcomes with $Z_1\ge\max(a_1,2.4)$ or $Z_1\le-\max(b_1,2.4)$, and final outcomes after continuation with $Z_2\ge2.4$ or $Z_2\le-2.4$. (3) That marginal probability includes final statistics from trials that stopped early; the correct contribution restricts to the continuation region and uses the joint distribution. (4) Only the two look-1 masses at 2.80 and the final mass at 2.80 qualify, so $0.006+0.007+0.008=0.021$.