Numerical Integration for Stagewise P-Values

Why the integration regions matter

A stagewise p-value is a null probability assigned to a set of stopped outcomes: the observed outcome and outcomes at least as extreme under the specified ordering. When a trial can stop early, that set is not just a tail of the statistic at the last look. Its probability must account for which paths reach each look and which stopped outcomes meet the ordering criterion. Numerical integration makes those restrictions explicit.

The introductory material on stagewise tail probabilities and stopped-outcome probabilities motivates decomposing a probability by stopping look. Here the focus is implementing that decomposition with recursive normal integration. In practice, the most consequential errors are often not in the numerical integration routine itself, but in omitted continuation restrictions, incorrect limits, or an ordering event that does not match the definition of the p-value.

Model and recursive calculation

Let $Z_1,\ldots,Z_K$ be the standardized test statistics at $K$ planned looks. Under the null hypothesis $H_0$, assume the canonical joint normal model: each $Z_j$ has a standard normal marginal distribution, and for $i<j$, $ \operatorname{Corr}(Z_i,Z_j)=\sqrt{I_i/I_j} $, where $I_j$ is the cumulative information at look $j$, with $0<I_1<\cdots<I_K$. This model corresponds to independent increments of information and a prespecified information schedule. The conditional transition is

Z_j\mid Z_{j-1}=z \sim N\left(\sqrt{\frac{I_{j-1}}{I_j}}z,\ 1-\frac{I_{j-1}}{I_j}\right),\qquad j=2,\ldots,K.

Write $C_j$ for the continuation region at look $j$, and $S_j$ for the stopping region. For example, with efficacy and futility boundaries $L_j<U_j$, a common continuation region is $C_j=(L_j,U_j)$, and its complement is $S_j$. The boundaries and any ordering regions must be specified before integration. At the final look, every path still continuing is stopped, so $C_K$ is empty for purposes of probability accounting.

Definition: For a stopped outcome at look $j$, let $A_j$ be the subset of paths that both stop at $j$ and are at least as extreme as the observed outcome under the chosen stagewise ordering. The stagewise p-value is the null probability of the union of these disjoint sets: $$ p_{\mathrm{st}}=\sum_{j=1}^K P_{H_0}(A_j). $$ Each $A_j$ must include continuation at all earlier looks. If the ordering depends on the entire path, its event must retain that path information in the integral.

The joint density factors into the first-look standard normal density and the conditional transition densities. Thus, for an event specified by a path region $A_j$, its contribution is the integral of that product density over $C_1,\ldots,C_{j-1}$ and the ordered stopping region at look $j$. When the ordering depends only on the current statistic, a forward recursion can reduce the computation to one-dimensional integrals at each look.

q_1(z)=\phi(z)\mathbf{1}_{C_1}(z),\qquad q_j(z)=\mathbf{1}_{C_j}(z)\int_{C_{j-1}}q_{j-1}(x)f_j(z\mid x)\,dx.

Here $q_j(z)$ is the null density of reaching look $j$, continuing there, and having statistic $Z_j=z$; $f_j(z\mid x)$ is the conditional normal density given above. For an ordered stopping set $B_j\subseteq S_j$ determined by the current statistic, the contribution is $ \int q_{j-1}(x)P(Z_j\in B_j\mid Z_{j-1}=x)\,dx $, with the corresponding first-look expression $ \int_{B_1}\phi(z)\,dz $. If the ordering also depends on earlier statistics, use the joint density over the full path instead of collapsing the history into $q_j$.

Worked calculations

Worked Example: Two-Look Stagewise Tail Integration

Suppose a hypothetical two-look trial has equal information increments, so $I_1/I_2=1/2$. Under $H_0$, let the first-look efficacy boundary be $2.0$, with continuation $Z_1<2.0$; the final look is at $Z_2$. The observed path continues at look 1 and has $Z_2=2.4$. For this example, use the upper-tail stagewise ordering: a stopped outcome is at least as extreme when its standardized statistic is at least $2.4$. The ordered first-look contribution is therefore $P(Z_1\geq2.4)$, not $P(Z_1\geq2.0)$; early stops between 2.0 and 2.4 do not meet the ordering cutoff.

The final-look contribution requires both reaching the final look and meeting the ordering cutoff:

P(Z_1<2,\ Z_2\geq2.4) =\int_{2.4}^{\infty}\phi(t)P(Z_1<2\mid Z_2=t)\,dt.

The correlation is $1/\sqrt{2}$. By the bivariate normal conditional distribution, $Z_1\mid Z_2=t\sim N(t/\sqrt{2},1/2)$, so $P(Z_1<2\mid Z_2=t)=\Phi(2\sqrt{2}-t)$. Hence the terminal contribution is

\int_{2.4}^{\infty}\phi(t)\Phi(2\sqrt{2}-t)\,dt \approx 0.00440.

The first-look contribution is $P(Z_1\geq2.4)=1-\Phi(2.4)\approx0.00820$. Adding the disjoint contributions gives $p_{\mathrm{st}}\approx0.00820+0.00440=0.01260$. The final contribution is about $0.537$ of the marginal tail $P(Z_2\geq2.4)\approx0.00820$, because only that fraction of the final-tail probability also has $Z_1<2$. The continuation restriction is essential: using the marginal final tail alone would count paths that stopped earlier.

Worked Example: Recursive Limits in a Three-Look Design

Consider a hypothetical three-look design under the canonical null model, with information levels $I_1=1$, $I_2=2$, and $I_3=3$. Let the first-look continuation region be $C_1=(-1.5,1.5)$, and the second-look continuation region be $C_2=(-1.8,1.8)$. Suppose the event of interest is all stopped outcomes, so this calculation checks probability accounting rather than a particular tail ordering.

At look 1, the probability of stopping is $1-[\Phi(1.5)-\Phi(-1.5)]\approx1-0.8664=0.1336$. The density of paths that continue to look 2 is $q_1(z_1)=\phi(z_1)\mathbf{1}_{(-1.5,1.5)}(z_1)$. The transition from look 1 to look 2 has conditional mean $z_1/\sqrt{2}$ and variance $1/2$, so the probability of continuing at look 2 is

m_2=\int_{-1.5}^{1.5}\phi(z_1) \left[ \Phi\left(\frac{1.8-z_1/\sqrt{2}}{1/\sqrt{2}}\right) -\Phi\left(\frac{-1.8-z_1/\sqrt{2}}{1/\sqrt{2}}\right) \right]dz_1.

This expression shows both required limits: the outer integration is restricted to paths that reached look 2, and the conditional probability is bounded by the look-2 continuation limits. The probability of stopping at look 2 is $0.8664-m_2$. At look 3, the transition has conditional mean $\sqrt{2/3}\,z_2$ and variance $1/3$. Integrating that conditional density over all $z_3$ gives 1, so the probability of reaching and stopping at look 3 is $m_2$. The total is $0.1336+(0.8664-m_2)+m_2=1.0000$, regardless of the numerical value of $m_2$. A p-value calculation replaces “all stopped outcomes” with its ordered subset, but retains the same reach-look and stopping restrictions.

Implementation and numerical checks

A reliable implementation separates the probability model, the stopping rule, and the ordering rule. The following sequence is useful for a stagewise tail probability whose ordered region is expressed in terms of the path:

Specify the null transitions.
Record the information levels and calculate each conditional mean and variance. Confirm that the variance $1-I_{j-1}/I_j$ is positive.

Write the path restrictions.
For a contribution at look $j$, include continuation at every earlier look and stopping at $j$. Intersect these restrictions with the ordered event $A_j$.

Integrate recursively.
Use the conditional normal density at each transition, integrating only over the specified continuation intervals. Integrate the final stopping event over its actual limits, including infinite endpoints where required.

Check the accounting.
Calculate stopping probabilities for all stopping regions, not only the ordered subsets. Their sum, including the final-look probability, must equal 1 up to numerical tolerance.

For probability conservation, let $m_{j-1}$ be the probability of reaching look $j$. If $d_j$ is the probability of stopping there and $m_j$ the probability of continuing, then $d_j+m_j=m_{j-1}$. This follows because the stopping and continuation regions partition the possible values of $Z_j$, and the conditional normal density integrates to 1. At the first look, $m_0=1$; after the final look, the remaining continuation probability is zero. Therefore the sum of all stopping probabilities is 1. This identity is a direct diagnostic: a material discrepancy usually indicates a missing or overlapping region, a transition-density error, or inadequate numerical integration.

Worked Example: Conservation With a Symmetric First-Look Boundary

Suppose a hypothetical two-look design has $I_1/I_2=1/2$, stops at look 1 when $Z_1\geq0$, and otherwise continues to the final look. At the first look, $Z_1\sim N(0,1)$, so the early-stop probability is $P(Z_1\geq0)=0.5000$. The probability of reaching the final look is $P(Z_1<0)=\Phi(0)=0.5000$.

The conditional density of $Z_2$ given $Z_1=z$ is normal with mean $z/\sqrt{2}$ and variance $1/2$. Since the final look stops every remaining path, its integrated contribution is

\int_{-\infty}^{0}\phi(z) \left[\int_{-\infty}^{\infty} \frac{1}{\sqrt{1/2}}\, \phi\left(\frac{w-z/\sqrt{2}}{\sqrt{1/2}}\right)dw\right]dz =\int_{-\infty}^{0}\phi(z)\,dz=0.5000.

The inner integral is 1 because it is a conditional normal density over its full support. Adding the early and final stopping probabilities gives $0.5000+0.5000=1.0000$. For a stagewise p-value, the final inner integral would instead be restricted to the ordered region, such as $w\geq c$; the continuation limit $z<0$ would remain unchanged.

Diagnostics, alternatives, and limitations

Numerical integration is particularly useful when stopping regions or orderings make a simple closed-form calculation inconvenient. A multivariate normal probability routine can be an alternative for a low-dimensional path event; it evaluates the same null probability by a different computational route. Recursive integration makes the sequential limits visible and can exploit the Markov transitions, but can become costly as the number of looks or the complexity of the ordering increases. Neither approach repairs a misspecified null model or an incorrectly encoded stopping rule.

Check the following before interpreting a computed p-value:

Reach-look limits: A contribution at look $j$ must exclude every path stopped earlier. Omitting an earlier continuation condition double-counts paths.
Boundary inclusion: For continuous normal statistics, including or excluding an exact boundary point does not change a probability. But the interval on either side of the boundary must be assigned to the correct stopping or continuation region.
Ordering limits: The ordered event must match the stated stagewise ordering. A stopping boundary is not automatically the p-value cutoff; the first worked example illustrates why the two thresholds can differ.
Probability conservation: The sum of probabilities of all disjoint stopping outcomes must be approximately 1. Check this independently of the p-value calculation.
Numerical stability: Integrate over the specified tails and intervals, and tighten tolerances or subdivide regions if results change materially with numerical settings. A plausible-looking result is not evidence that the limits are correct.

The method relies on the canonical normal model and the information levels used to derive its transitions. If the statistic has a different joint null distribution, the conditional densities must be changed accordingly. Further, an ordering that depends on the full stopped path may prevent a one-dimensional forward-density recursion; retaining the relevant history in the integration is necessary. Recursive integration provides a probability calculation, not a justification for the scientific estimand or the choice of ordering.

Wrong: “The final statistic exceeds the cutoff, so its full marginal tail probability is the stagewise p-value.”
Correction: Restrict the final-tail probability to paths that continued through all earlier looks, and add the ordered contributions from earlier stopping looks.
Wrong: “The stopping boundary and the p-value ordering cutoff are the same by definition.”
Correction: The stopping rule determines which look ends observation; the ordering determines which stopped outcomes count as at least as extreme. Encode both conditions.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In a three-look calculation for a look-3 event, which earlier-look restrictions belong in the integration region?
Under the canonical model, if $I_{j-1}/I_j=1/4$, what are the conditional mean and variance of $Z_j$ given $Z_{j-1}=z$?
Why can $P(Z_K\geq c)$ differ from the stagewise contribution at the final look?
A recursion reports that the total probability of stopping over all looks is 0.94. Give one diagnostic interpretation of this result.
For the two-look calculation in the first worked example, what integration limit enforces that the observed final-look path reached look 2?

Solutions. (1) The path must lie in $C_1$ and $C_2$, then in the ordered stopping region at look 3. (2) The conditional mean is $\sqrt{1/4}\,z=z/2$, and the variance is $1-1/4=3/4$. (3) The marginal tail includes paths that may have stopped before the final look; the final-look contribution includes only paths that continued to it and meet the ordering. (4) Probability mass is missing or the numerical error is too large; inspect the region limits, transition densities, and integration accuracy because total stopping probability should be 1. (5) The required restriction is $Z_1<2$, the first-look continuation region in that example.