Why a Sequential Intersection P-Value Needs the Whole Path
In a multi-arm, multi-stage trial, an intersection hypothesis asserts that several component null hypotheses are all true. The trial may examine the data at several stages, and a decision to reject that intersection may follow a boundary crossing at any one of them. A p-value for the intersection must therefore reflect the whole sequential decision rule, not just the statistic at the stage where the trial stopped.
For a fixed testing rule, its p-value summarizes the strength of the observed evidence against the null. For a sequential rule, the relevant evidence includes which looks were planned, how the statistics at different looks are related, and which paths would have led to rejection. The probability calculation must use that complete boundary-crossing distribution.
The regularity condition matters. Nesting alone does not ensure that a path whose infimum level is \(u\) belongs to \(R_I(u)\). Without that implication, the event \(\{p_I\leq u\}\) need not be contained in \(R_I(u)\), and the usual validity argument does not follow.
Validity and Boundary-Crossing Probabilities
Under the definition above, validity follows directly. For any parameter value \(\theta\) satisfying \(H_I\), and any \(u\in[0,1]\), regularity gives \(\{p_I(D)\leq u\}\subseteq\{D\in R_I(u)\}\). The test calibration then bounds the probability of the latter event.
Thus \(p_I\) is a valid p-value: under the intersection null, its probability of being at most any level \(u\) is no greater than \(u\). If the null distribution is continuous and the boundary is calibrated exactly, equality may hold. With discrete data or conservative boundaries, the inequality can be strict.
In a common formulation, the rejection rule is that a path statistic \(M_I(D)\) crosses a level-specific boundary \(c_I(u)\). If the boundary-crossing probability is calibrated under \(H_I\), then the p-value is the null probability of a crossing at least as extreme as the observed one. For a continuous scalar statistic with null cumulative distribution function \(F_I\), this becomes \(1-F_I(M_I^{\mathrm{obs}})\). With repeated looks, \(F_I\) is the distribution of the path statistic, such as the maximum over the planned looks; it is generally not the distribution of a single-stage statistic.
Worked Example: Two Correlated Looks Under a Single Intersection Null
Suppose a one-sided test of \(H_I\) uses standardized statistics \(Z_1\) and \(Z_2\) at two planned looks. Under \(H_I\), assume \((Z_1,Z_2)\) has a bivariate standard normal distribution with correlation \(0.5\). The design rejects if either statistic exceeds a common boundary \(c\). For the observed path \(z_1=z_2=0\), calculate the sequential p-value.
Assumptions and statistic. The joint null distribution is specified as bivariate normal, and both looks are included in the rejection rule. The path statistic is \(M=\max(Z_1,Z_2)\), so the observed value is \(M^{\mathrm{obs}}=0\). The p-value is the probability, under \(H_I\), that the maximum is at least zero.
For a bivariate standard normal pair with correlation \(\rho\), the probability that both variables are negative is \(1/4+\arcsin(\rho)/(2\pi)\). Here \(\rho=0.5\), so \(\arcsin(0.5)=\pi/6\). Therefore,
The p-value is approximately 0.6667. The two looks are not independent, so multiplying two single-look probabilities as if they were independent would be incorrect. In this example the observed statistics are not unusually large under the intersection null.
The calculation above also illustrates why a sequential p-value depends on the design. A different number of looks, correlation structure, or boundary rule gives a different null probability of crossing. Those features must be specified before the p-value can be computed.
Independent Stagewise Scores: A Direct Calculation
When the design specifies independent stagewise evidence scores, the path probability can be calculated by multiplying their non-crossing probabilities. This is a particular model, not a default assumption for cumulative group-sequential statistics. In a standard cumulative-data design, successive standardized statistics are usually correlated, so the joint distribution must be used instead.
Worked Example: Repeated-Look Adjustment With Independent Scores
Consider a hypothetical one-sided test of \(H_I\) with two planned looks. Suppose its stagewise scores \(Y_1,Y_2\) are independent standard normal variables under \(H_I\), and the design rejects if either exceeds a common boundary. The observed scores are \(y_1=2.20\) and \(y_2=1.80\). Find the sequential p-value and decide whether the path rejects at level \(0.05\).
Assumptions. Under \(H_I\), each score is standard normal and the two scores are independent. The rejection region at boundary \(c\) is \(\{Y_1\geq c\}\cup\{Y_2\geq c\}\). The observed maximum is \(2.20\). By independence, the probability that neither score reaches that value is \(\Phi(2.20)\Phi(1.80)\), where \(\Phi\) is the standard normal cumulative distribution function.
The sequential p-value is approximately 0.04933, so this path rejects \(H_I\) at level \(0.05\). The comparison is with the p-value for the union of the two stagewise crossing events, not with a p-value calculated from only one selected look.
As a calibration check, a common boundary \(c\) giving overall rejection probability \(0.05\) satisfies \(1-\Phi(c)^2=0.05\), hence \(\Phi(c)=\sqrt{0.95}\) and \(c=\Phi^{-1}(\sqrt{0.95})\approx1.9545\). The observed first score, 2.20, crosses this boundary. The calculation confirms the decision from the p-value.
Intersection Tests Across Arms and Looks
An intersection test may involve several treatment-versus-control contrasts as well as several looks. Its rejection region could, for example, reject \(H_I\) when any one of the included statistics crosses its specified boundary. The null distribution must then describe the joint path across both arms and stages. Shared controls can induce dependence between arm statistics, and repeated observations can induce dependence across looks. Ignoring either dependence can misstate the crossing probability.
The following example uses independence to make the joint calculation transparent. It is not a general prescription for multi-arm trials: the dependence assumptions must match the actual design.
Worked Example: Two Arms and Two Looks Under an Intersection Null
Suppose \(H_I=H_1\cap H_2\), where \(H_1\) and \(H_2\) are one-sided null hypotheses for two treatment arms. The local intersection test rejects if any of the four arm-by-look scores exceeds a common threshold. Assume all four scores are independent standard normal under \(H_I\), and suppose the observed scores are all 2.00. Calculate the p-value.
Assumptions and event. The four scores have the specified joint independent standard normal distribution under the intersection null. The observed maximum is 2.00, so an equally or more extreme boundary crossing occurs if at least one score is at least 2.00. Independence makes the probability of no such crossing the product of the four individual non-crossing probabilities.
The p-value is approximately 0.08794, so this local test does not reject \(H_I\) at level \(0.05\), despite each individual score being 2.00. The four opportunities to cross the boundary are reflected in the probability calculation. If the scores were dependent, the product formula would no longer apply; the joint distribution of all four scores would be needed.
Calculation Workflow and Diagnostics
A reliable calculation begins with the rejection rule, not with a preferred p-value formula. The rule determines which paths count as crossings, and the null model determines how often those paths occur. The following sequence makes the essential checks explicit.
List the component null hypotheses in \(H_I\), the planned looks, and which observations or statistics are available at each look. Include the rule that determines when testing stops.
State whether rejection follows a crossing at any look, a particular combination of stagewise evidence, or another prespecified sequential rule. Do not substitute a single-look event for the actual rule.
Specify the distribution across looks and arms, including correlations. Check that its assumptions are justified by the design and analysis model.
Find the smallest level whose rejection region includes the observed path, using a regular boundary family. Equivalently, calculate the appropriate null boundary-crossing probability.
Verify that a more extreme path cannot have a larger p-value under the chosen ordering, and confirm that the p-value agrees with the level-specific boundary decision.
Warning signs include using a final-look tail probability when earlier looks could already have caused rejection; treating correlated looks or shared-control contrasts as independent; and reporting an infimum-based p-value without ensuring that the rejection regions have the required regularity. A further diagnostic is to compare the calculated p-value with the design’s decision at a chosen level: for a properly inverted rejection family, the path should be in \(R_I(u)\) exactly when its p-value is at most \(u\).
What This P-Value Does—and Does Not—Adjust
A stagewise p-value for \(H_I\) accounts for the repeated opportunities to reject that particular intersection under its calibrated testing rule. It is not automatically an adjusted p-value for an individual treatment hypothesis, nor does it by itself establish familywise error control across every hypothesis in a closed-testing procedure. Those conclusions require the corresponding collection of tests and error-control argument. Here the target is the validity and calculation of the p-value for the specified intersection test.
The most direct alternative is to report a stage-specific p-value, but that answers a different question if the design permitted rejection at earlier looks. A conservative alternative is to allocate portions of the error level across looks and apply a separate bound at each stage. That can be simpler, but may be less efficient than calibrating the joint boundary-crossing distribution. When the joint distribution is known and the boundary is calibrated to it, the crossing probability uses the dependence structure rather than discarding it.
Worked Example: Why a Single-Look Tail Probability Can Mislead
In a hypothetical two-look design, suppose independent standard normal stagewise scores are used and rejection occurs if either score exceeds the common boundary calibrated to overall level \(0.05\). The boundary is approximately \(1.9545\), as calculated above. Consider an observed path with scores \(1.90\) and \(1.90\).
The path does not cross the overall boundary at either look, so the level-\(0.05\) sequential test does not reject. Its sequential p-value is \(1-\Phi(1.90)^2\). Since \(\Phi(1.90)\approx0.97128\),
The p-value exceeds \(0.05\), in agreement with the boundary decision. A single-look upper-tail probability would instead be \(1-\Phi(1.90)\approx0.02872\), which is below \(0.05\). That value does not account for the specified two-look rejection rule and would give the wrong decision if presented as the sequential p-value.
Common Pitfalls and Limitations
- Ignoring the stopping opportunities. A final-look tail probability generally does not describe a test that could reject earlier. Calculate the probability of the full sequential rejection event.
- Assuming independence for convenience. Independence must be a property of the specified stagewise statistics, not an assumption made to simplify arithmetic. Use the actual joint null distribution when looks or arms are correlated.
- Relying on nesting alone. The infimum definition needs a regularity condition linking \(p_I\leq u\) to membership in \(R_I(u)\). Without it, validity is not established by the simple event-inclusion argument.
- Confusing a local intersection p-value with an arm-adjusted p-value. A p-value for \(H_I\) pertains to that intersection test. It does not automatically provide every arm’s multiplicity-adjusted conclusion.
- Overlooking model misspecification. A correctly executed probability calculation can still be unreliable if the assumed null distribution, covariance structure, information times, or stopping rule does not match the design.
Check Your Understanding
Try each question, then compare your reasoning with the solutions.
- Why does nesting of rejection regions alone not justify the usual validity proof for an infimum-defined p-value?
- A one-sided test has two independent standard normal stagewise scores. At an observed maximum of 2.00, express the sequential p-value in terms of \(\Phi\). Is it a single-look tail probability?
- For a two-look design with correlated statistics, what distributional information is needed to calculate a boundary-crossing p-value?
- At level \(0.05\), a sequential p-value is 0.04933. State the decision and interpret the p-value without assigning a probability to the null hypothesis.
Solutions. (1) The infimum may fail to be attained: \(p_I\leq u\) need not imply \(D\in R_I(u)\). A regularity condition such as right-continuity with suitably defined boundaries is needed for that event inclusion. (2) The answer is \(1-\Phi(2.00)^2\), because neither of the two independent scores reaches 2.00 with probability \(\Phi(2.00)^2\). It is a two-look crossing probability, not a single-look tail probability. (3) The joint null distribution of the two statistics, including their correlation and the exact stopping and crossing rule, is required. (4) Reject \(H_I\) at level \(0.05\). Under \(H_I\), the probability of obtaining a path at least as extreme under the specified sequential rule is 0.04933; this is not the probability that \(H_I\) is true.