Applying Inverse-Normal Tests at the Phase Boundary

Combining Evidence at the Phase Boundary

In a seamless phase 2/3 trial, the phase boundary can involve treatment selection, dose selection, or other adaptations. A confirmatory claim may use evidence from both sides of that boundary, but simply pooling all observations and applying an ordinary fixed-design test can fail to account for how the data were used to make the adaptation. An inverse-normal combination test instead combines stage-specific p-values according to a prespecified rule.

This tutorial focuses on a single, directional confirmatory claim: the treatment is beneficial relative to its comparator, in a direction specified before examining the confirmatory evidence. The phase-specific p-values must test the same null hypothesis for that claim. If treatment selection or multiple possible claims affects a stagewise p-value, that p-value must already be valid for the claim after accounting for the selection or multiplicity involved.

Definition: For stagewise p-values $p_1$ and $p_2$, and fixed weights $w_1,w_2$ satisfying $w_1\geq 0$, $w_2\geq 0$, and $w_1^2+w_2^2=1$, the inverse-normal combination statistic is $$ Z_{\mathrm{comb}}=w_1\Phi^{-1}(1-p_1)+w_2\Phi^{-1}(1-p_2), $$ where $\Phi$ is the standard normal cumulative distribution function. Its combined p-value is $p_{\mathrm{comb}}=1-\Phi(Z_{\mathrm{comb}})$. The upper-tail rejection rule at one-sided significance level $\alpha$ is to reject $H_0$ when $Z_{\mathrm{comb}}\geq\Phi^{-1}(1-\alpha)$, equivalently when $p_{\mathrm{comb}}\leq\alpha$.

The function $\Phi^{-1}(1-p_i)$ places each stage’s p-value on a normal-score scale: smaller p-values for benefit produce larger positive scores. The weights determine how much each stage contributes. Their squares, rather than their ordinary sum, must equal one so that a weighted sum of independent standard normal scores has variance one. The weights are fixed by the design; choosing them after seeing the p-values is not covered by this rule.

Conditions for a Confirmatory Interpretation

A p-value is valid under a null hypothesis when, for every $u$ between zero and one, its probability of being at most $u$ is no greater than $u$. An exact continuous p-value has equality; a conservative p-value may have a smaller probability. For this combination test, each stagewise p-value must be valid for the same null claim, and the stagewise evidence must have the independence structure required by the design.

In a seamless trial, this requirement is more specific than merely having separate analyses at two times. The stage 2 p-value must remain valid under the null given the information used at the phase boundary, including the selected treatment or dose and the decision to continue. The stagewise p-values must be independent under the null, or have the appropriate conditional validity that yields the required independent-increments structure. The earlier material on independent increments explains this design feature. If those conditions do not hold, the standard inverse-normal calibration cannot be assumed to control the type I error rate.

The direction of the test also matters. Here, each $p_i$ is oriented so that evidence for benefit gives a small p-value and a positive normal score. A p-value for the opposite direction cannot be inserted unchanged while retaining the same upper-tail rule. In addition, the combination test does not itself fix multiplicity across several treatments, endpoints, or subgroups. The p-values and the overall testing plan must address any such multiplicity before this single-claim rejection rule is used.

Why the Rule Has Its Stated Calibration

When both stagewise p-values are independent and exactly uniform under $H_0$, each transformed score $\Phi^{-1}(1-p_i)$ has a standard normal distribution. Independence and the weight constraint then give a standard normal combined statistic.

\begin{aligned} Z_i&=\Phi^{-1}(1-p_i)\sim N(0,1), \qquad i=1,2,\\ E(Z_{\mathrm{comb}})&=0,\\ \operatorname{Var}(Z_{\mathrm{comb}}) &=w_1^2+w_2^2=1. \end{aligned}

Because a weighted sum of independent normal variables is normal, $Z_{\mathrm{comb}}\sim N(0,1)$ under this exact-null case. Thus, the probability of crossing the upper-tail critical value is $\alpha$. For valid but conservative stagewise p-values, the same test is conservative when the p-values are independent and the weights are nonnegative. To see the role of the signs, validity implies that each transformed score has no more upper-tail probability than a standard normal score. With nonnegative weights, increasing either score cannot decrease the weighted sum; the independent sum is therefore no more likely to exceed a fixed upper-tail threshold than the corresponding sum of independent standard normal scores.

This justification is for the stated conditions, not a blanket guarantee for any two numbers labelled p-values. In particular, dependence introduced by adaptation, invalid selection-adjusted p-values, or data-dependent weights can undermine the calibration. A separate design argument is needed if the trial departs from the conditions above.

Worked Example: Combining Two Independent Stage Results

Suppose a hypothetical seamless trial tests one prespecified beneficial treatment effect. The stagewise p-values are $p_1=0.040$ and $p_2=0.120$, and the prespecified weights are $w_1=w_2=1/\sqrt{2}$. Assume the stagewise p-values are valid for the same one-sided null and independent under that null. Test at $\alpha=0.025$.

Convert the p-values to normal scores: $$ \Phi^{-1}(1-0.040)=\Phi^{-1}(0.960)\approx1.751,\qquad \Phi^{-1}(1-0.120)=\Phi^{-1}(0.880)\approx1.175. $$

Substitute these values into the combination statistic: $$ Z_{\mathrm{comb}}=\frac{1.751+1.175}{\sqrt{2}} =\frac{2.926}{1.414}\approx2.069. $$

The combined p-value is $1-\Phi(2.069)\approx0.0193$. Since $0.0193<0.025$, the trial rejects $H_0$ for this confirmatory claim at the prespecified one-sided level. Under the assumptions of the test, the result is statistically significant evidence in the prespecified direction; it does not mean there is a 1.93% probability that $H_0$ is true.

Worked Example: A Combined Result That Does Not Cross the Boundary

In a second hypothetical trial, let $p_1=0.080$ and $p_2=0.100$, with the same equal weights and $\alpha=0.025$. Assume again that both p-values are valid, independent under the null, and directed toward the same benefit claim.

The normal scores are $$ \Phi^{-1}(0.920)\approx1.405,\qquad \Phi^{-1}(0.900)\approx1.282. $$ Thus, $$ Z_{\mathrm{comb}}=\frac{1.405+1.282}{\sqrt{2}} =\frac{2.687}{1.414}\approx1.900. $$

The combined p-value is $1-\Phi(1.900)\approx0.0287$. The critical value is $\Phi^{-1}(0.975)\approx1.960$, so $1.900<1.960$, and $0.0287>0.025$. The trial does not reject $H_0$ at the specified level. This conclusion is not evidence that the null is true; it means this test did not provide sufficient evidence to cross its prespecified rejection boundary.

Applying the Test After Selection

Selection at the phase boundary does not automatically invalidate a combination test, but it makes the construction of the stagewise p-values important. The p-value from the selection stage must account for the fact that the treatment or dose was chosen using data, if that selection affects its null distribution. The later p-value must also be valid for the selected option under the null, conditional on the information and decision available at the transition. These are properties of the stagewise p-values and design—not corrections supplied by the inverse-normal formula itself.

Worked Example: Combining Selection-Aware Stagewise Evidence

Consider a hypothetical seamless trial that selects one dose at the boundary. For illustration, suppose the selection-aware stage 1 p-value for the selected-dose claim is $p_1=0.060$, and the stage 2 p-value for that same claim is $p_2=0.040$. Assume these p-values satisfy the design’s validity and independence conditions, and use equal weights at one-sided $\alpha=0.025$.

The normal scores are $$ \Phi^{-1}(0.940)\approx1.555,\qquad \Phi^{-1}(0.960)\approx1.751. $$ The combination statistic is $$ Z_{\mathrm{comb}}=\frac{1.555+1.751}{\sqrt{2}} =\frac{3.306}{1.414}\approx2.337. $$

Therefore $p_{\mathrm{comb}}=1-\Phi(2.337)\approx0.0097$. Because this is below $0.025$, the result crosses the prespecified boundary for the selected-dose claim, under the stated assumptions. If $p_1$ were instead an unadjusted p-value made small by selecting the most favorable of several doses, the calculation alone would not justify this confirmatory conclusion.

Diagnostics, Comparisons, and Limitations

Before applying the rule, check that both p-values address the same estimand and directional hypothesis, that any adaptation or selection has been handled in the stagewise testing procedures, and that the independence or conditional-validity argument matches the actual trial design. Confirm that the weights were fixed in advance, are nonnegative, and have squared values summing to one. Finally, verify the tail and arithmetic: for the benefit-oriented test here, smaller p-values give larger scores, and rejection occurs in the upper tail.

The inverse-normal approach differs from two tempting alternatives. A final-stage-only analysis discards stage 1 evidence from the confirmatory test, which may be appropriate for some designs but does not use both stages. Naive pooling treats all observations as though the final comparison had been specified independently of the phase-boundary decisions; after selection or adaptation, its ordinary reference distribution may not be valid. The combination test uses stagewise evidence while requiring valid p-values and a design that supports their combination.

The method also has limits. It does not repair a poorly defined estimand, remove bias in the measurements, or guarantee clinical importance. Statistical significance should be considered alongside the estimated treatment effect, its uncertainty, safety findings, and the trial’s clinical objectives. If a stagewise p-value is exactly zero or one, its normal-score transformation is infinite; in practice, the p-value calculation and software convention must handle such boundary cases explicitly rather than silently treating an infinite score as an ordinary finite value.

Common Mistake: Ignoring the Sign of a Weight

A frequent error is to assume that $w_1^2+w_2^2=1$ alone is enough to ensure validity. That condition standardizes the variance for independent standard normal scores, but it does not control the direction in which a conservative p-value contributes. The stated conservative-p-value result also requires nonnegative weights.

For example, let $U$ be uniform on $(0,1)$, set $p_1=\sqrt{U}$, and use $w_1=-1,w_2=0$. The first p-value is conservative because $P(p_1\leq u)=u^2\leq u$. Yet at one-sided $\alpha=0.025$, the statistic rejects when $p_1>0.975$, which occurs with probability $$ P(\sqrt{U}>0.975)=P(U>0.975^2)=1-0.950625=0.049375. $$ This exceeds $0.025$. The correction is to use nonnegative weights for the inverse-normal rejection rule described here; weights for stages intended to contribute are typically strictly positive.

Check Your Understanding

For each question, first identify the relevant condition or calculation, then compare with the solutions.

With equal weights, $p_1=0.020$ and $p_2=0.200$. Give the formula for the combined statistic and state whether the test rejects at one-sided $\alpha=0.025$.
Why must a stage 2 p-value remain valid after conditioning on the treatment selected at the phase boundary?
Suppose two independent, exact-null stagewise p-values are uniform. What is the null distribution of the inverse-normal statistic when the weights’ squares sum to one?
A researcher uses $w_1=-0.6$ and $w_2=0.8$. Although the squared weights sum to one, why does this not meet the conditions for the stated conservative-p-value guarantee?

Solutions. (1) The scores are $\Phi^{-1}(0.980)\approx2.054$ and $\Phi^{-1}(0.800)\approx0.842$. Thus $Z_{\mathrm{comb}}=(2.054+0.842)/\sqrt{2}\approx2.048$, giving $p_{\mathrm{comb}}\approx0.0203$. Since this is below $0.025$, reject $H_0$, provided the stagewise assumptions hold. (2) The selection decision uses earlier information; without conditional validity, the stage 2 p-value may no longer have its stated null behavior for the selected claim. (3) Each transformed score is standard normal, and the independent weighted sum is standard normal because the squared weights sum to one. (4) The first weight is negative. A conservative p-value can then contribute in the rejection direction when it is large, so the nonnegative-weight validity argument does not apply.