Worked Example: A Weighted Inverse Normal Test

Purpose and Setup

A two-stage trial can combine evidence from its stages without treating all observations as though they had been analyzed in one prespecified pooled-data test. The weighted inverse normal method does this by converting each stage’s one-sided p-value to a normal score and taking a weighted sum. The weights determine how strongly each stage contributes to the final statistic.

This tutorial works through the calculation and decision for a hypothetical efficacy trial. The test is one-sided: larger stagewise statistics, and therefore smaller stagewise p-values, indicate evidence in the prespecified beneficial direction. The example assumes that stagewise p-values and the weights meet the conditions stated below. These conditions are essential to interpreting the final threshold.

Definition: For two stages with one-sided p-values \(p_1\) and \(p_2\), the weighted inverse normal statistic is \(Z_C=w_1\Phi^{-1}(1-p_1)+w_2\Phi^{-1}(1-p_2)\), where \(\Phi\) is the standard normal cumulative distribution function. The weights are prespecified, nonnegative, and satisfy \(w_1^2+w_2^2=1\). At one-sided significance level \(\alpha\), reject \(H_0\) when \(Z_C>z_{1-\alpha}\), where \(z_{1-\alpha}=\Phi^{-1}(1-\alpha)\).

Conditions for the Final Threshold

The usual exact calibration has a concise set of assumptions. Under \(H_0\), the stagewise p-values must be independent and exactly uniform on \((0,1)\). Exact uniformity is appropriate, for example, for a continuous test statistic with a correctly specified null distribution, when the null parameter is at the value giving the stated null distribution. The stagewise tests must also address the same null hypothesis and use the same prespecified direction of evidence.

The weights must be fixed independently of the observed stagewise p-values, and their squared values must sum to one. Weights based on planned information may be chosen before the trial outcomes are observed. A choice made after examining stagewise evidence is not covered by this calculation merely because the resulting numbers have squared sum one. The independence condition is usually supported by independent stage increments, as established in the earlier material on independent stage increments; overlapping data or shared outcomes can invalidate it.

For a one-sided test, the normal score \(Z_i=\Phi^{-1}(1-p_i)\) is large when stage \(i\) provides evidence in the beneficial direction. If the p-values are independent and exactly uniform under \(H_0\), the scores are independent standard normal random variables. The weighted sum then has a standard normal null distribution. This statement is not automatic for every valid p-value: a continuous but conservative p-value need not be uniform. With independent super-uniform p-values and nonnegative fixed weights, the resulting upper-tail calibration is conservative rather than necessarily exact.

Why the Weighted Statistic Has Its Reference Distribution

The null result follows from transforming the uniform distribution and then using the distribution of a linear combination of independent normal variables. For \(U\sim \mathrm{Uniform}(0,1)\), define \(Z=\Phi^{-1}(1-U)\). For any real \(z\),

\begin{aligned} P(Z\le z) &=P\{\Phi^{-1}(1-U)\le z\}\\ &=P\{U\ge 1-\Phi(z)\}\\ &=\Phi(z). \end{aligned}

Thus \(Z\) is standard normal. Applying this to independent, exactly uniform stagewise p-values gives independent \(Z_1,Z_2\sim N(0,1)\). Their weighted sum is normal, with mean zero and variance

E(Z_C)=0,\qquad \operatorname{Var}(Z_C)=w_1^2+w_2^2=1.

Therefore \(Z_C\sim N(0,1)\) under those assumptions, and \(P(Z_C>z_{1-\alpha}\mid H_0)=\alpha\). If the independent p-values are super-uniform rather than exactly uniform, each transformed score is no larger in the upper-tail stochastic ordering than a standard normal score. With nonnegative weights, their weighted sum is correspondingly conservative for this upper-tail rejection rule. This does not justify claiming exact size \(\alpha\) in that case.

Worked Calculation and Decision

Worked Example: Information-Based Weights and Two Stagewise P-Values

Hypothetical trial. Before enrollment begins, investigators plan for stage 1 to contribute 40% and stage 2 to contribute 60% of the total information. They choose inverse normal weights proportional to the square roots of these planned information fractions. At the end of the trial, the stagewise one-sided p-values for the prespecified beneficial direction are \(p_1=0.120\) and \(p_2=0.0180\). Test at one-sided \(\alpha=0.025\).

Check the weight choice. The square-root information weights are

w_1=\sqrt{0.40}=0.6325,\qquad w_2=\sqrt{0.60}=0.7746, \qquad w_1^2+w_2^2=0.40+0.60=1.

These weights are nonnegative and were set from planned information, not selected after seeing the p-values. For this worked calculation, assume that the two stagewise p-values are independent and exactly uniform under \(H_0\), and that both test the same one-sided hypothesis using valid stagewise null distributions.

Transform the p-values. The normal scores are

Z_1=\Phi^{-1}(1-0.120)=\Phi^{-1}(0.880)\approx1.175, \qquad Z_2=\Phi^{-1}(1-0.0180)=\Phi^{-1}(0.982)\approx2.097.

Combine the scores.

\begin{aligned} Z_C &=(0.6325)(1.175)+(0.7746)(2.097)\\ &\approx0.7432+1.6243\\ &\approx2.368. \end{aligned}

For a one-sided test with \(\alpha=0.025\), \(z_{1-\alpha}=z_{0.975}\approx1.960\). Since \(2.368>1.960\), reject \(H_0\) at the 2.5% level. The corresponding combined one-sided p-value is approximately \(1-\Phi(2.368)=0.00894\). Under the stated calibration assumptions, this is the probability under \(H_0\) of obtaining a combination statistic at least as large as the observed value. It is not the probability that \(H_0\) is true.

The conclusion is evidence against the null in the prespecified beneficial direction. The statistic itself is not an estimated treatment effect: it combines stagewise evidence, not treatment-effect units.

Choosing Weights and Checking the Arithmetic

With two stages, a common prespecified choice uses planned information fractions \(r_1\) and \(r_2\), where \(r_1+r_2=1\), and sets \(w_i=\sqrt{r_i}\). The square roots are used so that the weights, rather than their squares, multiply the normal scores while the combined variance remains one. This choice makes a stage with more planned information contribute more, but it does not make a weight the fraction of the final statistic contributed by that stage: the score also depends on its observed p-value.

A useful numerical check is to calculate the squared-weight sum before using any p-values. Another is to check that each transformation matches the stated direction: for the one-sided beneficial direction here, \(p=0.05\) maps to approximately 1.645, not to a negative score. Reversing the direction while retaining the same transform reverses the meaning of the result.

Worked Example: Confirming the Calculation From Stage Z Statistics

In a second hypothetical trial, planned information fractions are again 0.40 and 0.60, so \(w_1=0.6325\) and \(w_2=0.7746\). Suppose the stagewise one-sided statistics in the same beneficial direction are 1.10 and 1.90, and each stage uses the standard normal reference distribution to calculate its p-value.

The p-values are \(p_1=1-\Phi(1.10)\approx0.1357\) and \(p_2=1-\Phi(1.90)\approx0.02872\). Applying the inverse normal transform gives

\Phi^{-1}(1-p_1)=1.10,\qquad \Phi^{-1}(1-p_2)=1.90.

Thus the combination statistic is

Z_C=(0.6325)(1.10)+(0.7746)(1.90) =0.69575+1.47174 \approx2.167.

Since \(2.167>1.960\), the test rejects at one-sided \(\alpha=0.025\); the combined p-value is about \(1-\Phi(2.167)=0.0151\). This check works because these stagewise p-values were calculated directly from the stated one-sided standard normal statistics. In general, a p-value transform need not recover a reported stage statistic unless the p-value was calculated using the corresponding continuous reference distribution and direction.

Worked Example: A Small Stagewise P-Value Does Not Ensure Rejection

Consider a third hypothetical calculation with equal planned information, giving \(w_1=w_2=\sqrt{0.50}\approx0.7071\). Suppose the one-sided stagewise p-values are \(p_1=0.0400\) and \(p_2=0.200\). Assume again that the stagewise p-values are independent and exactly uniform under \(H_0\).

The normal scores are \(\Phi^{-1}(0.960)\approx1.751\) and \(\Phi^{-1}(0.800)\approx0.8416\). Therefore,

Z_C=(0.7071)(1.751)+(0.7071)(0.8416) \approx1.238+0.5951 =1.833.

The result does not exceed 1.960, so the test does not reject at one-sided \(\alpha=0.025\). The combined p-value is approximately 0.0334. The first stage’s p-value is below 0.05, but that fact alone does not determine the final decision: the rule combines both stages using the prespecified weights and compares the result with the final critical value.

Diagnostics, Alternatives, and Limitations

Before reporting a result, verify the design and computation in a fixed order:

Direction: Confirm that both stagewise p-values test the same beneficial direction and that smaller p-values map to larger normal scores.
Weights: Confirm that the weights were specified independently of observed stagewise evidence, are nonnegative, and satisfy \(w_1^2+w_2^2=1\).
Independence: Check that stagewise evidence is based on independent increments under the null. If stage data overlap or are dependent, the standard normal reference distribution may not apply.
Calibration: Establish whether p-values are exactly uniform under the null or only super-uniform. Do not report exact null size from the standard normal critical value when the required exactness assumptions have not been established.
Numerics: Recompute the transformed scores and weighted sum with adequate precision, then compare the unrounded statistic with the critical value.

The weighted inverse normal test is a combination test: it combines stagewise p-values while preserving a specified final error rate under its calibration conditions. A pooled-data test instead analyzes combined participant-level data under a model for those data; its statistic and assumptions need not match this combination rule. The earlier material on combination tests versus pooled-data tests explains why the two procedures are not interchangeable just because they use information from the same stages.

A further alternative is to use a different combination function, which can allocate sensitivity differently across patterns of stagewise evidence. The inverse normal rule has the practical advantage that its weights express the planned relative contribution of the stages and its final statistic has a direct normal reference distribution under the stated conditions. It does not guarantee the greatest power for every alternative pattern.

The method also has limits. It supplies a final test decision, not by itself a pooled treatment-effect estimate or confidence interval. It depends on valid stagewise p-values and the specified dependence structure; a valid-looking calculation cannot repair invalid stagewise testing. If weights are changed after observing interim evidence, or if the direction is chosen after seeing which direction looks favorable, the stated calibration does not automatically hold. Adaptive changes require a method that accounts for how the adaptation affects the final test.

Common Mistake

Common mistake: “The stagewise p-values are continuous and valid, so their inverse normal transforms must be independent standard normal variables.” Correction: Exact standard normal scores follow when the p-values are independent and exactly uniform under \(H_0\). Continuous conservative p-values can be super-uniform rather than uniform; with appropriate independence and nonnegative fixed weights, the usual upper-tail rule can be conservative, but it is not thereby exactly calibrated. State which condition applies before describing the final p-value or type I error rate.

Key takeaway: Prespecify nonnegative weights with squared sum one, transform stagewise p-values in the common one-sided direction, and compare the weighted score with the correctly calibrated final threshold. The exact standard normal threshold requires independent, exactly uniform stagewise p-values under the null.

Check Your Understanding

Try each question, then compare your reasoning with the solutions.

For information fractions 0.25 and 0.75, calculate the square-root information weights and verify their squared sum.
Under the exact-uniform null assumptions, what is the distribution of each transformed stage score and of the weighted sum when the weights’ squared sum is one?
With one-sided p-values 0.100 and 0.050 and weights \(0.600\) and \(0.800\), calculate the combination statistic and decide whether to reject at \(\alpha=0.025\).
Why does a continuous valid p-value not necessarily yield a standard normal score after the inverse normal transform?
Name two checks that should be made before interpreting a weighted inverse normal test’s final decision.

Solutions. (1) The weights are \(\sqrt{0.25}=0.5000\) and \(\sqrt{0.75}=0.8660\); their squared sum is \(0.25+0.75=1\). (2) With independent, exactly uniform stagewise p-values, each transformed score is \(N(0,1)\); independence and weights with squared sum one make the weighted sum \(N(0,1)\). (3) The scores are \(\Phi^{-1}(0.900)\approx1.282\) and \(\Phi^{-1}(0.950)\approx1.645\). The statistic is \((0.600)(1.282)+(0.800)(1.645)\approx0.7692+1.316=2.085\). Since \(2.085>1.960\), reject at the one-sided 2.5% level, subject to the calibration assumptions. (4) A valid p-value may be conservative and therefore super-uniform, not exactly uniform; its transform is not necessarily standard normal. (5) Appropriate checks include common direction, prespecified weights with squared sum one, independent stage increments, and exact versus conservative p-value calibration.