The Weighted Inverse Normal Combination Statistic

Why Weight Stagewise Evidence?

A combination test uses evidence from separate stages of a trial without treating the stagewise results as though they came from one unplanned analysis. The inverse normal method first expresses each stage's p-value as a normal score, then combines those scores. Weighting lets prespecified stage contributions differ: a stage with a larger weight has more influence on the combined statistic.

The calculation is simple, but its interpretation depends on details that must be fixed and checked: which tail the stagewise p-values use, how the weights are normalized, and whether the stagewise evidence has the independence and null calibration needed for a standard normal reference distribution. This tutorial uses one-sided p-values for a common planned alternative: smaller p-values indicate stronger evidence in that direction.

Definition: For $K\geq 2$ stages, let $p_i$ be the stagewise one-sided p-value at stage $i$, with small values supporting the same planned alternative at each stage. Let $w_i$ be prespecified, nonnegative weights, not all zero. Define the stagewise normal score $Z_i=\Phi^{-1}(1-p_i)$, where $\Phi$ is the standard normal cumulative distribution function. The weighted inverse normal combination statistic is $$ Z_{\mathrm{comb}}= \frac{\displaystyle\sum_{i=1}^{K} w_i Z_i} {\displaystyle\sqrt{\sum_{i=1}^{K}w_i^2}} = \frac{\displaystyle\sum_{i=1}^{K}w_i\Phi^{-1}(1-p_i)} {\displaystyle\sqrt{\sum_{i=1}^{K}w_i^2}}. $$

The denominator is the square root of the sum of squared weights. If the weights have already been normalized so that $\sum_{i=1}^{K}w_i^2=1$, the denominator is one and the statistic is just the weighted sum. If raw weights are supplied instead, the denominator standardizes that sum. Multiplying every weight by the same positive constant leaves $Z_{\mathrm{comb}}$ unchanged.

Null Distribution and Its Justification

The standard normal reference distribution requires more than the presence of p-values in the formula. Under the global null, assume each stagewise p-value is continuous and uniform on $[0,1]$, and that the stagewise p-values are mutually independent. In adaptive trials, independence is commonly supported by using separate stage increments and valid stagewise tests. The planned alternative and p-value tail convention must also agree across stages.

The normal-score transformation maps a continuous uniform p-value to a standard normal variable. To see this, let $U\sim\mathrm{Uniform}(0,1)$ and set $Z=\Phi^{-1}(1-U)$. For any real $z$,

P(Z\leq z) =P\bigl(U\geq 1-\Phi(z)\bigr) =\Phi(z).

Thus each $Z_i$ is standard normal under the null. Independence of the p-values implies independence of their normal scores. A weighted sum of independent normal variables is normal; its mean is zero and its variance is the sum of the squared weights. Dividing by the square root of that variance gives

\sum_{i=1}^{K}w_iZ_i \sim N\left(0,\sum_{i=1}^{K}w_i^2\right), \qquad Z_{\mathrm{comb}}\sim N(0,1).

This argument explains why the weights are squared in the normalization. It also shows where the reference distribution can fail: if stagewise p-values are dependent, the variance of the weighted sum generally includes covariance terms, so dividing only by $\sqrt{\sum_i w_i^2}$ need not produce a standard normal statistic.

Worked Example: Combining Two Stagewise p-Values

Suppose a hypothetical two-stage trial has one-sided stagewise p-values $p_1=0.0800$ and $p_2=0.0150$. Before the trial, its protocol specifies weights $w_1=0.6$ and $w_2=0.8$. Assume the p-values refer to the same planned alternative and are independent and continuously uniform under the global null.

First calculate the normal scores using the upper-tail convention in the definition: $$ Z_1=\Phi^{-1}(1-0.0800)=\Phi^{-1}(0.9200)\approx1.4051, \qquad Z_2=\Phi^{-1}(1-0.0150)=\Phi^{-1}(0.9850)\approx2.1701. $$

The weights already have squared sum $0.6^2+0.8^2=1$. Substituting the scores and weights gives $$ Z_{\mathrm{comb}} =\frac{0.6(1.4051)+0.8(2.1701)}{\sqrt{0.6^2+0.8^2}} \approx 2.5791. $$

The combined statistic is therefore approximately 2.579 standard normal units in the direction of the planned alternative. If expressed as a one-sided combined p-value under the stated null assumptions, it is $1-\Phi(2.5791)\approx0.00495$. That p-value summarizes the combined evidence under the null; it is not the probability that the null hypothesis is true.

Calculation With More Than Two Stages

For $K$ stages, the procedure has the same form: transform each stagewise p-value separately, multiply each score by its corresponding prespecified weight, add those weighted scores, and divide by the square-root normalization. Keep the index pairing intact: $w_i$ multiplies the score from stage $i$, not another stage's score.

Worked Example: A Three-Stage Calculation With Raw Weights

Consider hypothetical one-sided stagewise p-values $p_1=0.2000$, $p_2=0.1000$, and $p_3=0.4000$, with prespecified raw weights $1,2,2$. Assume the stagewise p-values are independent and continuous under the null and use the same planned alternative.

The three normal scores are $$ Z_1=\Phi^{-1}(0.8000)\approx0.8416,\qquad Z_2=\Phi^{-1}(0.9000)\approx1.2816,\qquad Z_3=\Phi^{-1}(0.6000)\approx0.2533. $$

The squared weights sum to $1^2+2^2+2^2=9$, so the normalization is $\sqrt{9}=3$. Therefore, $$ Z_{\mathrm{comb}} =\frac{1(0.8416)+2(1.2816)+2(0.2533)}{3} =\frac{3.9114}{3} \approx1.3038. $$

The combined score is positive but modest. Its one-sided standard normal tail area is about $0.0962$. This is the tail area associated with the combined statistic under the assumptions above; whether it meets a trial's prespecified decision rule depends on that rule, not on the calculation alone.

Weights, Normalization, and Interpretation

Weights describe relative influence, not probabilities and not sample sizes by themselves. The statistic uses them linearly in the numerator and accounts for their overall magnitude in the denominator. For example, raw weights $(2,2)$ yield the same statistic as normalized weights $(1/\sqrt{2},1/\sqrt{2})$, because both sets have the same relative weighting.

Worked Example: Checking Scale Invariance

Suppose a hypothetical trial has two stagewise p-values $p_1=0.5000$ and $p_2=0.0010$, with equal raw weights $w_1=w_2=2$. Assume the one-sided p-value convention is consistent across stages and the null assumptions for the inverse normal reference distribution hold.

The scores are $Z_1=\Phi^{-1}(0.5000)=0$ and $Z_2=\Phi^{-1}(0.9990)\approx3.0902$. Using the raw weights, $$ Z_{\mathrm{comb}} =\frac{2(0)+2(3.0902)}{\sqrt{2^2+2^2}} =\frac{6.1804}{\sqrt{8}} \approx2.1851. $$

Rescale the weights to $1,1$. The result is $$ \frac{1(0)+1(3.0902)}{\sqrt{1^2+1^2}} =\frac{3.0902}{\sqrt{2}} \approx2.1851. $$ The statistic is unchanged because both weights were divided by the same positive constant. The example also illustrates that one stage with a strong score can substantially influence the combined statistic, while the weight structure governs its relative contribution.

Assumptions, Diagnostics, and Common Errors

Before calculating or interpreting a weighted inverse normal statistic, verify the ingredients that determine its meaning. In particular, check that the weights were fixed in advance of observing the stagewise results, that every p-value uses the intended tail and alternative, and that the stagewise p-values satisfy the null calibration and independence conditions used for the standard normal reference.

Check the p-value convention. The formula $Z_i=\Phi^{-1}(1-p_i)$ assigns a large positive score to a small upper-tail p-value. Feeding in lower-tail p-values without converting them consistently can reverse the score's meaning.
Check stage-to-weight matching. Write down the ordered pairs $(p_i,w_i)$ before summing. A swapped weight can change the result even if the same values appear in the calculation.
Check the normalization. For raw weights, calculate $\sqrt{\sum_i w_i^2}$, not $\sum_i w_i$. The latter generally does not standardize the sum to unit variance.
Check independence and calibration. Reusing data across stages or using a stagewise test whose null p-value is not uniform can invalidate the stated standard normal reference. A numerical score can still be computed, but its usual null interpretation is no longer justified by this derivation.
Check when weights were chosen. Choosing weights after seeing which stages had favorable p-values makes the weights data-dependent. The fixed-weight null argument above does not automatically account for that additional selection.

A frequent mistake is to say, “The weights sum to one, so no further normalization is needed.” The relevant condition is instead $\sum_i w_i^2=1$. For instance, weights $0.5,0.5$ sum to one but have squared sum $0.5$; the denominator must be $\sqrt{0.5}$. The correction is to use the displayed definition directly, or deliberately normalize the weights to have squared sum one before using the simplified formula.

How This Statistic Differs From Other Analyses

The weighted inverse normal method is one way to combine stagewise evidence. It is not automatically the same as analyzing all patient-level data together, because the calculation operates on stagewise p-values rather than on a pooled-data test statistic. Nor does every inverse normal test give stages equal influence: the weights specify that balance in advance.

Approach	What is combined	Key distinction
Weighted inverse normal	Normal scores transformed from stagewise p-values	Prespecified weights set relative stage contributions; squared-weight normalization gives the standard null scale under the stated assumptions.
Equal-weight inverse normal	The same type of stagewise normal scores	All stages receive equal relative weights; this is a special weighting choice, not a different transformation.
Pooled-data analysis	Patient-level data across stages	It uses a pooled analysis model and its assumptions; it is not generally interchangeable with combining stagewise p-values.

The combination statistic does not, by itself, describe the treatment effect in clinical units, such as a mean difference or hazard ratio. It summarizes statistical evidence. Effect estimation and reporting therefore require appropriate analyses and interpretation beyond the value of $Z_{\mathrm{comb}}$.

Limitations and Practical Use

The standard normal result is exact under independent, continuous uniform stagewise p-values and fixed weights. In other settings, including dependent stagewise results or p-values that are conservative rather than uniform, the reference distribution may differ. The formula remains a defined numerical combination, but a valid decision rule then requires justification appropriate to the actual design and p-value behavior.

The calculation also depends on a common direction of evidence: the transformation shown here interprets smaller one-sided p-values as stronger evidence for the planned alternative. It should not be applied mechanically when stages use incompatible hypotheses or tail conventions. Finally, although weights can express a planned emphasis across stages, this tutorial defines and computes the statistic; the rationale for selecting a particular weight scheme is a separate design question.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

For two stages with p-values $0.1000$ and $0.0500$, and raw weights $3$ and $4$, calculate the weighted inverse normal statistic.
With two equal weights and stagewise p-values $0.2500$ and $0.2500$, what is the combined statistic? Show why its sign is positive.
Why does the standard normal null distribution require independence of the stagewise p-values in the derivation given here?
A researcher says weights $(0.5,0.5)$ need no denominator because they sum to one. Identify the error and give the correct denominator.

Solutions. (1) The scores are $\Phi^{-1}(0.9000)\approx1.2816$ and $\Phi^{-1}(0.9500)\approx1.6449$. Thus, $$ Z_{\mathrm{comb}} =\frac{3\Phi^{-1}(0.9000)+4\Phi^{-1}(0.9500)}{\sqrt{3^2+4^2}} =\frac{10.4241}{5} \approx2.0848. $$ (2) Each score is $\Phi^{-1}(0.7500)\approx0.6745$. With equal weights, the statistic is $(0.6745+0.6745)/\sqrt{2}\approx0.9539$, which is positive because each p-value is below $0.5$, giving a positive score. (3) Independence makes the variance of the weighted sum equal to the sum of the individual weighted variances, $\sum_i w_i^2$, without covariance terms; it also supports the stated normal-sum argument. (4) The squared weights sum to $0.5$, not one. The denominator is $\sqrt{0.5}$, so the statistic is $(0.5Z_1+0.5Z_2)/\sqrt{0.5}$.