Deriving the Inverse-Normal Reference Distribution

Why the Reference Distribution Matters

An inverse-normal combination test turns stagewise p-values into normal scores, combines those scores, and compares the result with a standard normal reference distribution. That final comparison is justified only when the combined statistic really has a standard normal null distribution. The justification depends on both the stage scores and the weights: the scores must be independent standard normal random variables under the null hypothesis, and the fixed weights must have squared values summing to one.

The important point is that a weighted sum does not retain unit variance merely because its inputs have unit variance. Each weight contributes its square to the variance. A weight vector with the wrong squared length therefore changes the reference distribution and can change the test’s Type I error rate.

Stage Scores and the Required Assumptions

Definition: Let $Z_1,\ldots,Z_K$ be the stage scores for $K$ stages. Under the null hypothesis $H_0$, suppose they are mutually independent and each has distribution $N(0,1)$. For fixed, nonrandom weights $w_1,\ldots,w_K$, define the combined score $$ T=\sum_{i=1}^{K}w_i Z_i. $$ If $\sum_{i=1}^{K}w_i^2=1$, then $T\sim N(0,1)$ under $H_0$.

Here, “fixed” means the weights are determined without using the observed stage scores. This tutorial establishes the reference distribution under the stated assumptions; it does not establish that a particular adaptive design preserves those assumptions. Independence is an assumption about the joint distribution of the stage scores, not a consequence of each score separately having a standard normal distribution.

In an inverse-normal combination test, a one-sided stage p-value $p_i$ is transformed to $Z_i=\Phi^{-1}(1-p_i)$, where $\Phi$ is the standard normal cumulative distribution function. If, under $H_0$, each $p_i$ is exactly uniform on $(0,1)$, this transformation gives a standard normal score. Indeed, for any real $z$, $$ P(Z_i\le z)=P\bigl(\Phi^{-1}(1-p_i)\le z\bigr) =P\bigl(p_i\ge 1-\Phi(z)\bigr)=\Phi(z). $$ The last equality uses the uniform distribution of $p_i$. For the combined statistic to have the reference distribution in the definition, the stage scores must also be independent under $H_0$. Discrete or conservative p-values need not yield exactly standard normal scores, and dependence between stages can change the distribution of the sum.

For a one-sided test in which larger scores indicate evidence against $H_0$, the combined p-value is $1-\Phi(T)$ when the assumptions hold. For a level-$\alpha$ test, rejection occurs when $T>z_{1-\alpha}$, where $z_{1-\alpha}$ is the $1-\alpha$ quantile of the standard normal distribution.

Deriving the Distribution

There are two connected facts: an independent linear combination of normal random variables is normal, and its variance is determined by the squared weights. The following derivation makes both parts explicit.

Write the moment-generating function of each weighted score.
For $Z_i\sim N(0,1)$, the moment-generating function of $w_iZ_i$, evaluated at $t$, is $E[\exp(tw_iZ_i)]=\exp(t^2w_i^2/2)$.

Use independence to combine the functions.
Because the scores are independent, the moment-generating function of their sum is the product of their moment-generating functions.

Identify the resulting normal distribution.
The product equals $\exp\{t^2\sum_i w_i^2/2\}$, the moment-generating function of a normal variable with mean zero and variance $\sum_i w_i^2$.

\begin{aligned} M_T(t) &=E\left[\exp\left(t\sum_{i=1}^{K}w_iZ_i\right)\right]\\ &=\prod_{i=1}^{K}E\left[\exp(tw_iZ_i)\right]\\ &=\prod_{i=1}^{K}\exp\left(\frac{t^2w_i^2}{2}\right)\\ &=\exp\left(\frac{t^2}{2}\sum_{i=1}^{K}w_i^2\right). \end{aligned}

The moment-generating function of $N(0,v)$ is $\exp(vt^2/2)$. Thus the derivation gives the more general result $T\sim N(0,\sum_i w_i^2)$, not automatically $N(0,1)$. The squared-weight condition makes that variance equal to one, proving the stated reference distribution. Equivalently, because $E(Z_i)=0$, $Var(Z_i)=1$, and the scores are independent, $E(T)=0$ and $Var(T)=\sum_iw_i^2$; the moment-generating-function argument additionally establishes that $T$ is normal.

Worked Example: Checking the Reference Distribution

Suppose a hypothetical two-stage test has independent stage scores $Z_1,Z_2\sim N(0,1)$ under $H_0$. Its fixed weights are $w_1=0.6$ and $w_2=0.8$. Verify the reference distribution and calculate the one-sided p-value if the observed scores are $Z_1=0.5$ and $Z_2=1.5$.

First check the assumptions: the example stipulates that both scores are standard normal under $H_0$, that they are independent, and that the weights are fixed. Their squared sum is $$ 0.6^2+0.8^2=0.36+0.64=1. $$ Therefore the combined statistic has distribution $N(0,1)$ under $H_0$.

The observed combined score is $$ T=0.6(0.5)+0.8(1.5)=0.3+1.2=1.5. $$ Its one-sided p-value is $1-\Phi(1.5)\approx 0.0668$. At the one-sided $0.05$ level, the critical value is approximately $1.6449$. Since $1.5<1.6449$, this result does not meet the rejection criterion. The p-value is the probability, under $H_0$ and the stated reference model, of a combined score at least as large as $1.5$; it is not the probability that $H_0$ is true.

What Goes Wrong When the Weights Are Not Normalized

If the weights are fixed and the stage scores are independent standard normal variables, the sum remains normal even when the squared weights do not add to one. Its variance is then $\sum_i w_i^2$. Using a standard normal cutoff without adjusting for that variance gives the wrong null rejection probability.

Worked Example: An Incorrectly Scaled Test

Consider hypothetical independent scores $Z_1,Z_2\sim N(0,1)$ under $H_0$, with fixed weights $w_1=w_2=\sqrt{0.75}$. Suppose someone uses $T=w_1Z_1+w_2Z_2$ but applies the usual one-sided $0.05$ standard normal cutoff, $1.6449$.

The squared weights sum to $0.75+0.75=1.5$, so the derivation gives $T\sim N(0,1.5)$, not $N(0,1)$. The actual null rejection probability from using the cutoff $1.6449$ is $$ P(T>1.6449\mid H_0) =1-\Phi\left(\frac{1.6449}{\sqrt{1.5}}\right) \approx 1-\Phi(1.3431) \approx 0.0896. $$ Thus the test rejects about 8.96% of the time under this null model, rather than 5% of the time.

To obtain a standard normal statistic, divide the original sum by its standard deviation $\sqrt{1.5}$. The resulting normalized weights are $w_1/\sqrt{1.5}=w_2/\sqrt{1.5}=\sqrt{0.5}$, whose squared values sum to one. Alternatively, retaining the original statistic requires comparing it with $\sqrt{1.5}\,z_{1-\alpha}$, not $z_{1-\alpha}$.

Independence Is Part of the Result

Standard normal marginal distributions alone are not enough. If two scores are jointly normal but correlated, a weighted sum is still normal, but its variance includes a covariance term. For two scores with correlation $\rho$, $$ Var(w_1Z_1+w_2Z_2)=w_1^2+w_2^2+2w_1w_2\rho. $$ Consequently, weights satisfying $w_1^2+w_2^2=1$ do not generally give a unit-variance sum when $\rho\ne 0$. For more stages, the variance includes all pairwise covariance terms. The independent-score derivation avoids those terms because independence implies zero covariance.

Worked Example: The Effect of Correlated Scores

Suppose two hypothetical stage scores each have a standard normal marginal distribution under $H_0$, but their correlation is $0.3$. Use fixed weights $w_1=w_2=\sqrt{0.5}$. Assume the scores are jointly normal so their weighted sum is normal. These weights have squared sum one, but the scores are not independent.

The variance is $$ Var(T)=0.5+0.5+2(\sqrt{0.5})(\sqrt{0.5})(0.3)=1+0.3=1.3. $$ Thus $T\sim N(0,1.3)$, not $N(0,1)$. If the standard normal $0.05$ cutoff $1.6449$ were used, the null rejection probability would be $$ 1-\Phi\left(\frac{1.6449}{\sqrt{1.3}}\right) \approx 1-\Phi(1.4421) \approx 0.0746. $$ The unit-variance reference fails here because the covariance contributes $0.3$ to the variance. Merely observing that each stage score has a standard normal marginal distribution would not reveal this problem.

Interpretation, Diagnostics, and Limits

The derivation gives a precise check for the inverse-normal reference distribution:

Under $H_0$, each stage score must have the stated standard normal distribution. When it is obtained by inverse transformation, exact uniformity of the corresponding one-sided p-value is one sufficient condition.
The stage scores must be mutually independent under $H_0$. Dependence adds covariance terms to the variance and can invalidate the reference distribution.
The weights must be fixed for this result, and their squared values must sum to one for the combined score to have unit variance.
The test’s direction must match its tail rule. If larger scores represent evidence against $H_0$, use the upper tail; a different alternative or score convention requires the corresponding tail.

A useful diagnostic is to calculate $\sum_iw_i^2$ explicitly rather than relying on how the weights were described. If it is not one, either normalize the sum or use its correct variance in the reference calculation. Separately assess whether the stage scores are independent and whether their null distributions are adequately represented by standard normal distributions. A unit squared sum does not repair non-normal stage scores or dependence.

This normal-score combination is not the only way to combine evidence. A direct pooled analysis may be more efficient when its model and data structure are appropriate, while Fisher’s method combines p-values using a different reference distribution. Those approaches do not follow from this weighted-normal-sum derivation and should not be treated as interchangeable formulas. The inverse-normal result is especially transparent about how stage contributions enter, but it relies on the validity and independence of the stage scores and on the selected fixed weights.

The result is a reference-distribution statement, not by itself a guarantee that every adaptive design controls Type I error. If adaptation changes the relationship between stage scores, or if the weights depend on observed data, the assumptions used in this derivation may no longer hold. The formula should be applied only after the score distributions, dependence structure, and weight rule have been justified for the design at hand.

Key takeaway: For independent $N(0,1)$ stage scores, a fixed weighted sum is normal with mean zero and variance equal to the sum of squared weights. It has the standard normal reference distribution exactly when that squared sum is one.

Check Your Understanding

Try each problem, then check the worked answers below.

Three independent standard normal stage scores have fixed weights $0.2$, $0.4$, and $0.8$. What is the null distribution of their weighted sum?
For independent standard normal scores, a combined statistic is $T=0.3Z_1+0.4Z_2$. What standard deviation should be used to normalize it?
Two independent standard normal scores have fixed weights $\sqrt{0.25}$ and $\sqrt{0.75}$. If the observed scores are $1.0$ and $1.0$, calculate the upper-tail combined p-value.
Why does a squared-weight sum of one fail to establish a standard normal reference distribution when stage scores are correlated?

Solutions. (1) The squared weights sum to $0.2^2+0.4^2+0.8^2=0.04+0.16+0.64=0.84$. Under independence, the sum is $N(0,0.84)$. (2) Its variance is $0.3^2+0.4^2=0.25$, so its standard deviation is $\sqrt{0.25}=0.5$. Divide the original statistic by $0.5$ to obtain unit variance. (3) The weights’ squared sum is $0.25+0.75=1$, so the statistic is standard normal under the null. Its observed value is $\sqrt{0.25}(1.0)+\sqrt{0.75}(1.0)=0.5+0.8660=1.3660$. The upper-tail p-value is $1-\Phi(1.3660)\approx 0.0860$. (4) Correlation contributes covariance terms to the variance. With two scores, the extra term is $2w_1w_2\rho$, so the variance need not equal one even if $w_1^2+w_2^2=1$.