Inverse-Normal Combination Tests

Why Combine Stagewise Evidence?

In an adaptive sample size design, a first-stage analysis may guide how much additional information to collect. The final test must use evidence from both stages while maintaining the intended type I error under the null hypothesis. The inverse-normal combination test does this by expressing each stage’s one-sided evidence on a standard normal scale and combining the resulting scores.

The key ingredients are the direction of the test, the stagewise scores, and weights fixed in advance. A one-sided upper-tail test treats larger positive scores as evidence against $H_0$. Its rejection threshold is set on the combined-score scale, not by applying the original stagewise critical value separately to each score.

Definition: Let $Z_1,\ldots,Z_K$ be independent stagewise standard normal scores under $H_0$, with positive prespecified weights $w_1,\ldots,w_K$ satisfying $\sum_{i=1}^{K} w_i^2=1$. The inverse-normal combination statistic is $$ Z_{\mathrm{comb}}=\sum_{i=1}^{K} w_i Z_i. $$ For a one-sided upper-tail test at significance level $\alpha$, reject $H_0$ when $Z_{\mathrm{comb}}\ge z_{1-\alpha}$, where $z_{1-\alpha}=\Phi^{-1}(1-\alpha)$.

Stagewise Scores, Weights, and the Rejection Rule

A stagewise score can be obtained directly from a test statistic whose null distribution is standard normal. It can also be obtained from an exact one-sided stagewise p-value $p_i$ by the inverse-normal transformation $Z_i=\Phi^{-1}(1-p_i)$. Under a continuous exact null distribution, this transformation gives a standard normal score. The upper-tail convention matters: small p-values correspond to large positive scores.

The weights determine how much each stage contributes to the final statistic. They must be specified before the stagewise results are observed and normalized so their squared values sum to one. They do not generally sum to one. For two stages, for example, $w_1=0.6$ and $w_2=0.8$ satisfy the required normalization because $0.6^2+0.8^2=1$.

Under the stated null assumptions, a fixed weighted sum of the independent standard normal scores has mean zero and variance $\sum_i w_i^2=1$, and is therefore standard normal. This supplies the null reference for the decision rule. The detailed distributional derivation is a separate matter; here the practical consequence is that the critical value is the usual upper standard normal quantile $z_{1-\alpha}$.

Worked Example: Two Stages With Equal Weights

Suppose a hypothetical two-stage trial has independent stagewise scores $Z_1=1.20$ and $Z_2=1.50$ under the assumed null model. Use equal weights and test at one-sided $\alpha=0.05$. Assume the scores are valid standard normal scores under $H_0$, the stage-two data are independent of stage one, and the equal weights were prespecified.

Equal weights with squared sum one are $w_1=w_2=1/\sqrt{2}$. Thus

Z_{\mathrm{comb}}=\frac{1}{\sqrt{2}}(1.20+1.50) =\frac{2.70}{1.4142}=1.9092.

The one-sided critical value is $z_{0.95}=1.6449$. Since $1.9092\ge 1.6449$, reject $H_0$ at the 0.05 level. The combined upper-tail p-value is approximately $1-\Phi(1.9092)=0.0281$. This is evidence against $H_0$ in the prespecified positive direction; it is not the probability that $H_0$ is true.

Worked Example: Unequal Prespecified Weights

In a hypothetical adaptive design, suppose the stagewise scores are $Z_1=1.10$ and $Z_2=1.80$. The analysis plan specifies $w_1=0.6$ and $w_2=0.8$, and the one-sided significance level is $\alpha=0.025$. Assume the stagewise scores meet the null and independence conditions, including that the stage-two score remains valid for the sample size selected after stage one.

The weights are normalized because $0.6^2+0.8^2=1$. The combined statistic is

Z_{\mathrm{comb}}=(0.6)(1.10)+(0.8)(1.80) =0.66+1.44=2.10.

At $\alpha=0.025$, the upper-tail critical value is $z_{0.975}=1.9600$. Since $2.10\ge1.9600$, reject $H_0$. The combined p-value is approximately $1-\Phi(2.10)=0.01786$, which is below 0.025. The larger second-stage weight gives its score a larger contribution, but the weights were not chosen in response to the observed scores.

Justifying the Threshold and Its Scale

The rejection threshold follows from the assumed null distribution of the combined statistic. Independence and standard normality give a normal weighted sum; the squared-weight condition makes its variance one. Therefore the upper-tail probability at the cutoff is $\alpha$:

P_{H_0}(Z_{\mathrm{comb}}\ge z_{1-\alpha}) =1-\Phi(z_{1-\alpha})=\alpha.

This also explains why the normalization is essential. If the weights have squared sum $c\ne1$, the weighted sum has null variance $c$, rather than one. One can either normalize the weights or account for the resulting variance when setting a threshold; the definition above uses normalized weights, so the standard normal cutoff applies directly.

For $K$ stages with equal weights, each weight is $1/\sqrt{K}$, and $Z_{\mathrm{comb}}=(\sum_i Z_i)/\sqrt{K}$. For five stages at $\alpha=0.05$, the equivalent rule on the unstandardized sum is $\sum_i Z_i\ge \sqrt{5}\,z_{0.95}$. Numerically, $\sqrt{5}\,z_{0.95}\approx(2.2361)(1.6449)=3.678$. The equivalent normalized-statistic rule remains $Z_{\mathrm{comb}}\ge1.6449$.

Worked Example: Five Equal-Weight Stages

Consider a hypothetical five-stage test with independent standard normal stagewise scores $0.50,\ 0.80,\ 1.00,\ 0.90,\ 1.20$ under $H_0$. Use equal prespecified weights and a one-sided level of 0.05. The assumptions are that all five scores have the stated null distributions, are independent, and use the same favorable direction.

Each weight is $1/\sqrt{5}$, and the score sum is $0.50+0.80+1.00+0.90+1.20=4.40$. Hence

Z_{\mathrm{comb}}=\frac{4.40}{\sqrt{5}}=1.9677.

Because $1.9677\ge z_{0.95}=1.6449$, reject $H_0$. Equivalently, the unstandardized sum $4.40$ exceeds the cutoff $\sqrt{5}\,z_{0.95}\approx3.678$. These are the same decision expressed on two different scales; the threshold must be expressed on the same scale as the statistic being compared.

Worked Example: Calculating a Combined P-Value

Suppose a hypothetical two-stage analysis uses prespecified weights $w_1=0.6$ and $w_2=0.8$, with independent valid stagewise scores $Z_1=1.2000$ and $Z_2=2.41925$. The weights have squared sum one, and the one-sided test is conducted at $\alpha=0.01$.

The combination statistic is

Z_{\mathrm{comb}}=(0.6)(1.2000)+(0.8)(2.41925) =0.7200+1.9354=2.6554.

The one-sided combined p-value is $1-\Phi(2.6554)\approx0.003961$. The critical value is $z_{0.99}=2.3263$. Since $2.6554\ge2.3263$, equivalently since $0.003961<0.01$, reject $H_0$ at the 0.01 level. The p-value is the null probability of a combined score at least this large in the specified direction, under the reference assumptions.

Assumptions, Diagnostics, and Alternatives

The calculation is only as reliable as the stagewise scores and their null behavior. Before applying the rule, verify that the favorable direction was specified, each stagewise test gives a valid standard normal score under $H_0$, the stages use independent data, and the weights were fixed before observing the stagewise evidence. In an adaptive sample size design, stage two should use new observations, and its score must remain valid under the sample size selected using stage-one information. Independence of the raw data alone does not repair a stage-two test statistic whose null calibration is invalid.

Useful diagnostics include checking the analysis plan and implementation against the formula: verify that weights are positive, fixed, and satisfy $\sum_i w_i^2=1$; verify that upper-tail p-values were transformed as $\Phi^{-1}(1-p_i)$; and confirm that the final comparison uses $z_{1-\alpha}$, not a stagewise boundary or a two-sided cutoff. Simulation of the complete design under $H_0$, as in operating-characteristic evaluation, can reveal implementation errors or a failure of the assumed null calibration. Simulation is a diagnostic, not a substitute for establishing the assumptions.

A common mistake is to choose a larger stage-two weight after seeing a promising stage-one result and then use the usual standard normal cutoff. The standard argument for this cutoff relies on weights fixed independently of the stagewise scores. Changing a weight in response to the observed interim result changes the procedure and requires a validity argument beyond the fixed-weight rule.

Another mistake is to add stagewise p-values or compare each stagewise score separately with $z_{1-\alpha}$. Neither operation is the inverse-normal test. The specified weighted sum is the test statistic, and its single final cutoff controls the one-sided null rejection probability under the assumptions. Compared with continuing an original fixed-sample design unchanged, a combination test gives an explicit way to use stagewise evidence after an adaptive change while retaining a prespecified final test rule. Compared with conditional-error recalibration, it defines the final decision through a combination statistic rather than by recalibrating a later-stage boundary. These approaches are not interchangeable without checking their respective conditions.

The method has limitations. It does not make invalid stagewise p-values valid, eliminate bias from problematic data handling, or automatically validate arbitrary data-dependent weights. It also depends on a prespecified one-sided direction; if the scientific question is two-sided, the rejection rule must be designed for that question rather than treating an upper-tail rule as two-sided. When the null scores are only approximate standard normal, the resulting type I error control is likewise approximate.

Key takeaway: For independent valid stagewise standard normal scores and fixed normalized weights, calculate $Z_{\mathrm{comb}}=\sum_i w_iZ_i$ and reject the one-sided null exactly when $Z_{\mathrm{comb}}\ge z_{1-\alpha}$. The assumptions and the statistic’s scale must match the cutoff.

Check Your Understanding

Answer each question, then check the solution that follows it.

With two equal weights, $Z_1=0.40$ and $Z_2=1.80$, what is the combined statistic and the decision at one-sided $\alpha=0.05$?
For $w_1=0.6,\ w_2=0.8,\ Z_1=1.00,\ Z_2=1.00$, calculate the combined statistic. Does the test reject at $\alpha=0.025$?
What upper-tail critical value is used for a one-sided test at $\alpha=0.01$?
Why is it not justified to select the weights after inspecting the stagewise scores and then automatically use the usual cutoff?
If the prespecified alternative is in the lower direction, can the upper-tail rule be used unchanged?

Solutions. (1) $Z_{\mathrm{comb}}=(0.40+1.80)/\sqrt{2}=1.5556$. Since this is below $z_{0.95}=1.6449$, do not reject $H_0$ at the 0.05 level. (2) $Z_{\mathrm{comb}}=(0.6)(1.00)+(0.8)(1.00)=1.40$. The $\alpha=0.025$ cutoff is $z_{0.975}=1.9600$, so do not reject. (3) $z_{0.99}=2.3263$. (4) The fixed-weight null reference argument assumes the weights are specified independently of the observed scores; choosing them from those scores changes the test and requires a separate validity argument. (5) No. The direction must be incorporated into the stagewise scores and rejection rule; an upper-tail rule tests for positive-direction evidence.