Why the Two Representations Can Agree—or Differ
A combination test can represent each stage’s evidence as a signed Z statistic or as a p-value transformed to a normal score. These representations can produce exactly the same weighted combination, but only when the p-value and the Z statistic encode the same tail probability and direction. If a p-value uses a different reference distribution, or if it has discarded the direction of the stage effect, the transformed score generally differs from the original Z statistic.
This distinction matters because the stage contributions are added on a numerical scale. Changing a stage score can change the combined statistic, even if the p-value still ranks that stage’s result as more or less extreme in the same way. The earlier material on transforming p-values to normal scores and the weighted inverse normal combination provides the basic construction; here the focus is on exactly when that construction recovers the stagewise Z values.
The inverse normal combination statistic is typically written \(C=\sum_{i=1}^{K}w_i Z_i^*\), where \(Z_i^*\) is the stage score used in the combination and the weights satisfy \(\sum_{i=1}^{K}w_i^2=1\). When \(Z_i^*=Z_i\) stage by stage, combining direct Z statistics and combining their corresponding normal scores are not merely similar procedures: they calculate the same statistic.
Conditions for Exact Agreement
For exact equality at a stage, the p-value must be computed from the same standard normal statistic using the matching tail convention. For an upper-tail alternative, a large positive \(Z_i\) is evidence in the specified direction, so the p-value is the area above it. Applying the inverse upper-tail transformation recovers that statistic:
For a lower-tail alternative, the small p-value is the area below the statistic. The matching transformation is therefore \(\Phi^{-1}(p_i)\), which also returns \(Z_i\). The tail convention is part of the definition: using the upper-tail transformation on a lower-tail p-value does not recover the original statistic.
A two-sided p-value requires a direction-preserving transformation if it is to recover a signed Z statistic. With a standard normal reference, the two-sided p-value is \(p_i=2\{1-\Phi(|Z_i|)\}\). It records the magnitude of the evidence, but not whether \(Z_i\) is positive or negative. The sign must be supplied separately.
These identities give a simple criterion for equality of weighted combinations. If each stage’s transformed score equals its direct score, and the same weights are applied, then the sums are equal. If even one stage’s score changes, the sums need not agree. Under the null, the usual standard-normal calibration additionally requires the stage scores to be independent standard normal variables and the weights to be fixed with sum of squares one. Independence and the role of stage increments are central to the combination test principle.
A Short Justification for the Weighted Result
Suppose the stagewise statistics \(Z_1,\ldots,Z_K\) are independent and each has a standard normal distribution under the null hypothesis. If the p-values are calculated from those statistics using matching tails, the transformations above show that \(Z_i^*=Z_i\) for every stage. Therefore the p-value-based combination is identical to the direct-statistic combination. Its null distribution follows from the normality and independence assumptions:
The first equality uses the fact that a weighted sum of independent normal random variables is normal; its mean is the weighted sum of their means, and its variance is the sum of their weighted variances. The last equality uses the normalization of the weights. Thus, under these assumptions, both representations have the same statistic and the same null reference distribution. Equality of the formulas does not, by itself, establish the assumptions: stage dependence, a mismatched p-value reference distribution, or data-dependent weights require separate attention.
Worked Example: Matching Upper-Tail P-Values Recover the Z Combination
Suppose two independent stages produce standard normal statistics \(Z_1=1.2816\) and \(Z_2=0.5244\). Consider a prespecified upper-tail test with weights \(w_1=0.6\) and \(w_2=0.8\). The weights are normalized because \(0.6^2+0.8^2=1\). Assume each p-value is calculated from the stated standard normal statistic.
The stage p-values are approximately \(p_1=1-\Phi(1.2816)=0.1000\) and \(p_2=1-\Phi(0.5244)=0.3000\). Transforming each p-value with the upper-tail inverse gives
The direct-statistic combination is
The p-value-based combination uses the same two scores and weights, so it also equals \(1.18848\). Under the independent standard normal null model, its one-sided combined p-value is \(1-\Phi(1.18848)\), approximately \(0.1173\). The agreement follows from the matched upper-tail normal p-values, not just from the fact that both methods use weights.
Two-Sided P-Values: Magnitude Is Not Direction
A common source of disagreement is treating a two-sided p-value as though it were an upper-tail p-value. A two-sided p-value is compatible with both a positive and a negative statistic of the same absolute size. Applying \(\Phi^{-1}(1-p_i)\) to it produces a nonnegative score, so it cannot generally reproduce a signed stagewise statistic.
Worked Example: Restoring the Sign Before Combining
Suppose two stages have \(Z_1=1.5\) and \(Z_2=-1.5\), and each reports a two-sided standard normal p-value. For either stage,
The signed two-sided transformation recovers the original statistics:
With weights \(0.6\) and \(0.8\), the direct and signed-score combinations both equal \(0.6(1.5)+0.8(-1.5)=0.9-1.2=-0.300\). In contrast, the incorrect upper-tail transform applied to both two-sided p-values gives \(\Phi^{-1}(1-0.1336)\approx1.11\) for each stage, and a weighted sum of about \(0.6(1.11)+0.8(1.11)=1.55\). That is a different statistic because the signs have been lost. The example also illustrates why a two-sided p-value alone cannot tell whether stages point in the same direction.
If a test is specified as one-sided, use the p-value tail corresponding to its prespecified direction. If two-sided p-values are being used but a signed inverse normal combination is intended, retain the direction from each stage’s estimate or test statistic and use the signed transformation. If the direction is unavailable, the p-values alone do not identify the signed Z values; assigning all scores positive silently changes the procedure.
When the P-Value Uses Another Reference Distribution
The equality above depends on how the p-value was calculated. A p-value is a tail probability relative to a specified null distribution. If the statistic is compared with a \(t\), permutation, or other reference distribution, then applying a standard normal inverse tail produces a normal-equivalent score, not necessarily the original test statistic. The transformation can still be used as a combination score, but it is not interchangeable with the original statistic.
Worked Example: A t-Based P-Value Does Not Transform Back to t
Consider a hypothetical stage with a test statistic \(T=2.000\) compared with a \(t\) distribution with \(df=10\), using an upper-tail test. Assume that the \(t\) reference distribution is appropriate for this stage’s analysis. Its p-value is approximately \(p=0.03669\). The upper-tail normal score is
Thus the original statistic is \(T=2.000\), while the score obtained by transforming its p-value is approximately \(1.791\). They are not equal, because the p-value was calculated using a \(t_{10}\) tail rather than a standard normal tail. In this example the p-value transform can serve as a normal-scale contribution, but it should not be described as recovering the original statistic. Combining the raw \(t\) values as though they were independent standard normal scores would also require justification that is not supplied by this transformation.
More generally, a p-value may be exactly uniform under its continuous null distribution without being calculated from a standard normal statistic. Its inverse normal transform can then have a standard normal null distribution, but it need not equal the original statistic. This distinction separates two claims: the transformed score has a standard normal null distribution and the transformed score equals the stagewise statistic. The first can hold while the second fails.
Checks, Alternatives, and Limitations
Before claiming that the direct-Z and p-value approaches agree, check each stage’s analysis. The p-value must come from the same stagewise statistic and the stated null reference distribution; the tail must match the alternative; and the signed direction must be retained if a signed combination is intended. For the standard inverse normal calibration, also check independence under the null and the weight normalization. In particular, numerical agreement at one data point does not prove that the null model assumptions hold.
| Stage input | Score to use to recover a signed standard normal statistic | What can go wrong |
|---|---|---|
| Upper-tail standard normal p-value | \(\Phi^{-1}(1-p_i)\) | Using the wrong tail reverses or changes the score. |
| Lower-tail standard normal p-value | \(\Phi^{-1}(p_i)\) | Using the upper-tail inverse does not recover the statistic. |
| Two-sided standard normal p-value with direction retained | \(\operatorname{sign}(Z_i)\Phi^{-1}(1-p_i/2)\) | Without the sign, the stage’s direction is lost. |
| P-value based on another reference distribution | Its normal-equivalent score may be \(\Phi^{-1}(1-p_i)\) for an upper-tail test | The result generally differs from the original test statistic. |
Direct Z combination is transparent when every stage supplies a comparable signed Z statistic with the required null behavior. The p-value route is useful when stage evidence is naturally reported as p-values and a normal-score combination is planned. It also makes clear how a p-value maps onto a common normal scale. Neither representation automatically fixes dependence, an inappropriate stagewise reference distribution, or a mismatch between the test direction and the score sign.
A practical diagnostic is to recompute a stage’s p-value from its stated statistic and reference distribution, then apply the intended inverse transformation. If the result does not reproduce the statistic, identify why before combining: the reference distribution may differ, the p-value may be two-sided, the tail may be mismatched, or rounding may have altered the value. Rounded p-values can yield only approximate equality. A value reported as zero cannot be directly inverted to a finite normal score; use an adequately precise underlying value rather than treating the printed zero as exact.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- A stage reports \(Z=0.8416\) and an upper-tail standard normal p-value. What is the p-value, and which inverse transform recovers \(Z\)?
- A two-sided standard normal p-value is \(0.05\). What is the magnitude of its normal score, and what additional information is needed to recover a signed score?
- A stage reports an upper-tail p-value based on a \(t\) distribution. Is \(\Phi^{-1}(1-p)\) necessarily the original test statistic? Explain.
- Two independent standard normal stage statistics are combined with weights \(0.3\) and \(0.4\). Is the usual standard normal null calibration justified as written?
Solutions. (1) \(p=1-\Phi(0.8416)\approx0.2000\); \(\Phi^{-1}(1-p)=\Phi^{-1}(0.8000)=0.8416\). (2) The magnitude is \(\Phi^{-1}(1-0.05/2)=\Phi^{-1}(0.975)\approx1.960\); the sign of the stage statistic or effect estimate is needed. (3) No. The inverse normal transform gives a normal-equivalent score; it equals the original statistic only if the p-value was calculated from that same statistic using the corresponding standard normal tail. (4) No. The squared weights sum to \(0.3^2+0.4^2=0.25\), not 1. Under the stated independence and normality assumptions, the combination has null variance \(0.25\), so it is not standard normal without an appropriate normalization.