Fisher’s Combination Statistic

Why transform stagewise p-values?

In a multi-stage trial, each stage can provide evidence against a common null hypothesis. Fisher’s combination statistic converts the stagewise p-values into additive quantities: a smaller p-value contributes more to the combined statistic. This makes the statistic straightforward to calculate and reveals how the observed evidence is apportioned across stages.

The statistic is not itself a p-value, and calculating it does not by itself determine whether a trial rejects its null hypothesis. That decision also requires a valid reference distribution or another calibrated rejection rule. The earlier material on discrete stagewise p-values is relevant here: a p-value’s validity under the null and the calibration of a combination statistic are related, but they are not interchangeable.

Because \(p_i\leq 1\), each contribution is nonnegative. A p-value of 1 contributes 0; a smaller p-value contributes a larger positive amount. The contribution records the stage’s additive share of the observed statistic. It is not a percentage of the trial’s evidence in some universal sense, nor does it say how much the stage caused a treatment effect.

The product form gives an equivalent way to calculate the statistic: multiply the p-values and take minus twice the natural logarithm of that product. For a fixed value of the other stage’s p-value, lowering one stage’s p-value increases \(T_F\). If a p-value is reduced by a factor \(r>1\), its contribution increases by \(2\log(r)\):

Thus, a tenfold reduction in a stagewise p-value adds \(2\log(10)\approx 4.605\) to the statistic, regardless of the p-value’s starting value. This is a property of the transformation, not a claim that every stage has equal design importance or equal power.

Calculating and reading the contributions

The calculation assumes stagewise p-values have been obtained from prespecified, appropriate stagewise tests. For a standard Fisher reference calibration, the familiar exact result applies when, under the null, the p-values are independent and each has a continuous Uniform(0,1) distribution. Independence can be supported, for example, by using non-overlapping stage data with the corresponding null assumptions. The continuous-uniform condition is important: a discrete-test p-value is often super-uniform rather than exactly uniform, and dependence between stages also changes the reference distribution.

Fisher’s statistic can still be calculated when these conditions do not hold. What may fail is the usual calibration of that statistic into a final p-value or rejection threshold. In adaptive designs, one must establish that the stagewise tests and the combination rule support the chosen calibration under the design, rather than assuming that a familiar formula applies automatically. The derivation of the standard chi-square reference distribution is treated separately; here the focus is the statistic and its decomposition.

The same decomposition can be expressed as a share of the observed statistic: \(C_i/T_F\), provided \(T_F>0\). Such shares add to 1, but they are only descriptive shares of this transformed statistic. They are not stage-specific p-values, weights chosen in advance, or causal attribution of a trial result.

Conditions, calibration, and alternatives

It is useful to separate three questions. First, are the stagewise p-values valid for their stated stagewise tests? Second, does the combination statistic have a known or correctly calibrated null distribution under the trial design? Third, does the prespecified rejection rule control the desired type I error? A correct answer to the first question does not automatically answer the other two.

Under independent continuous Uniform(0,1) null p-values, the usual Fisher reference result supplies a standard way to calibrate \(T_F\). In applications, however, p-values may be discrete, may depend through shared observations, or may be affected by design features that alter their joint null behavior. In those settings, the ordinary reference distribution should not be used without justification. A design-specific calibration, such as one based on the appropriate joint null distribution, may be needed. The preceding material on calibrating combination-test rejection thresholds supplies the broader principle: the rejection threshold must match the null behavior of the chosen combination statistic.

Fisher’s transformation is one alternative to the inverse-normal combination statistic. Fisher’s statistic works on the logarithmic scale of p-values; the inverse-normal method maps p-values to normal scores and combines those scores using specified weights. The methods can respond differently to patterns of stagewise evidence. Fisher’s sum is especially easy to inspect additively, while an inverse-normal statistic makes its weights explicit. Neither statistic removes the need to choose stagewise tests, specify the null and alternative consistently, and justify final calibration.

Interpret the p-values in a coherent direction. Fisher’s statistic uses their magnitudes, not the signs of estimated treatment effects. If one stage favors benefit and another favors harm, small p-values from both stages can still produce a large Fisher statistic. The numerical combination does not resolve that scientific conflict. Stagewise effect estimates and directional consistency should therefore be examined under the trial’s prespecified interpretation, rather than inferred from \(T_F\) alone.

Diagnostics and common pitfalls

Before interpreting a computed statistic, verify that the inputs are the intended stagewise p-values, each lies in \((0,1]\), and both tests address the same prespecified hypothesis in a compatible direction. Confirm that no stage data are inadvertently reused in a way that makes the p-values dependent, and check whether discreteness or adaptation affects the null distribution. Then verify the arithmetic using both the sum of contributions and the product identity. Small discrepancies after rounding are expected; discrepancies larger than rounding error suggest a calculation or input problem.

• Treating \(T_F\) as a p-value. It is a statistic on a transformed scale, not a probability. A calibrated null distribution or rejection rule is needed to obtain a final inferential conclusion.
• Assuming any valid stagewise p-values yield the standard Fisher calibration. Validity of each p-value separately does not guarantee independence or continuous uniformity of their joint null behavior.
• Calling a larger contribution a larger treatment effect. A contribution depends on the p-value, which reflects the test, information, variability, and other design features as well as the estimated effect. It is not an effect estimate.
• Ignoring direction. Fisher’s statistic uses p-value magnitudes. It does not encode whether estimated effects point in the same direction.
• Confusing rounded arithmetic with the exact calculation. Compute the total from the original p-values, then round the final statistic. If adding rounded contributions, label that sum as an intermediate rounded sum.

A reported zero p-value needs special scrutiny because \(\log(0)\) is undefined. A displayed zero may reflect rounding or numerical limits rather than an exactly zero underlying p-value. Use the underlying valid p-value or an appropriate numerical method; do not silently insert zero into the formula. Likewise, report enough digits to make the calculation reproducible without implying precision beyond that supported by the stagewise results.

The main limitation is that Fisher’s statistic compresses the stagewise p-values into one number. That compression is useful for combination, but it loses information about effect estimates and their directions unless those are considered separately. Its apparent simplicity also makes calibration assumptions easy to overlook. A defensible analysis states the input p-values, the statistic, the null conditions used for calibration, and the rejection rule.