Worked Example: Fisher Combination Across Two Stages

From Two Stagewise Results to One Test

In a two-stage clinical trial, each stage can produce a p-value for the same null hypothesis. The final test must use both results according to a specified combination rule. Fisher’s combination method converts the two p-values into one statistic and compares it with a chi-square reference distribution. This tutorial works through that calculation, including the tail probability and the final decision.

The numerical calculation is straightforward; the conditions that justify its reference distribution matter just as much. The stages must provide independent evidence under the null, and their p-values must be calibrated for the hypotheses being combined. In the examples, each stage uses a prespecified test of the same null hypothesis, and the stagewise p-values are treated as exact and unrounded.

Definition: For two stagewise p-values $p_1$ and $p_2$, Fisher’s combination statistic is $$ T=-2\left(\ln p_1+\ln p_2\right)=-2\ln(p_1p_2). $$ If, under $H_0$, $p_1$ and $p_2$ are independent Uniform$(0,1)$ random variables, then $T\sim\chi^2_4$. The combined p-value is the upper-tail probability $P(\chi^2_4\geq T_{\mathrm{obs}})$.

The notation $T_{\mathrm{obs}}$ denotes the value of the statistic calculated from the observed p-values. The final p-value measures how often a statistic at least this large would occur under the specified null reference distribution. At significance level $\alpha$, reject $H_0$ when the combined p-value is at most $\alpha$; otherwise, do not reject $H_0$.

Calculating the Chi-Square Tail

For four degrees of freedom, the chi-square upper tail has a convenient closed form:

P(\chi^2_4\geq t)=e^{-t/2}\left(1+\frac{t}{2}\right),\qquad t\geq 0.

One way to justify this expression is to use the chi-square density for four degrees of freedom, $f(t)=\frac{t}{4}e^{-t/2}$ for $t\geq0$. Integrating the density from $t$ to infinity and integrating by parts gives

\int_t^\infty \frac{x}{4}e^{-x/2}\,dx = \left[-\left(\frac{x}{2}+1\right)e^{-x/2}\right]_t^\infty = \left(1+\frac{t}{2}\right)e^{-t/2}.

Because $T_{\mathrm{obs}}=-2\ln(p_1p_2)$, the exponential term in this formula equals the product of the two p-values:

e^{-T_{\mathrm{obs}}/2}=p_1p_2.

Thus, for hand calculations, the combined tail probability can also be computed as $p_1p_2(1+T_{\mathrm{obs}}/2)$. The statistic and its tail probability should be calculated from the same unrounded p-values; rounding intermediate values can cause avoidable discrepancies.

Worked Example: A Strong Result in One Stage and a Modest Result in the Other

Suppose two independent trial stages test the same prespecified null hypothesis and yield $p_1=0.020$ and $p_2=0.300$. Treat the p-values as exact under the null. Find Fisher’s statistic and combined p-value, then decide at $\alpha=0.05$.

First multiply the stagewise p-values and take the natural logarithm:

p_1p_2=(0.020)(0.300)=0.00600, \qquad T_{\mathrm{obs}}=-2\ln(0.00600)=10.2319916\approx10.232.

Under the independence and calibration assumptions, the reference distribution is $\chi^2_4$. The upper-tail probability is

P(\chi^2_4\geq10.2319916) =e^{-10.2319916/2}\left(1+\frac{10.2319916}{2}\right) =(0.00600)(1+5.1159958) =0.03669597\approx0.03670.

Since $0.03670<0.05$, reject $H_0$ at the 5% significance level. Under the stated assumptions, the combined evidence is statistically significant at that level. This decision does not mean that the null hypothesis has a 3.670% probability of being true.

The result uses evidence from both stages: the first p-value is small, while the second is not. Fisher’s rule combines their logarithms rather than requiring each stage to meet the 5% threshold on its own.

Two More Calculations and Their Decisions

A reliable calculation follows the same sequence every time: confirm the input p-values and assumptions, calculate their product and $T_{\mathrm{obs}}$, obtain the $\chi^2_4$ upper tail, and compare the resulting p-value with the prespecified $\alpha$. The next examples illustrate a significant result and a nonsignificant result.

Worked Example: Two Moderate Stagewise Results

In a hypothetical two-stage study, independent stagewise tests of the same null hypothesis produce $p_1=0.040$ and $p_2=0.120$. Assume both p-values are exact under $H_0$. Calculate the combined p-value and decide at $\alpha=0.05$.

The product and statistic are

p_1p_2=(0.040)(0.120)=0.00480, \qquad T_{\mathrm{obs}}=-2\ln(0.00480)=10.6782787\approx10.678.

Using the four-degree-of-freedom tail formula,

P(\chi^2_4\geq10.6782787) =e^{-10.6782787/2}\left(1+\frac{10.6782787}{2}\right) =(0.00480)(1+5.3391394) =0.03042787\approx0.03043.

The combined p-value is below $0.05$, so reject $H_0$ at the 5% level. This is a test decision about the combined evidence under the stated null model; it is not a claim that each stage separately would have produced a significant result.

Worked Example: A Combined Result That Is Not Significant

Suppose two independent stages yield $p_1=0.080$ and $p_2=0.250$ for the same null hypothesis, with exact stagewise p-values under $H_0$. Calculate the combined p-value and decide at $\alpha=0.05$.

The product and statistic are

p_1p_2=(0.080)(0.250)=0.0200, \qquad T_{\mathrm{obs}}=-2\ln(0.0200)=7.8240460\approx7.824.

The reference upper tail is

P(\chi^2_4\geq7.8240460) =e^{-7.8240460/2}\left(1+\frac{7.8240460}{2}\right) =(0.0200)(1+3.9120230) =0.09824046\approx0.09824.

Because $0.09824>0.05$, do not reject $H_0$ at the 5% level. This outcome is not proof that the null hypothesis is true. It says only that the combined p-value does not cross the prespecified rejection threshold.

Assumptions, Diagnostics, and Method Choice

The exact $\chi^2_4$ reference distribution follows when both stagewise p-values are independent and Uniform$(0,1)$ under $H_0$. This condition is met, for example, when stages use independent data and each stagewise test has a continuous null distribution and is correctly calibrated. The tests must address the same null hypothesis with a compatible interpretation of the alternative; combining p-values for unrelated hypotheses does not produce a test of a single common null.

Independence is a design or modeling condition, not a consequence of having two p-values. Reusing participants or outcomes across stages, or allowing information from one stage to affect the other stage’s test in a way that links their null p-values, can invalidate the independent-reference calculation. Confirm how each stage’s data and test were defined before using the formula.

For discrete tests, p-values may not be exactly Uniform$(0,1)$ under the null. Likewise, a displayed p-value rounded to a small number of decimal places is not necessarily the value used in the analysis. The preceding material on discrete p-values and on zero or rounded p-values is relevant here: use a calibrated stagewise p-value and retain adequate numerical precision. A printed value of 0 is not an admissible input to the logarithm; it usually indicates rounding or numerical underflow, not a literally zero probability.

A practical diagnostic is to check whether the inputs are both positive, correspond to the intended stagewise tests, and are available at enough precision to reproduce the calculation. Then verify that the computed combined p-value lies between 0 and 1 and that its direction is consistent: smaller input p-values should not yield weaker Fisher evidence when the other input is held fixed. If assumptions fail or inputs are not calibrated as required, the stated $\chi^2_4$ tail should not be presented as an exact combined p-value.

Fisher’s method and the inverse normal combination method are both ways to combine stagewise evidence, but they transform and aggregate that evidence differently. Fisher uses $-2\ln(p_i)$, so small p-values make large contributions. The inverse normal method transforms p-values to normal scores and combines those scores with specified weights. Which method is appropriate depends on the prespecified testing plan and the intended combination rule; changing the rule after seeing the results can invalidate the claimed error control.

A frequent mistake is to compare $T_{\mathrm{obs}}$ directly with $0.05$, or to treat $T_{\mathrm{obs}}$ as the final p-value. The statistic is on the chi-square scale, while the combined p-value is its upper-tail probability. Another common mistake is to round the statistic and then calculate a tail probability that does not match the reported inputs. Keep sufficient precision through the calculation and round only the reported result.

Key takeaway: With two independent, correctly calibrated stagewise p-values for the same null, calculate $T_{\mathrm{obs}}=-2\ln(p_1p_2)$, obtain its upper-tail probability from $\chi^2_4$, and compare that probability—not the statistic itself—with the prespecified $\alpha$.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

For independent, exact stagewise p-values $p_1=0.050$ and $p_2=0.200$, calculate Fisher’s statistic and combined p-value. What is the decision at $\alpha=0.05$?
Why is $P(\chi^2_4\geq T_{\mathrm{obs}})$, rather than $T_{\mathrm{obs}}$, compared with $\alpha$?
What assumption is questionable if the same patients contribute outcomes to both stages?
A report gives $p_1=0.000$ after rounding. Why should this value not be substituted directly into Fisher’s formula?

Solutions. (1) The product is $0.050(0.200)=0.0100$, so $T_{\mathrm{obs}}=-2\ln(0.0100)=9.21034$. The combined tail is $0.0100(1+9.21034/2)=0.05605$, approximately; since this exceeds $0.05$, do not reject $H_0$. (2) $T_{\mathrm{obs}}$ is a statistic on the chi-square scale; its upper-tail probability is the p-value on the probability scale used for comparison with $\alpha$. (3) Independence of the stagewise p-values may fail when data overlap; the $\chi^2_4$ reference then needs separate justification. (4) The logarithm of zero is undefined, and a displayed zero usually reflects rounding or underflow; obtain the unrounded or otherwise properly calibrated p-value instead.