Deriving Fisher’s Chi-Square Reference Distribution

Why Fisher’s Statistic Has a Chi-Square Reference Distribution

Fisher’s combination statistic converts small stagewise p-values into positive evidence contributions and adds those contributions across stages. To assess the combined evidence, the statistic needs a reference distribution under the null hypothesis. That distribution is not automatic: it follows from specific conditions on the stagewise p-values, especially their validity and independence.

This tutorial derives the reference distribution from first principles. The result applies to $K$ stages, including two-stage adaptive designs, provided the relevant null conditions hold. It does not require the stagewise tests to use identical data or test statistics; it does require their p-values to have the specified joint null distribution.

Definition: For $K$ stagewise p-values $P_1,\ldots,P_K$, Fisher’s combination statistic is $$ X_F=-2\sum_{i=1}^{K}\ln(P_i). $$ Here $\ln$ is the natural logarithm. If the p-values are independent and each has a Uniform$(0,1)$ distribution under the null hypothesis, then $X_F$ has a chi-square distribution with $2K$ degrees of freedom.

The chi-square reference is an upper-tail reference: unusually large values of $X_F$ indicate combined evidence against the null. For a test at significance level $\alpha$, reject when $X_F$ exceeds the $1-\alpha$ quantile of the chi-square distribution with $2K$ degrees of freedom. Equivalently, the combined p-value is the probability, under that reference distribution, of obtaining a statistic at least as large as the observed one.

Conditions for the Exact Reference Distribution

A p-value is exactly Uniform$(0,1)$ under a null model when the null distribution of the test statistic is continuous and the p-value is calculated from that distribution in the usual exact way. For composite null hypotheses, exact uniformity may hold only at particular null parameter values; for type I error control, a p-value is often required to be superuniform instead, meaning $P(P_i\le u)\le u$ for every $u$ between 0 and 1. Superuniformity alone does not establish the exact chi-square reference distribution.

The second condition is independence across stages under the null. In a fixed design, this can follow when stages use independent data and each stagewise p-value is calculated from its own stage data. In an adaptive design, the second-stage data collection or analysis may depend on information from the first stage. A useful sufficient condition is stronger and explicit: each stagewise p-value is conditionally Uniform$(0,1)$ under the null, given the information available before that stage’s data are observed. This conditional validity must hold for the adaptations actually used.

Condition: The standard exact reference $X_F\sim\chi^2_{2K}$ holds if, under the null hypothesis, every $P_i$ is Uniform$(0,1)$ and the $K$ p-values are mutually independent. Conditional Uniform$(0,1)$ validity at each stage, given the preceding stages and adaptation history, is a sufficient condition for this joint independence and uniformity. Marginal uniformity alone is not sufficient.

A p-value based on a discrete test statistic generally cannot take every value between 0 and 1. It is often superuniform rather than exactly uniform. Likewise, dependence between stagewise p-values can change the distribution of their sum of negative logarithms. In either case, applying a chi-square reference without further justification may give an incorrectly calibrated test. The earlier material on discrete stagewise p-values and calibration addresses why exact calibration needs care in that setting.

Deriving the Distribution

First consider one null p-value $P\sim\text{Uniform}(0,1)$, and define its Fisher contribution $Y=-2\ln(P)$. Since $0<P<1$, $Y$ is nonnegative. For any $y\ge 0$, the event $Y\le y$ is equivalent to $P\ge e^{-y/2}$. Therefore:

P(Y\le y) =P(P\ge e^{-y/2}) =1-e^{-y/2},\qquad y\ge 0.

This is the cumulative distribution function of an exponential random variable with rate $1/2$, which is the chi-square distribution with 2 degrees of freedom. The density, obtained by differentiating the cumulative distribution function, is $f_Y(y)=\frac12 e^{-y/2}$ for $y\ge0$. Thus, a single uniform p-value contributes one $\chi^2_2$ random variable.

For $K$ independent uniform p-values, the contributions $Y_i=-2\ln(P_i)$ are independent and each has this $\chi^2_2$ distribution. Their sum is $X_F$. One way to verify the distribution of the sum is to use moment-generating functions. For a chi-square variable with $\nu$ degrees of freedom, the moment-generating function is $(1-2t)^{-\nu/2}$ for $t<1/2$. Independence makes the moment-generating function of a sum the product of the individual functions:

M_{X_F}(t) =\prod_{i=1}^{K}(1-2t)^{-1} =(1-2t)^{-K} =(1-2t)^{-(2K)/2},\qquad t<\frac12.

This is the moment-generating function of a chi-square random variable with $2K$ degrees of freedom. Since the moment-generating function determines the distribution in a neighborhood of zero, $X_F\sim\chi^2_{2K}$. The degrees of freedom are twice the number of independent uniform p-values, not twice the number of parameters in the clinical model.

Worked Example: Transforming One Uniform Null P-Value

Suppose, hypothetically, that one stage produces an exact null p-value of $P_1=0.25$. Its Fisher contribution is:

Y_1=-2\ln(0.25) =-2(-1.3863) =2.7726.

Under the null, $Y_1\sim\chi^2_2$. The chance of a contribution at least this large is:

P(Y_1\ge2.7726) =P(P_1\le0.25) =0.25.

This agrees with the uniformity of $P_1$: a p-value of 0.25 is not itself a combined test result, but its transformed contribution has the corresponding $\chi^2_2$ upper-tail probability under the null.

Using the Reference Distribution for Two Stages

With two independent, exact uniform stagewise p-values, the combined statistic follows $\chi^2_4$. For four degrees of freedom, the upper-tail probability at $x\ge0$ has the closed form $e^{-x/2}(1+x/2)$. This gives a convenient calculation while illustrating an important point: Fisher’s combined p-value is computed from the chi-square distribution, not by multiplying the stagewise p-values.

Worked Example: Combining Two Independent Uniform P-Values

Consider hypothetical stagewise p-values $P_1=0.20$ and $P_2=0.05$. Assume both are exact uniform p-values under the null and are independent. The statistic is:

X_F=-2\{\ln(0.20)+\ln(0.05)\} =-2\ln(0.010) =9.2103.

Because there are two stages, the null reference is $\chi^2_4$. The combined upper-tail p-value is:

P(\chi^2_4\ge9.2103) =e^{-9.2103/2}\left(1+\frac{9.2103}{2}\right) =0.0100(5.6052) =0.05605.

The combined p-value is approximately 0.0561. At significance level $0.05$, this result does not cross the rejection threshold. The product of the stagewise p-values is 0.010, but that product is not the calibrated Fisher p-value; the chi-square tail calculation accounts for the distribution of the sum of both contributions.

Conditional Validity and Adaptation

An adaptive design can use first-stage information to choose features of the second stage, such as its sample size or the treatment comparison to continue. Such adaptation does not by itself invalidate Fisher’s reference distribution, but it makes the conditions on the second-stage p-value consequential. It is not enough that the second-stage p-value would be uniform under a fixed design if the actual adaptation changes its null distribution.

Conditional validity provides a direct justification. Let $H_{i-1}$ denote all information available before stage $i$, including earlier p-values and adaptation decisions. Suppose that under the null, $P_i\mid H_{i-1}$ is Uniform$(0,1)$ for every possible history. For any sets $A_1,\ldots,A_K$ in $[0,1]$, iterated conditioning gives:

P(P_1\in A_1,\ldots,P_K\in A_K) =\prod_{i=1}^{K}|A_i|,

where $|A_i|$ denotes the interval length (or, more generally, the uniform probability) of $A_i$. The factorization is the joint distribution of independent Uniform$(0,1)$ variables. The argument works because, at each stage, the conditional probability of $P_i\in A_i$ is $|A_i|$, regardless of the preceding history. Thus the assumptions needed for the chi-square result can be checked stage by stage even when earlier information informs later design choices.

Worked Example: Checking a Conditional-Validity Argument

Suppose, hypothetically, that a two-stage trial chooses a second-stage sample size using the first-stage result. Under the null, assume the first-stage p-value $P_1$ is Uniform$(0,1)$, and that for every first-stage result and every sample size chosen by the prespecified adaptation rule, the second-stage p-value satisfies $P_2\mid P_1\sim\text{Uniform}(0,1)$.

For any $0\le a,b\le1$, conditioning on $P_1$ gives:

P(P_1\le a,\ P_2\le b) =E\!\left[1(P_1\le a)P(P_2\le b\mid P_1)\right] =E[1(P_1\le a)b] =ab.

The joint probability factors into the two marginal probabilities, so the p-values are independent and uniform under the null. Therefore Fisher’s statistic has a $\chi^2_4$ reference distribution. The conclusion depends on the stated conditional validity for the actual adapted second stage; the fact that the adaptation uses $P_1$ does not, by itself, establish that condition.

Failure Modes and Alternatives

A central diagnostic is to ask what justifies the joint null distribution of the p-values. Check how each stagewise p-value was constructed, whether the test is continuous or discrete, and whether later-stage validity holds conditional on the information used to adapt the design. If stages share observations or otherwise induce dependence, the independent-sum derivation does not apply. Marginal uniformity cannot repair that problem.

The following comparison summarizes the scope of the standard reference result:

Stagewise null p-values	Does the exact $\chi^2_{2K}$ reference follow?	Reason
Independent and exactly Uniform$(0,1)$	Yes	Each contribution is $\chi^2_2$, and independent chi-square variables add their degrees of freedom.
Conditionally Uniform$(0,1)$ at each stage given the prior history	Yes, if the condition holds for every relevant history	Conditional validity implies the joint independent uniform distribution.
Uniform marginally, but dependent across stages	Not necessarily	The sum of dependent contributions need not have the stated chi-square distribution.
Discrete or only superuniform p-values	Not as an exact reference without additional justification	The transformation need not have the exact $\chi^2_2$ distribution.

Alternative approaches depend on which condition fails. If the stagewise tests are discrete, calibration may be needed rather than an exact uniformity argument. If the p-values are dependent, a reference distribution that reflects their dependence, or a design-specific calibration, is needed. The inverse normal combination method is another combination approach, but it also requires justified assumptions about the joint distribution of its stagewise inputs; changing the combination statistic does not automatically remove a dependence or validity problem.

A common mistake is to see that every stagewise p-value is marginally uniform under the null and conclude that Fisher’s statistic must follow $\chi^2_{2K}$. The missing step is independence. For example, if two p-values are both the same Uniform$(0,1)$ random variable $U$, each is marginally uniform, but their Fisher statistic is $-4\ln(U)$, not the sum of two independent $\chi^2_2$ variables. The next example quantifies the resulting mismatch.

Worked Example: Marginal Uniformity Does Not Ensure the Reference

Consider the hypothetical null construction $P_1=P_2=U$, where $U\sim\text{Uniform}(0,1)$. Each stagewise p-value is individually uniform, but the two p-values are perfectly dependent. Their Fisher statistic is:

X_F=-2\{\ln(U)+\ln(U)\}=-4\ln(U).

At the value $x=9.2103$, the actual upper-tail probability under this construction is:

P(X_F\ge9.2103) =P\!\left(U\le e^{-9.2103/4}\right) =e^{-2.3026} \approx0.1000.

If the independent-p-value $\chi^2_4$ reference were incorrectly used, the reported upper-tail probability would instead be approximately 0.0561. The disagreement shows why the independence condition is substantive: the same marginal distributions do not guarantee the same distribution for the combined statistic.

Key takeaway: The chi-square reference for Fisher’s statistic is derived from two facts: $-2\ln(P_i)\sim\chi^2_2$ for an exact uniform null p-value, and independent chi-square contributions add their degrees of freedom. In an adaptive trial, conditional null validity at each stage can establish the required independence; marginal validity alone cannot.

Check Your Understanding

Answer each question, then compare with the solutions.

For $K=3$ independent exact uniform stagewise p-values, what is the null reference distribution of Fisher’s statistic?
Explain why $-2\ln(P)$ has a $\chi^2_2$ distribution when $P$ is Uniform$(0,1)$.
Two independent exact uniform p-values are 0.50 and 0.10. Calculate Fisher’s statistic and give its degrees of freedom.
If every stagewise p-value is marginally uniform but two of them are dependent, is the standard chi-square reference guaranteed? Why or why not?
In an adaptive two-stage design, what conditional property of the second-stage p-value is sufficient to support the standard reference distribution?

Solutions. (1) $\chi^2_6$, because the degrees of freedom are $2K=6$. (2) For $y\ge0$, $P(-2\ln(P)\le y)=P(P\ge e^{-y/2})=1-e^{-y/2}$, the $\chi^2_2$ cumulative distribution function. (3) $X_F=-2\ln(0.50\times0.10)=-2\ln(0.05)=5.9915$, with $2K=4$ degrees of freedom. (4) No. Dependence changes the distribution of the sum, even if both marginal distributions are uniform. (5) The second-stage p-value must be Uniform$(0,1)$ under the null conditional on the information available before the second-stage data, including the first-stage result and the adaptation decision, for every relevant history.

Stagewise null p-values	Does the exact \(\chi^2_{2K}\) reference follow?	Reason
Independent and exactly Uniform\((0,1)\)	Yes	Each contribution is \(\chi^2_2\), and independent chi-square variables add their degrees of freedom.
Conditionally Uniform\((0,1)\) at each stage given the prior history	Yes, if the condition holds for every relevant history	Conditional validity implies the joint independent uniform distribution.
Uniform marginally, but dependent across stages	Not necessarily	The sum of dependent contributions need not have the stated chi-square distribution.
Discrete or only superuniform p-values	Not as an exact reference without additional justification	The transformation need not have the exact \(\chi^2_2\) distribution.