Why Combine Stagewise Evidence?
A seamless Phase 2/3 trial may use evidence from more than one stage to support a confirmatory conclusion. For example, an early stage may contribute data collected before treatment selection, while a later stage contributes new confirmatory data. A combination test turns the stage-specific p-values into one test statistic with a calibrated null distribution. This retains stagewise evidence without simply pooling observations and ignoring the design.
Fisher’s method is one such combination test. It rewards small p-values through their logarithms and has a simple exact reference distribution when the stagewise p-values are independent and exactly uniform under the intersection null. Its simplicity does not remove the need to establish that each p-value tests an appropriate hypothesis and is valid under the trial’s design. In particular, selection and adaptation must be handled in the construction of the stagewise p-values, as considered in the earlier material on valid stagewise p-values after treatment selection.
The intersection null is the hypothesis that all stage-specific null hypotheses hold. Rejecting it is evidence against at least one component null; by itself, it does not establish that every stage showed efficacy, that effects are identical across stages, or that a particular treatment effect has been established in each stage. The confirmatory claim must still match the hypotheses and estimand defined for the trial.
Why the Reference Distribution Is Chi-Square
The result follows from a transformation of a uniform random variable and the additivity of independent chi-square variables. Under a continuous null distribution and a correctly specified test, a p-value is uniform under its null. This statement applies to each stage’s p-value under its corresponding null; the combination result additionally requires independence across stages under the intersection null.
If \(P_i\sim\mathrm{Uniform}(0,1)\), then for \(x\geq 0\), \(P(-2\log P_i\leq x)=P(P_i\geq e^{-x/2})=1-e^{-x/2}\). This is the cumulative distribution function of \(\chi^2_2\).
Thus each \(-2\log P_i\sim\chi^2_2\). If the p-values are independent, these transformed variables are independent, and their sum has a chi-square distribution with the degrees of freedom added: \(T_F\sim\chi^2_{2k}\).
For a prespecified significance level \(\alpha\), reject the intersection null when \(T_F\) exceeds the upper-tail critical value of \(\chi^2_{2k}\), or report the upper-tail probability \(P(\chi^2_{2k}\geq T_F)\).
For \(k=2\), the reference distribution is \(\chi^2_4\). Its survival probability has a convenient expression: if \(t\) is the observed statistic, then \(P(\chi^2_4\geq t)=e^{-t/2}(1+t/2)\). This formula is useful for checking numerical calculations, but for other degrees of freedom a standard chi-square distribution calculation is ordinarily used.
Two-Stage Calculation
Worked Example: Combining Two Independent Stagewise Tests
Suppose, hypothetically, that a seamless trial has two independent stages testing the same prespecified treatment hypothesis. The stagewise p-values are \(p_1=0.10\) and \(p_2=0.05\). Assume that each is exactly uniform under its stage null and that the p-values are independent under the intersection null.
The Fisher statistic is $$ T_F=-2\{\log(p_1)+\log(p_2)\} =-2\log(p_1p_2). $$
Substituting the reported values gives $$ T_F=-2\log(0.10\times0.05) =-2\log(0.005) \approx 10.5966. $$
With two stages, the null reference is \(\chi^2_4\). Here \(t/2\approx5.2983\), so the upper-tail probability is $$ e^{-10.5966/2}(1+10.5966/2) \approx 0.005(1+5.2983) \approx 0.03149. $$
At \(\alpha=0.05\), the combined test rejects the intersection null because \(0.03149<0.05\). The conclusion is that the combined stagewise evidence is statistically significant under the stated exact-uniformity and independence conditions. Neither stage’s individual p-value is below 0.05: the first is 0.10 and the second is exactly 0.05. Fisher’s result does not prove efficacy separately in both stages.
Unequal Stage Information
The ordinary Fisher statistic gives each stage’s p-value the same coefficient, \(-2\), regardless of sample size or information. It does not weight a stage directly by its sample size, information fraction, or precision. Under the intersection null, this symmetry is what permits the same \(\chi^2_{2k}\) calibration for unequal stage sizes, provided the p-values are independent and exactly uniform.
Unequal information matters in a different way: it affects how much evidence a stage is likely to contribute under an alternative. A larger, more informative stage may produce smaller p-values more often when an effect is present. Fisher’s method uses the resulting p-values, not the information amounts themselves. Therefore, it can be valid even when stages are unequal, but its balance of evidence and power may not match the design’s desired emphasis.
Worked Example: Unequal Stage Sizes, Standard Fisher Calibration
Consider a hypothetical trial with 40 participants in its first stage and 160 in its second. Suppose the prespecified stagewise tests produce \(p_1=0.03\) and \(p_2=0.20\). Assume independent stage data, p-values that are exactly uniform under the corresponding stage hypotheses, and a common intersection null.
The information imbalance does not alter the ordinary Fisher formula: $$ T_F=-2\log(0.03\times0.20) =-2\log(0.006) \approx 10.232. $$
For \(\chi^2_4\), the upper-tail probability is $$ e^{-10.232/2}(1+10.232/2) \approx 0.006(1+5.116) \approx 0.03670. $$
Thus the combined test rejects at the 0.05 level, given the assumptions. The small first-stage p-value contributes substantial evidence even though that stage enrolled fewer participants; Fisher’s method does not reduce its contribution because of its smaller size. This result should not be described as showing that the stages had equal precision or that their treatment effects were equal.
When the design calls for contributions explicitly linked to stage information, an inverse-normal combination test with prespecified weights is a natural alternative. Its weights affect how stagewise evidence contributes to the statistic. Fisher’s unweighted method is instead symmetric in the p-values. Neither method makes an invalid or dependent set of stagewise p-values valid merely by combining them.
Worked Example: Three Stages and the Degrees of Freedom
Suppose, hypothetically, three independent stages yield p-values that are exactly uniform under their corresponding components of the intersection null, with observed values \(0.20\), \(0.04\), and \(0.01\). The product is $$ 0.20\times0.04\times0.01=0.00008. $$
The Fisher statistic is $$ T_F=-2\log(0.00008)\approx18.867. $$
There are three stages, so the reference distribution is \(\chi^2_6\), not \(\chi^2_4\). For six degrees of freedom the upper-tail probability is \(e^{-t/2}\{1+t/2+(t/2)^2/2\}\). Substituting \(t=18.867\), \(t/2\approx9.4335\), and \(e^{-t/2}=0.00008\): $$ P(\chi^2_6\geq18.867) \approx0.00008\{1+9.4335+(9.4335)^2/2\} \approx0.00439. $$
The combined result is significant at 0.05 under the assumptions. The degrees of freedom count two per independent stage; they are not determined by the number of patients or by the total sample size.
Assumptions, Diagnostics, and Failure Modes
Before interpreting a Fisher test in a seamless trial, check the conditions that justify its calibration. Each p-value must correspond to a prespecified or otherwise valid stagewise test of the relevant component null. The hypotheses should concern the intended treatment, endpoint, population, and direction of effect. If the first-stage treatment was selected using data, the p-value used for combination must account for that selection rather than being an unadjusted value that is no longer valid for the claimed test.
The p-values must also be exactly uniform under their respective nulls and independent under the intersection null for the exact standard \(\chi^2_{2k}\) reference distribution. If independent p-values are only super-uniform, the usual chi-square calibration is generally conservative rather than exact. Independence can be plausible when stages use disjoint patient cohorts and independent increments, with no shared observations creating dependence. Reusing patients, overlapping data, or deriving stagewise tests from shared control observations can induce dependence. When dependence is present, the ordinary chi-square calibration is not automatically justified; use a method whose null distribution accounts for the dependence or redesign the stagewise evidence construction.
A further check is whether all p-values address compatible hypotheses. Combining p-values from different endpoints or treatment contrasts may test an intersection of distinct nulls, but it does not automatically establish a single coherent efficacy claim. Likewise, p-values with opposite directional meanings should not be combined as if they all represented evidence in one prespecified direction. For two-sided tests, the combined result can reflect evidence in either direction; the direction and clinical meaning of the effects still need to be examined.
| Situation | Consequence for ordinary Fisher calibration | What to check |
|---|---|---|
| Independent stages; exactly uniform p-values | The exact \(\chi^2_{2k}\) reference applies under the intersection null. Independent super-uniform p-values generally give a conservative calibration instead. | Confirm each test and null were defined appropriately. |
| Unequal sample sizes, but independent exactly uniform p-values | Unequal sizes alone do not change the null reference distribution. | Consider whether an unweighted combination suits the design’s aims. |
| Overlapping data or shared observations | Independence may fail, so the standard reference may be wrong. | Establish the dependence structure and calibrate accordingly. |
| Selection-adjusted p-value omitted | The input p-value may not be valid for the claimed test. | Use a stagewise p-value valid after the adaptation. |
Common Pitfall and Interpretation
Wrong reasoning: “The stages have very different sample sizes, so Fisher’s method must multiply the larger stage’s log p-value by a larger weight; otherwise its chi-square result is invalid.” The correction is that the standard Fisher statistic is deliberately unweighted, and unequal stage sizes do not invalidate its null distribution. Under independence and exact uniformity of the p-values under the intersection null, the reference is \(\chi^2_{2k}\). Unequal information may affect power and whether the method’s symmetric treatment of stagewise p-values is desirable, but that is a design question, not a reason to change the statistic after seeing the data.
Another error is to read a significant combination test as proof that every stage was individually significant. Fisher’s test evaluates the combined evidence against the intersection null; it does not impose a requirement that all component tests reject. Nor does a non-significant combination prove there is no clinically meaningful effect. Report the component results and the combined result, identify the tested hypotheses and direction, and state the assumptions supporting the reference distribution.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Two independent stagewise tests have p-values 0.08 and 0.12. Write the Fisher statistic and identify its null degrees of freedom.
- Does a 4:1 ratio in stage sample sizes change the degrees of freedom of the ordinary two-stage Fisher test? Explain.
- What assumption is threatened if some participants contribute data to both stagewise tests?
- Suppose a significant Fisher result combines p-values for different endpoints. What does the result establish, and what does it not establish automatically?
Solutions. (1) \(T_F=-2\log(0.08\times0.12)=-2\log(0.0096)\approx9.292\), with \(df=4\) when the exact-uniformity and independence conditions for the chi-square reference hold. (2) No. The ordinary two-stage statistic has four null degrees of freedom because \(2k=2(2)=4\); unequal sample sizes alone do not alter that calibration. (3) Independence may fail because shared participants can make the p-values dependent; the standard chi-square reference then needs justification or an alternative calibration. (4) It provides evidence against the intersection null that all component endpoint nulls hold, under valid independent p-values. It does not by itself establish efficacy on every endpoint or a single endpoint-specific claim unless the trial’s hypotheses and claim rules support that interpretation.