Why Independent Stage Increments Matter

Why Stage Independence Is a Calibration Question

A combination test uses evidence from separate stages of a trial to make one final decision. Its reference distribution depends not only on whether each stagewise p-value is valid on its own, but also on how the p-values relate to one another. If the p-values are independent uniforms under the null hypothesis, a combination statistic can be calibrated from their joint distribution. If they are dependent, the same reference distribution may no longer control the type I error rate.

Non-overlapping stage data are important because they can support independent stage increments: each stage contributes information from observations not used in the other stage. But non-overlap alone is not a proof of independence. The analysis model, the way the design adapts, shared nuisance estimates, and any reuse of data can also affect the joint distribution. The key requirement is stated conditionally: after all stage-1 results and design choices are known, the stage-2 p-value must still have its stated null distribution.

The phrase complete stage-1 history matters. If the second-stage sample size is chosen using stage-1 results, conditioning only on that sample size may leave other stage-1 information relevant to the stage-2 statistic. A conditional distribution specified only given the selected sample size does not, by itself, establish independence from the full stage-1 history.

From Conditional Validity to Independence

Suppose \(P_1\) is exactly uniform on \([0,1]\) under \(H_0\), and suppose that for every possible stage-1 history \(H_1=h\), the conditional distribution of \(P_2\) is also uniform on \([0,1]\). For any \(a,b\in[0,1]\), the probability that both p-values are at most their respective thresholds can be factored by conditioning on \(H_1\):

The result is the joint distribution function of two independent uniform random variables. Thus conditional uniformity of the second-stage p-value, given the full first-stage history, is sufficient to make \(P_1\) and \(P_2\) independent under the null. The argument extends to more stages by conditioning successively on the entire history before each new stage.

A common route to this condition is independent stage increments. For example, if stage 1 and stage 2 use disjoint sets of independently randomized participants, and the stage-2 statistic is calculated from stage-2 observations using a null model that remains valid for the chosen design, then its null distribution may be unchanged after conditioning on stage 1. The word “may” is essential: the conditional model must actually hold. Reusing stage-1 observations, or estimating a common nuisance quantity from both stages, can break the required conditional distribution.

How Independence Calibrates a Common Combination Test

For two independent, exactly uniform stagewise p-values, Fisher’s combination statistic is \(T=-2\log(P_1P_2)\). Its null reference distribution follows directly from the distribution of a transformed uniform random variable. If \(U\) is uniform on \((0,1)\), then for \(x\geq0\),

This is the cumulative distribution function of a chi-square random variable with 2 degrees of freedom. Since independent transformations of independent p-values remain independent, the sum of two such variables has a chi-square distribution with 4 degrees of freedom. Therefore \(T\sim\chi^2_4\) under the null. More generally, for \(K\) independent, exactly uniform stagewise p-values, \(-2\sum_{k=1}^K\log(P_k)\sim\chi^2_{2K}\).

Why Non-Overlap Helps—and Why It Is Not Enough

Non-overlapping observations remove one direct source of dependence: the same participant’s outcome cannot contribute random variation to both stage statistics. Under a correctly specified null model, independent participants, and separate stage-specific statistics, this separation can make stage increments independent. It is especially useful when the design changes after an interim analysis, because stage 2 can be analyzed using data not already incorporated into stage 1.

Nevertheless, separate participant sets do not automatically imply independent p-values. Dependence can remain if the analysis uses a shared estimated nuisance parameter, if randomization or sampling units are clustered across stages, if outcome measurements are linked, or if stage-2 validity is asserted only for a selected sample size rather than conditional on the full history. The relevant claim is about the joint null distribution of the test statistics or p-values, not merely about who was enrolled in each stage.

Diagnostics, Alternatives, and Limits

Independence is a property of the design and analysis under the null, not something established merely because observed stagewise p-values look uncorrelated in one trial. Useful checks include tracing which participants and outcomes enter each statistic; verifying whether any nuisance estimates or control observations are shared; and deriving the conditional null distribution of each later-stage statistic given the full preceding history and design. When that derivation is difficult, simulation under the null can assess whether the proposed calibration controls the rejection probability across adaptation paths.

If exact uniformity is unavailable—for example, because a test is discrete—its p-value may be super-uniform, meaning \(P(P_k\leq u\mid H_0)\leq u\) for \(0\leq u\leq1\). That is a valid conservative p-value, but it does not have the exact uniform distribution used in the chi-square derivation above. The reference distribution must match the actual joint null behavior. Likewise, if p-values are dependent, possible responses include deriving a dependence-aware reference distribution, using a prespecified valid method that accounts for the dependence, or designing the stages and analysis to obtain independent increments. The appropriate choice depends on the trial’s model and estimand.

The main alternative to an independent-increments combination analysis is not automatically “invalid”; it may be a pooled-data analysis with its own model and calibration. But a method calibrated under independence cannot simply be carried over to dependent inputs. Choosing between combination and pooled-data approaches involves additional questions about how information is used and what the test targets; the essential point here is narrower: each method needs a reference distribution justified for its own joint data structure.

Practical Conditions for Using an Independence Calibration

Before applying a common combination-test reference distribution based on independent p-values, verify the following points:

• Each stagewise test is valid under the null for its stated analysis population and design.
• The later-stage p-value has its stated null distribution conditional on the complete preceding history and adaptation decisions.
• Any stagewise p-values treated as exactly uniform are in fact continuous and exactly uniform under the null; otherwise, account for discreteness or conservatism.
• The combination statistic’s reference distribution follows from the established joint distribution, not from marginal p-value validity alone.

These conditions make the calibration argument transparent and auditable. They also identify where a failure can occur: a broken stagewise test, a conditional distribution that changes after adaptation, or dependence between otherwise valid marginal p-values.