Why Adapt the Second Stage?
An interim analysis can show that a trial needs more participants than originally planned, perhaps because the treatment effect is smaller than expected or outcome variability is greater. Increasing the stage-two sample size may improve precision and power. But the interim results also contain evidence about the treatment effect. If the final analysis simply pools all observations and treats the resulting sample size as though it had been fixed in advance, the usual reference distribution need not apply: the sample size was chosen using data related to the outcome.
A combination test handles the issue differently. It keeps the stage-one evidence in a stage-one p-value, obtains a p-value from the new stage-two data, and combines the two. The key is not that the stage-two sample size stays fixed. It is that the stage-two p-value remains valid conditional on the stage-one information used to choose that size. With that condition and fixed combination weights, the usual inverse normal calibration remains available.
Conditions for Validity After Adaptation
The stage-two sample size may be selected using stage-one results, including the stage-one treatment effect or p-value. The test remains valid when the following conditions hold. They describe what the p-values and adaptation must accomplish, rather than requiring a particular rule for choosing the sample size.
- Stage-one validity. Under \(H_0\), \(p_1\) is uniform or super-uniform: for every \(u\) between 0 and 1, \(P(p_1\leq u)\leq u\).
- Conditional stage-two validity. Given all stage-one information used in the adaptation, including the selected stage-two sample size, \(p_2\) is uniform or super-uniform under \(H_0\). In symbols, \(P(p_2\leq u\mid \mathcal{F}_1)\leq u\) for every \(u\), where \(\mathcal{F}_1\) denotes the information available at the end of stage one.
- Stage separation. Stage-two inference is based on stage-two data, not a second use of the stage-one observations. The stage-two procedure must be valid at the sample size actually selected.
- Prespecified combination rule. The weights, testing direction, hypothesis, and rejection rule are fixed independently of the stage-two results. In particular, do not change the weights merely because the realized stage-two size differs from its plan.
- Appropriate handling of other choices. Any adaptation involving endpoints, treatment arms, doses, or hypotheses must preserve conditional validity and account for any multiplicity it creates. A valid p-value for one fixed hypothesis does not automatically adjust for selecting that hypothesis.
These requirements are often met when the stage-two participants provide new, independent observations and the stage-two test is correctly specified for the selected sample size. The p-value must also account appropriately for the trial’s randomization, outcome distribution, and any nuisance parameters. Independence of stage data alone is not enough if the stage-two p-value is miscalibrated.
Why the Null Calibration Is Preserved
First suppose the p-values are exact and continuous under \(H_0\). Then \(Z_1\) has a standard normal distribution. Conditional stage-two validity means that, given \(\mathcal{F}_1\), \(Z_2\) also has a standard normal distribution. Because that conditional distribution does not depend on \(\mathcal{F}_1\), \(Z_2\) is independent of the stage-one information, and hence of \(Z_1\). Therefore:
The calculation uses the fixed weights and their squared-sum condition; it does not require the stage-two sample size to equal its original target. The sample size affects the distribution of the stage-two test statistic under alternatives, and thus the test’s power, but under the null the conditional p-value remains calibrated.
The result also allows conservative p-values. A super-uniform p-value gives no more chance of a small p-value than an exact uniform one. Its normal-score transform is correspondingly no more favorable to rejection than a standard normal score. Applying this fact first conditionally to stage two and then to stage one shows that the upper-tail rejection probability is at most \(\alpha\). Thus the test controls type I error, though it can be conservative if either stagewise p-value is conservative.
Worked Calculations
Worked Example: Increasing Stage-Two Enrollment Without Changing Weights
Suppose a hypothetical trial plans 50 participants in each stage. After stage one, investigators use the interim results to increase the stage-two target to 100 participants. The protocol specified \(w_1=0.6\) and \(w_2=0.8\), so \(w_1^2+w_2^2=0.36+0.64=1\). Assume the same one-sided hypothesis is retained and that stage-two inference uses only the new stage-two data.
For illustration, suppose the resulting stagewise p-values are \(p_1=0.12\) and \(p_2=0.08\). Their normal scores are approximately \(Z_1=\Phi^{-1}(0.88)=1.175\) and \(Z_2=\Phi^{-1}(0.92)=1.405\). The p-value and the validity of the stage-two procedure are assumed to be properly calibrated at the selected size.
Apply the prespecified weights:
At one-sided \(\alpha=0.05\), the critical value is \(\Phi^{-1}(0.95)=1.6449\). Since \(1.829>1.6449\), reject \(H_0\). The combined upper-tail p-value is approximately \(1-\Phi(1.829)=0.0337\). The decision uses the original weights, not weights recalculated to reflect the realized increase from 50 to 100 stage-two participants.
Worked Example: A Weighted Test With a Non-Significant Result
Consider a hypothetical two-stage trial with prespecified weights \(w_1=0.5\) and \(w_2=\sqrt{0.75}\approx0.8660\). Their squared weights sum to one. Stage one yields a one-sided p-value \(p_1=0.040\). Based on the interim information, the stage-two sample size is increased; the stage-two-only analysis gives \(p_2=0.300\). Assume that the stage-two p-value is valid conditional on the stage-one information and the selected size.
Transform the p-values to normal scores: \(Z_1=\Phi^{-1}(0.960)\approx1.751\) and \(Z_2=\Phi^{-1}(0.700)\approx0.5244\). The combination statistic is:
The second product, using the displayed rounded inputs, is \(0.8660\times0.5244=0.4541304\), which rounds to \(0.4541\). At one-sided \(\alpha=0.05\), the critical value is \(1.6449\), so \(1.3296<1.6449\) and the test does not reject \(H_0\). The combined p-value is approximately \(1-\Phi(1.3296)=0.0918\). The stage-one p-value is relatively small, but the weaker stage-two evidence and the fixed weighting yield a non-significant combined result.
Worked Example: Choosing the Stage-Two Size From an Interim Rule
Suppose a hypothetical protocol specifies the following rule: enroll 80 participants in stage two if the stage-one normal score is below 1, and otherwise enroll 40. The one-sided test direction and weights \(w_1=0.5\), \(w_2=0.8660\) are set in advance. The observed stage-one p-value is \(p_1=0.20\), giving \(Z_1=\Phi^{-1}(0.80)=0.8416\). Since \(0.8416<1\), the rule selects 80 stage-two participants.
Assume the independent stage-two data at that selected size give a valid one-sided stage-two p-value \(p_2=0.15\), so \(Z_2=\Phi^{-1}(0.85)\approx1.0364\). The combination statistic is:
At the one-sided 5% level, \(1.3183<1.6449\), so the result is not significant; the combined p-value is approximately \(1-\Phi(1.3183)=0.0937\). The interim rule determines the amount of new data, while conditional validity of the stage-two p-value permits its use with the unchanged weights.
What Can Go Wrong?
A common mistake is to reason: “The stage-two sample size was selected after seeing interim results, so combining the two p-values must inflate type I error.” That conclusion is too broad. The size change is not itself the decisive problem. If stage two is independent of stage one and its p-value is valid conditional on the adaptation, the inverse normal calibration above still applies. Conversely, simply using a combination statistic does not repair an invalid stage-two p-value.
- Recalculating weights from realized sizes. Replacing the fixed weights with values computed from the adapted stage-two size creates a data-dependent combination rule. The standard argument above no longer applies automatically. Use such weights only with a design-specific calibration that accounts for the adaptation.
- Reusing stage-one observations. A nominally stage-two p-value calculated from pooled data generally reuses the first-stage evidence. It need not be conditionally uniform given stage one, so it cannot be substituted for the independent stage-two p-value without a justified method.
- Using a p-value valid only before adaptation. If the selected sample size, test statistic, or analysis procedure changes the null distribution, a p-value calibrated for the original fixed-size plan may not be valid at the selected size.
- Changing the target after seeing stage-two outcomes. The stated argument covers adaptation based on stage-one information. Letting stage-two results influence further enrollment or the final analysis requires additional sequential calibration.
- Changing the hypothesis or direction. Choosing the endpoint, dose, or favorable direction after seeing data can add selection error. The original one-sided critical value does not automatically account for that choice.
Before interpreting a result, check the adaptation record, the selected sample size, the stage-specific data used for each p-value, and whether the weights and testing direction match the prespecified rule. A small final p-value is not evidence that these conditions held; they are properties of the design and analysis, not diagnostics that can be inferred from the p-value alone.
How This Approach Differs From Alternatives
| Approach | What it does | Key consideration with adaptation |
|---|---|---|
| Inverse normal combination test | Transforms stagewise p-values to normal scores and combines them with fixed weights. | Allows a stage-one-driven size change when stage-two p-values are conditionally valid. |
| Ordinary pooled-data test | Analyzes the accumulated observations as one data set using its usual fixed-design reference distribution. | That reference distribution may not account for a sample size selected using interim outcomes. |
| Prespecified fixed-size design | Collects the planned number of participants without changing the target based on interim evidence. | Avoids this particular adaptation, but may be less responsive to interim information about the required size. |
The combination approach is not automatically more powerful than a pooled-data method or a fixed-size design. Its benefit here is a transparent calibration based on stagewise evidence despite the specified adaptation. Its limitations include dependence on correctly computed stagewise p-values, loss of power if the weights are poorly chosen for the true information pattern, and the need to plan the rule before using the interim results.
Check Your Understanding
Answer each question using the conditional-validity argument and the stated one-sided test convention.
- What must be true of \(p_2\) when the stage-two size is selected using stage-one results?
- For \(w_1=0.6\), \(w_2=0.8\), \(p_1=0.20\), and \(p_2=0.10\), calculate \(C\) and decide at one-sided \(\alpha=0.05\).
- Why is recalculating the weights from the realized stage sizes not automatically justified?
- Why may a p-value from a pooled analysis fail to serve as \(p_2\) in this combination test?
Solutions. (1) Under \(H_0\), \(p_2\) must be uniform or super-uniform conditional on all stage-one information used in selecting the size, including the selected size. (2) \(Z_1=\Phi^{-1}(0.80)=0.8416\) and \(Z_2=\Phi^{-1}(0.90)=1.2816\). Thus \(C=(0.6)(0.8416)+(0.8)(1.2816)=0.5050+1.0253=1.5303\). Because \(1.5303<1.6449\), do not reject \(H_0\) at the 5% level. (3) The usual null calibration relies on weights fixed independently of the observed stagewise results; a data-dependent weight rule needs its own calibration. (4) A pooled analysis reuses stage-one data, so its p-value need not be conditionally valid given stage one and may not have the required stage-two null distribution.