Why Conditional Error Matters at the Phase Boundary
A seamless trial may reach the phase boundary with a confirmatory plan that no longer fits the clinical or operational situation. The selected treatment may differ from the originally planned treatment, or phase 3 may use a different sample size or design. The key statistical question is not whether the new plan looks reasonable in isolation. It is whether, given the phase 2 data already observed, the continuation spends no more Type I error than the original confirmatory procedure had available.
Conditional error provides that remaining allowance. It is calculated under the null hypothesis from the planned test and the observed phase 2 history. A changed continuation can be valid if, under the relevant null, its conditional probability of rejection is no greater than this allowance for every possible phase 2 history. This is a way to transfer the error control of an already specified test; it does not by itself resolve multiplicity from selecting among treatments or define a new clinical estimand.
The conditioning history must include every phase 2 quantity that can affect the planned rejection rule, including a treatment-selection decision when one was part of that rule. A formula based only on a single phase 2 statistic is sufficient only when that statistic contains all relevant information for the original test’s continuation boundary.
Calculating Conditional Error for an Inverse-Normal Test
Consider a one-sided two-stage inverse-normal test. Under \(H_0\), suppose \(Z_1\) and \(Z_2\) are independent standard normal statistics from the two stages, and the original test rejects when $$ w_1Z_1+w_2Z_2\geq c, $$ where \(w_1^2+w_2^2=1\). The critical value \(c\) is fixed by the planned test; for a one-sided level \(\alpha=0.025\), a common choice is \(c=1.96\). Independence under the null is the independent-increments condition used here. The calculation also assumes that the null distribution of the future statistic is the stated standard normal distribution.
For the usual inverse-normal weights, \(w_2>0\). Given \(Z_1=z_1\), rejection requires \(Z_2\geq(c-w_1z_1)/w_2\), so the remaining conditional error is the upper-tail probability beyond that threshold. Sign matters: the formula must be changed if \(w_2\) is zero or negative.
Here \(\Phi\) is the standard normal cumulative distribution function, and \(\mathbf{1}\{\cdot\}\) equals 1 when its condition is true and 0 otherwise. For \(w_2<0\), dividing the rejection inequality by \(w_2\) reverses the inequality, giving a lower-tail probability. For \(w_2=0\), the future statistic has no influence on rejection, so the conditional rejection probability is determined entirely by \(z_1\).
Worked Example: Remaining Error After a Favorable Phase 2 Result
Suppose the original one-sided test uses \(w_1=0.6\), \(w_2=0.8\), and \(c=1.96\). These weights satisfy \(0.6^2+0.8^2=1\). The observed phase 2 statistic is \(z_1=1.20\). Calculate the remaining conditional error.
Since \(w_2=0.8>0\), use the upper-tail formula: $$ \gamma(1.20)=1-\Phi\!\left(\frac{1.96-(0.6)(1.20)}{0.8}\right). $$
The observed first-stage contribution is \((0.6)(1.20)=0.72\). Thus the required second-stage statistic is $$ Z_2\geq\frac{1.96-0.72}{0.8}=1.55, $$ and $$ \gamma(1.20)=1-\Phi(1.55)\approx 1-0.9394=0.0606. $$
Given this phase 2 result, the original test would reject with probability about 0.0606 under \(H_0\), over the still-unobserved second-stage statistic. This conditional probability can exceed the original overall level of 0.025: the favorable observed first-stage result has already used part of the evidence needed for rejection. The conditional error is not a new overall significance level to apply regardless of the phase 2 history.
Transferring the Error to a Changed Continuation
Let the proposed continuation produce a p-value \(P_{\mathrm{new}}\). For the same null hypothesis, a sufficient condition for valid transfer is that this p-value is conditionally super-uniform: for every phase 2 history \(H_1=h_1\) and every \(u\) from 0 to 1, $$ P_{H_0}(P_{\mathrm{new}}\leq u\mid H_1=h_1)\leq u. $$ If it is exactly uniform conditionally, equality holds. A straightforward transferred rule is to reject when \(P_{\mathrm{new}}\leq\gamma(H_1)\). More generally, the continuation may use any rejection rule whose conditional rejection probability is at most \(\gamma(H_1)\).
The condition is pointwise in the phase 2 history, not merely an average over histories. If the selected treatment changes, the p-value must be valid for the selected claim conditional on the selection and the other phase 2 information. If the phase 3 design changes, its null distribution must justify the conditional p-value used. A new design’s nominal p-value is not automatically conditionally valid just because it would be valid in a trial started from scratch.
Worked Example: Using the Error Bound for a Changed Phase 3 Design
Continue the previous hypothetical trial. At \(z_1=1.20\), the calculated conditional error is approximately 0.0606. Suppose the team changes the phase 3 sample size but retains the same null hypothesis and estimand. Assume the new phase 3 statistic \(Z_{\mathrm{new}}\), conditional on all phase 2 data, is standard normal under \(H_0\). The new design reports \(p_{\mathrm{new}}=1-\Phi(Z_{\mathrm{new}})\).
Because \(Z_{\mathrm{new}}\) is conditionally standard normal under \(H_0\), \(p_{\mathrm{new}}\) is conditionally uniform. The transferred rule is therefore to reject when $$ p_{\mathrm{new}}\leq 0.0606. $$ Equivalently, reject when \(Z_{\mathrm{new}}\geq\Phi^{-1}(1-0.0606)\approx1.55\). Suppose the hypothetical observed phase 3 statistic is \(1.70\). Then $$ p_{\mathrm{new}}=1-\Phi(1.70)\approx1-0.9554=0.0446. $$
Since \(0.0446<0.0606\), the transferred rule rejects \(H_0\). This conclusion depends on the stated conditional null distribution and on the p-value testing the same relevant null claim. A change in sample size alone does not establish those conditions; the new statistic and its calibration must be justified for the actual design.
Why the Transfer Controls the Overall Error
The justification is the law of total probability. Conditional on a phase 2 history, the chance of rejection by the adapted test is bounded by the original test’s conditional chance of rejection. Averaging that pointwise inequality over phase 2 histories gives the overall bound.
Under \(H_0\), suppose \(P_{H_0}(R_{\mathrm{new}}\mid H_1)\leq\gamma(H_1)\) for every possible history.
The law of total probability gives \(P_{H_0}(R_{\mathrm{new}})=E_{H_0}[P_{H_0}(R_{\mathrm{new}}\mid H_1)]\leq E_{H_0}[\gamma(H_1)]\).
By the definition of conditional error, \(E_{H_0}[\gamma(H_1)]=E_{H_0}[P_{H_0}(R_{\mathrm{old}}\mid H_1)]=P_{H_0}(R_{\mathrm{old}})\leq\alpha\).
Combining the inequalities yields \(P_{H_0}(R_{\mathrm{new}})\leq\alpha\). If the original test has exactly level \(\alpha\), and the new conditional rejection probability equals the conditional error at every history, the overall rejection probability equals that of the original test. If the new conditional probability is strictly smaller for some histories, the adapted test may be conservative.
Worked Example: Correcting the Tail for a Negative Weight
For illustration, take \(w_1=0\), \(w_2=-1\), and \(c=1\), which satisfy \(w_1^2+w_2^2=1\). The original rule is \(-Z_2\geq1\), or \(Z_2\leq-1\). For any observed \(z_1\), the conditional error is $$ \gamma(z_1)=P(Z_2\leq-1)=\Phi(-1)\approx0.1587. $$
The negative-weight formula gives the same result: $$ \Phi\!\left(\frac{1-0\cdot z_1}{-1}\right)=\Phi(-1)=0.1587. $$ Using \(1-\Phi((c-w_1z_1)/w_2)\) instead would give \(1-\Phi(-1)\approx0.8413\), the wrong tail. This example shows why the sign of the second-stage weight must be checked before applying a conditional-error formula.
Diagnostics, Alternatives, and Limitations
Before transferring conditional error, verify that the original rejection rule is fully specified, the conditioning history includes the data-dependent decisions made at the boundary, and the conditional null distribution used in the calculation is appropriate. For a continuous statistic with a known conditional distribution, compare the computed \(\gamma(h_1)\) with the conditional rejection probability implied by the proposed continuation. For example, if a conditionally uniform p-value is compared with a threshold \(g\), that probability is \(g\). If the p-value is only conservative, its rejection probability is at most \(g\).
With nuisance parameters or a composite null, a conditional calculation based on one convenient parameter value may not provide control over the whole null. The continuation must be valid for every parameter value covered by the null, or use a justified conservative bound on the conditional error. Likewise, if a selected treatment’s p-value is not conditionally valid after selection, simply comparing it with \(\gamma(H_1)\) does not repair the selection problem. The null hypothesis, selection process, and p-value calibration must align.
Conditional-error transfer and combination tests offer different ways to organize evidence across stages. A combination test specifies how stagewise evidence is combined, often through a prespecified function of stagewise p-values. Conditional-error transfer instead starts from an original valid test and asks how much conditional rejection probability remains after the observed history. A continuation can be chosen to match that bound, but the conditional distribution and claim still need justification. Neither approach makes an arbitrary unadjusted treatment selection valid.
| Approach | What is specified | Key check at the transition |
|---|---|---|
| Conditional-error transfer | The original test’s conditional rejection probability and a new continuation rule | New conditional rejection probability is no greater than \(\gamma(H_1)\) |
| Combination test | A rule combining valid stagewise evidence | Stagewise p-values and the combination rule meet their validity conditions |
| Naive nominal test after adaptation | A conventional fixed-design cutoff, applied without reference to the original rule | No general guarantee that the adaptive procedure retains its planned Type I error |
A frequent mistake is to assume that the original nominal level can simply be applied to the new phase 3 p-value. In the first worked example, a continuation threshold of 0.0606 is justified at \(z_1=1.20\), while other histories can yield a much smaller conditional error. For example, with the same weights and \(z_1=0\), the required second-stage statistic is \(1.96/0.8=2.45\), so \(\gamma(0)=1-\Phi(2.45)\approx0.00714\). A fixed continuation threshold of 0.025 would exceed that history’s allowance and could violate the transfer condition. The reverse can also occur: using 0.025 when the allowance is 0.0606 is conservative at that particular history.
Conditional error is therefore a validity device, not a complete design solution. It does not decide which treatment should be selected, whether the confirmatory claim is clinically meaningful, or how to handle multiple claims. A changed continuation must also preserve the intended estimand and have a defensible null distribution.
Check Your Understanding
Work through each question, then compare your reasoning with the solutions.
- For \(w_1=0.6\), \(w_2=0.8\), \(c=1.96\), and \(z_1=0\), calculate the conditional error.
- At a phase 2 history with conditional error 0.0606, a valid conditionally uniform new p-value is 0.080. Does the transferred rule reject? What if it is 0.040?
- For \(w_1=0.6\), \(w_2=-0.8\), \(c=1.96\), and \(z_1=0.5\), which tail applies and what is the conditional error?
- In one sentence, explain why a pointwise conditional-error bound controls the unconditional Type I error.
Solutions. (1) Since \(w_2>0\), \(\gamma(0)=1-\Phi(1.96/0.8)=1-\Phi(2.45)\approx0.00714\). (2) A p-value of 0.080 is greater than 0.0606, so do not reject; a p-value of 0.040 is smaller, so reject. (3) Because \(w_2<0\), use the lower tail: \(\gamma(0.5)=\Phi((1.96-(0.6)(0.5))/-0.8)=\Phi(-2.075)\approx0.0190\). (4) Averaging the conditional rejection probabilities over phase 2 histories bounds the overall rejection probability by the original test’s overall Type I error, which is at most \(\alpha\).