Why a Design Change Needs a Conditional Constraint
After interim data have been observed, a trial team may change its planned sample size, test statistic, or other design features. The revised procedure must account for what the interim data have already contributed to the chance of rejecting \(H_0\). Controlling the rejection probability only on average over all possible interim data is not enough to justify a rule that makes different decisions in different interim histories.
The relevant quantity is the conditional error: the original design’s probability of eventual rejection, conditional on the interim information already observed. A revised continuation rule preserves Type I error control if, at every relevant interim history, its conditional null rejection probability is no larger than that original conditional error.
Here \(P_0\) denotes probability under the null model being considered. The constraint applies to the histories at which the trial continues. If the original design has already stopped and made a decision, that decision is not replaced by a later continuation rule. For a simple null, the constraint is evaluated under that null distribution. For a composite null, it must be justified for every null parameter value for which control is claimed.
A design selected using interim data is part of the conditioning information: the revised plan must be determined by \(\mathcal{F}_1\), and its conditional null distribution must be valid given that information and the chosen plan. This allows the future rule to change without treating the interim data as if they had never been observed.
Why the Constraint Preserves Type I Error
The justification follows from the law of total probability, also called the tower property of conditional expectation. The chance that the revised procedure rejects is the average of its conditional rejection probabilities over interim histories. If none of those conditional probabilities exceeds the original conditional error, their average cannot exceed the original design’s rejection probability.
Write the revised rejection probability as \(P_0(B)=E_0[P_0(B\mid\mathcal{F}_1)]\).
By the design constraint, \(P_0(B\mid\mathcal{F}_1)\leq a(\mathcal{F}_1)\) almost surely.
Taking null expectations gives \(P_0(B)\leq E_0[a(\mathcal{F}_1)]\).
Since \(a(\mathcal{F}_1)=P_0(A\mid\mathcal{F}_1)\), the tower property gives \(E_0[a(\mathcal{F}_1)]=P_0(A)\).
Thus a revised procedure cannot have a greater null rejection probability than the original procedure. If the original procedure has Type I error at most \(\alpha\), so does the revised procedure. If the revised rule uses the entire conditional budget at every history, then its overall rejection probability equals the original one, provided the conditional probabilities are well-defined and the bound is attained.
Turning a Conditional Budget Into a Rejection Rule
A direct way to meet the constraint is to calibrate the revised test’s critical value using its conditional null distribution. Suppose that, given \(\mathcal{F}_1\) and the selected plan, a revised statistic \(T\) has continuous conditional null cumulative distribution function \(F_0(t\mid\mathcal{F}_1)\). For an upper-tail test, rejecting when \(T\geq c(\mathcal{F}_1)\) gives conditional rejection probability \(1-F_0(c(\mathcal{F}_1)\mid\mathcal{F}_1)\). Choosing the cutoff from the conditional-error budget yields the rule below.
For a continuous distribution, this rule spends exactly \(a(\mathcal{F}_1)\) conditional error. A more conservative cutoff can spend less. If the conditional null distribution is discrete, a cutoff alone may not attain the budget exactly; the rejection probability must still be no greater than the budget, with randomization only if the procedure explicitly permits it. When \(a(\mathcal{F}_1)=0\), the rule must have zero conditional rejection probability; when it is 1, the conditional constraint places no additional restriction.
This quantile construction is one way—not the only way—to satisfy the conditional-error constraint. The revised statistic can differ from the original test, but its null distribution must be valid conditional on the interim information and the selected continuation plan. A statistic that is nominally calibrated without that conditioning does not establish the required bound.
Worked Example: Spending a Conditional Error Budget
Suppose a hypothetical one-sided trial has observed interim information for which the original design’s conditional error is \(a(\mathcal{F}_1)=0.012\). A proposed revised continuation uses a statistic \(T\) that, under \(H_0\), has a standard normal distribution conditional on this interim information. The revised rule rejects for large values of \(T\).
The conditional null distribution is \(F_0(t\mid\mathcal{F}_1)=\Phi(t)\), where \(\Phi\) is the standard normal cumulative distribution function. The cutoff that uses the full budget is $$ c=\Phi^{-1}(1-0.012)=\Phi^{-1}(0.988)\approx 2.257. $$
Thus the revised rule rejects when \(T\geq 2.257\). Its conditional null rejection probability is $$ P_0(T\geq 2.257\mid\mathcal{F}_1) =1-\Phi(2.257)\approx 0.012. $$
The required conditional bound is met at equality. This calculation is valid only under the stated assumption that the revised statistic is standard normal under the null conditional on the interim information and the revised plan. An unconditional standard normal approximation would not, by itself, verify that assumption.
Worked Example: Different Histories, Different Budgets
Consider a hypothetical design with two possible interim-history groups, \(G_1\) and \(G_2\). Under \(H_0\), suppose \(P_0(G_1)=0.40\) and \(P_0(G_2)=0.60\). The original design has conditional errors 0.040 on \(G_1\) and 0.015 on \(G_2\). Assume the revised statistic is standard normal conditional on either group, and use an upper-tail rule that spends each group’s full budget.
In group \(G_1\), the cutoff is $$ \Phi^{-1}(1-0.040)=\Phi^{-1}(0.960)\approx 1.751, $$ so the conditional rejection probability is 0.040. In group \(G_2\), the cutoff is $$ \Phi^{-1}(1-0.015)=\Phi^{-1}(0.985)\approx 2.170, $$ so the conditional rejection probability is 0.015.
The overall null rejection probability is the probability-weighted average: $$ P_0(B)=(0.40)(0.040)+(0.60)(0.015)=0.016+0.009=0.025. $$
The original overall rejection probability is also \((0.40)(0.040)+(0.60)(0.015)=0.025\). The revised rule therefore preserves the original error rate in this example, while using a less stringent cutoff in histories with a larger conditional budget and a more stringent one in histories with a smaller budget. The group probabilities and budgets are hypothetical and are assumed to describe the stated null model.
Worked Example: A Nominal Stage-Two Test Can Overspend
Suppose an interim history leads to a revised plan whose original conditional error budget is 0.010. Under the null, assume the revised one-sided statistic \(T\) is standard normal conditional on that history and plan. A team proposes to use the usual one-sided 0.025 cutoff, \(1.960\), without accounting for the remaining budget.
The proposed conditional rejection probability is $$ P_0(T\geq 1.960\mid\mathcal{F}_1)=1-\Phi(1.960)\approx 0.025. $$ Since \(0.025>0.010\), this rule violates the conditional-error constraint. Its nominal level is not justified merely because it is a familiar test level.
To use the full available budget, the cutoff must instead be $$ \Phi^{-1}(1-0.010)=\Phi^{-1}(0.990)\approx 2.326. $$ Then \(P_0(T\geq 2.326\mid\mathcal{F}_1)\approx 0.010\), meeting the bound. This conclusion depends on the assumed conditional standard normal distribution; if that conditional model is wrong, the numerical cutoff need not control the conditional rejection probability.
Assumptions, Diagnostics, and Alternatives
The conditional-error argument is distributional, not merely algebraic. Before using it, verify that the interim information is correctly represented by \(\mathcal{F}_1\), that the original conditional error is defined under the null model of interest, and that the revised plan is selected using only the information allowed at the interim. Most importantly, the conditional null distribution used to calibrate the revised rule must remain valid after conditioning on both the interim history and the selected plan.
Independent increments and stagewise independence, covered in the earlier material on those subjects, can help justify conditional distributions for future information. They do not automatically validate every statistic or selection rule. If the choice of plan depends on information beyond the stated interim data, or if the proposed conditional distribution ignores a dependence induced by selection, the resulting cutoff may overspend the budget.
Useful checks include calculating or simulating the revised conditional rejection probability across representative interim histories, comparing it with the conditional budget at each history, and verifying that the revised overall rejection probability does not exceed the original design’s. A check of only the overall average can conceal histories where the conditional bound is violated. Simulation is a diagnostic, not a substitute for a valid conditional null model or a proof of the bound.
The main alternatives answer somewhat different questions. Retaining the original design and its original rejection rule avoids recalibrating a changed continuation, but may not accommodate the planned modification. A combination test instead combines stagewise evidence using a prespecified valid reference distribution; it is a separate route to Type I error control and does not remove the need to satisfy its own validity conditions. Conditional-error preservation is more general about the form of the revised continuation, but it requires knowing or controlling the original conditional error and establishing the revised rule’s conditional null behavior.
The method also has practical limitations. Conditional error can be difficult to evaluate for complicated original designs, and the revised conditional null distribution may be hard to derive. Approximate calculations must be accurate enough for the claimed error control. Finally, preserving Type I error does not guarantee adequate power, unbiased effect estimation, or operational feasibility; these are separate design considerations.
Check Your Understanding
Answer each question using the conditional-error constraint. Solutions follow the questions.
- If \(P_0(A)=0.025\) and a revised rule satisfies \(P_0(B\mid\mathcal{F}_1)\leq P_0(A\mid\mathcal{F}_1)\) almost surely, what bound follows for \(P_0(B)\)?
- For an upper-tail statistic that is conditionally standard normal and a conditional error budget of 0.020, give the cutoff formula and approximate cutoff.
- Why is it insufficient to check only that the revised test has an unconditional rejection probability below \(\alpha\) in one simulated set of interim histories?
- At a particular interim history, the conditional budget is 0.008, but a proposed rule has conditional null rejection probability 0.012. Does the rule satisfy the constraint? What must change?
Solutions. (1) The tower-property argument gives \(P_0(B)\leq P_0(A)=0.025\). (2) The cutoff is \(\Phi^{-1}(1-0.020)=\Phi^{-1}(0.980)\approx 2.054\); rejecting above it spends 0.020 conditionally. (3) An average can mask histories at which the conditional rejection probability exceeds the original budget; the conditional bound must hold almost surely under the null, and simulation alone does not prove it. (4) No: \(0.012>0.008\). The revised rule must be made more stringent or otherwise changed so its conditional null rejection probability is at most 0.008.