Why Conditional Error Gives a Design-Level Guarantee
After interim data are observed, a trial’s remaining design may be changed: for example, the continuation rule may be modified. The relevant question is not whether the new rule looks more conservative overall, but whether its chance of eventual rejection, given each interim history, is controlled. A bound on that conditional chance can be turned into a bound on the overall Type I error by averaging over histories.
The result is useful because it separates two tasks. First, specify how much rejection probability the original design allocated after each possible history. Then ensure the revised continuation has no greater conditional rejection probability at any history that can occur under the null. This tutorial proves that this condition is sufficient; it does not require a particular form for the revised test.
For histories at which the design continues, \(C(z)\) is the original design’s conditional chance of eventual rejection. Histories at which rejection or futility has already occurred can be included as well: the corresponding conditional probability is 1 for prior rejection and 0 for a path that cannot reject. Including these outcomes makes the definitions apply to the complete design.
The notation \(P_0\) represents a simple null distribution. For a composite null hypothesis, the argument must hold under every null distribution \(P_\theta\), with \(\theta\) in the null parameter space. In either case, conditional probabilities are defined up to sets of probability zero, so the required inequality need only hold for almost every history under the null distribution.
Assumptions and the Conditional Error Principle
The key requirement is that the interim history have the same distribution under the null whether the original or revised continuation is used. This is ordinarily the case when the same interim data are collected before the adaptation and only the future design changes. The revised rule may depend on \(Z\), but the distribution of \(Z\) itself must not be changed by that rule.
The conditional probabilities must also be computed under the relevant null model and with the actual revised procedure, including its stopping and rejection rules. If the test is randomized, the probabilities include the randomization. These requirements matter: a bound evaluated under the wrong conditional model, or under a different distribution of interim histories, does not establish the result.
The statement is pointwise in the history, not merely an assertion that the revised design has a smaller average conditional error under some other distribution. The conditional probabilities are compared at the same \(z\), and then averaged over the common null distribution of \(Z\).
Why the Bound Holds
The proof is an application of the law of total probability, equivalently the defining property of conditional expectation. Given the interim history, the conditional probability of eventual rejection is \(C(Z)\) for the original design and \(Q(Z)\) for the revised design. Averaging each conditional probability recovers the corresponding unconditional rejection probability.
The inequality follows because \(Q(Z)\leq C(Z)\) almost surely, so the expectation of \(C(Z)-Q(Z)\) is nonnegative. The common distribution of \(Z\) is essential to this comparison: both expectations are taken over the same null distribution. If the original design has \(P_0(R_{\mathrm{orig}})\leq\alpha\), transitivity gives \(P_0(R_{\mathrm{new}})\leq\alpha\).
The difference in error rates can also be written directly:
This identity shows that any unused conditional error contributes to a reduction in overall rejection probability, averaged over the null distribution of interim histories. It does not say that the revised test must have exactly the original Type I error. A strict conditional inequality on histories with positive probability can make the revised test more conservative.
Worked Example: Averaging Conditional Errors Over Discrete Histories
Suppose a hypothetical interim assessment has three possible histories, \(z_1,z_2,z_3\), with null probabilities \(0.40, 0.35, 0.25\). The original design’s conditional errors are \(0.02, 0.08, 0.16\), respectively. A proposed revision has conditional rejection probabilities \(0.01, 0.08, 0.12\). Assume these are the complete possible histories and their probabilities are unchanged by the revision.
The original Type I error is the probability-weighted sum of its conditional errors:
At each history, the revised conditional rejection probability is no larger than the original conditional error: \(0.01\leq0.02\), \(0.08\leq0.08\), and \(0.12\leq0.16\). Therefore,
The revised Type I error is \(0.062\), below the original value of \(0.076\). In particular, if the original design was controlled at \(\alpha=0.08\), the revised design remains below that level. The calculation uses the same history probabilities for both designs and checks the conditional bound separately for every history.
A Conditional-Normal Example
The conditional error need not be tabulated directly. It may be calculated from the conditional distribution of a future statistic. Consider a hypothetical one-sided test with a standardized future statistic \(W\). Suppose that, under the null and conditional on either interim stratum, \(W\sim N(0,1)\). In stratum \(z_1\), the original design rejects when \(W>1.6449\); in stratum \(z_2\), it rejects when \(W>1.2816\). The respective original conditional errors are approximately \(0.05\) and \(0.10\), since these are the upper-tail probabilities of a standard normal distribution.
Worked Example: Comparing Conditional Normal Tails
Suppose \(P_0(Z=z_1)=0.70\) and \(P_0(Z=z_2)=0.30\), and the revised design uses the threshold \(W>1.9600\) in both strata. Verify the conditional-error condition and calculate both overall rejection probabilities.
Under the stated conditional null model, the revised conditional rejection probability in either stratum is \(P_0(W>1.9600\mid Z=z_i)=1-\Phi(1.9600)\approx0.0250\), where \(\Phi\) is the standard normal cumulative distribution function. Thus \(0.0250\leq0.05\) in \(z_1\), and \(0.0250\leq0.10\) in \(z_2\). The conditional bound holds in both strata.
The original overall Type I error is
The revised overall Type I error is
So the revised rule has a smaller null rejection probability under these assumptions. The thresholds, conditional distributions, and stratum probabilities are all part of the example’s specified model; if any changes, the conditional probabilities must be recalculated.
When the Conditional Bound Fails
The condition is sufficient, not necessary. If a revised conditional probability exceeds the original conditional error for some histories, the theorem no longer guarantees control. That alone does not prove the overall Type I error exceeds \(\alpha\): decreases at other histories might offset the excess. But without the pointwise bound, control must be established by some other valid argument rather than inferred from this principle.
Worked Example: A History With Excess Conditional Error
Suppose two interim histories have null probabilities \(0.60\) and \(0.40\). The original conditional errors are \(0.02\) and \(0.095\), so the original Type I error is
Now suppose the revised conditional rejection probabilities are \(0.02\) and \(0.12\). The second exceeds the original conditional error, because \(0.12>0.095\). The revised Type I error is
Here the Type I error increases from \(0.050\) to \(0.060\). This example shows the possible consequence of spending more than the original conditional error at a history with positive null probability. It does not mean every violation necessarily increases the overall error; it means the conditional error guarantee cannot be invoked.
Diagnostics, Alternatives, and Limitations
A useful diagnostic is to list, or otherwise evaluate, \(C(z)\) and \(Q(z)\) under the null across the histories that can occur. For a discrete history space, check every history with positive null probability. For a continuous history, verify the inequality under the specified conditional model, at least almost surely; checking only a few simulated histories does not prove a bound for all relevant histories. Simulation can help identify coding errors or regions of concern, but its finite-sample results are not themselves a proof of pointwise domination.
A common mistake is to compare only the two unconditional averages, or to reason that a revised design is safe because it is more conservative for a typical interim result. The correction is to compare conditional rejection probabilities history by history, under the null, and then integrate using the common distribution of those histories. A second mistake is to use the original history probabilities after changing the interim data collection or analysis in a way that changes their null distribution. In that case the expectation step in the proof does not follow as written.
Compared with retaining the original design unchanged, conditional-error control permits changes to the continuation while preserving a formal Type I error bound. Compared with recalibrating a new rule by calculating its overall null rejection probability, it gives a local, history-specific sufficient condition. The latter approach may establish control even when the pointwise condition is not met, but it requires a valid overall calculation under the actual adaptive procedure. Neither approach excuses incorrect null modelling or an unaccounted change in the interim-data distribution.
The principle is a guarantee about Type I error, not a guarantee about power, expected sample size, or other operating characteristics. It also does not specify how to calculate conditional error from a particular original design. Those are separate design and evaluation tasks. For a composite null, the conditional comparison and the original size bound must be valid for every parameter value in the null space; checking only one convenient null value is not enough.
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- An original design has conditional errors \(0.04\) and \(0.10\) for histories with null probabilities \(0.75\) and \(0.25\). A revised design has conditional probabilities \(0.03\) and \(0.08\). Calculate both Type I errors and state whether the conditional-error condition holds.
- Why must the original and revised designs use the same null distribution of interim history for the theorem’s expectation comparison?
- If \(Q(z)>C(z)\) on a history that has positive null probability, does that alone prove the revised Type I error exceeds the original one? Explain.
- For a composite null, what must be checked for the conditional error argument to establish control over the whole null hypothesis?
Solutions. (1) The original error is \((0.75)(0.04)+(0.25)(0.10)=0.030+0.025=0.055\). The revised error is \((0.75)(0.03)+(0.25)(0.08)=0.0225+0.0200=0.0425\). The conditional bound holds in both histories, and the revised error is smaller. (2) The proof averages both conditional probabilities over the same distribution of histories; if that distribution changes, the pointwise comparison alone does not compare the resulting expectations. (3) No. An excess at one history may be offset by reductions elsewhere, but the conditional-error theorem no longer certifies control. (4) The relevant conditional domination and the original Type I error bound must hold under every null parameter value, not only one selected value.