Proof of Conditional Error Preservation

Why a State-by-State Bound Matters

An interim analysis divides a trial into information already observed and outcomes yet to be collected. The original test may have very different probabilities of eventual rejection at different interim results. Its conditional error records that probability at the observed interim state. If a revised continuation rule uses no more rejection probability at each state than the original test had available there, the unconditional Type I error cannot increase. The reason is an averaging identity, but the comparison must hold state by state—not just on average.

This result is about preserving the error rate of a test under a null hypothesis. It does not establish that an adapted test has good power, nor does it make a test valid if the original test was invalid. The original test's null error control and the validity of the conditional probability calculations are essential.

For example, if the interim information is summarized by a statistic \(Z\), these quantities may be written as functions \(e_\theta(Z)\) and \(q_\theta(Z)\). The value \(e_\theta(z)\) is the original test's probability of eventual rejection, conditional on interim state \(Z=z\). The value \(q_\theta(z)\) is the adapted test's corresponding probability. The conditional error is not generally the nominal level \(\alpha\) at every interim state.

The Preservation Result

The comparison required for preservation is \(q_\theta\leq e_\theta\) with probability one under the null parameter \(\theta\). “With probability one” allows the inequality to fail only on interim states that have probability zero under that parameter. The original test must also satisfy \(P_\theta(R_0)\leq\alpha\). The interim data must have the same distribution before the adaptation in the original and adapted procedures, so that both conditional quantities are averaged over the same interim-data distribution.

The proof uses the defining property of conditional probability: the expectation of a conditional rejection probability is the unconditional probability of rejection. Since the pointwise bound holds almost surely, taking expectations preserves the inequality.

The first and fifth equalities are the law of total probability expressed using conditional expectation. The inequality follows because \(q_\theta\leq e_\theta\) almost surely. Thus the adapted test's null rejection probability is at most the original test's null rejection probability, which is at most \(\alpha\). Applying the argument at each null parameter value gives the claimed control across a composite null.

A Normal-Increments Example

The same comparison can be made when interim information is continuous. Consider a hypothetical one-sided test under a simple null hypothesis. Assume the cumulative statistic at information fraction \(t\), where \(0<t<1\), is \(Z_1\sim N(0,t)\), and the future increment \(Z_2\sim N(0,1-t)\) is independent of \(Z_1\). The final statistic is \(Z_1+Z_2\), which has a standard normal distribution under the null. These distributional and independence assumptions justify the conditional calculation below.

Suppose the original test rejects when \(Z_1+Z_2>c\), with \(c=1.96\), approximately the one-sided \(0.025\) critical value. Given \(Z_1=z\), rejection requires \(Z_2>c-z\). Since \(Z_2/\sqrt{1-t}\) is standard normal, the original conditional error is

where \(\Phi\) is the standard normal cumulative distribution function. Now let the adapted rule reject when \(Z_1+Z_2>c+\delta(z)\), where \(\delta(z)\geq0\) for every interim state. Its conditional rejection probability is

Because \(\delta(z)\geq0\) and \(\Phi\) is nondecreasing, the argument of \(\Phi\) in \(q(z)\) is at least the argument in \(e(z)\). Subtracting from one reverses that comparison, so \(q(z)\leq e(z)\) at every state. The preservation result therefore applies.

Why an Average Bound Is Not Enough

A common error is to compare only the average conditional rejection probabilities and assume that this verifies conditional error preservation. An average comparison can sometimes establish an unconditional error rate directly, if it is calculated under the correct null distribution. But it is not the statewise condition in the theorem, and it does not establish that each interim state respects its original conditional error budget. In adaptive settings, statewise protection is important precisely because the continuation procedure is chosen in light of the interim state.

Assumptions, Diagnostics, and Alternatives

The proof is short, but its assumptions carry the substance of the result. In particular, the original test's error control must be valid under the null, the interim distribution used for averaging must be correct, and the adapted conditional rejection probability must describe the procedure that will actually be run. If the null is composite, checking the argument at only one convenient null parameter does not establish control for the whole null.

Useful diagnostics follow directly from the proof:

• Check the comparison by interim state. For discrete interim outcomes, list \(q_\theta(z)\) and \(e_\theta(z)\) for each possible state. For continuous outcomes, justify the inequality as a function of the interim data, as in the normal example.
• Check the null distribution used for integration. The weights or density for interim states must be those under the null parameter being assessed. A correct conditional inequality combined with incorrect weights does not produce a valid error calculation.
• Check what the conditional probability includes. It must account for the actual continuation test and all of its possible future outcomes, not only a selected estimate or an expected test statistic.
• Check the original test's size. The theorem transfers its error bound; it cannot repair an original test whose null rejection probability exceeds \(\alpha\).

One alternative is to derive the adapted procedure's unconditional null rejection probability directly, integrating its rejection probability over interim and future data. That can be appropriate when the full adapted design has a tractable null distribution. Conditional error preservation instead offers a statewise comparison to an already valid test. Simply reusing the original nominal level \(\alpha\) for a new continuation test is not an equivalent argument: \(\alpha\) is an overall error bound, whereas the original test's available error can vary across interim states.

The result is limited to Type I error control under the null configurations for which the conditional inequality and original test guarantee have been established. It does not show that the adapted test is more powerful, that its conditional bound has been computed correctly, or that a particular adaptation is scientifically or operationally appropriate. Those questions require additional arguments.