Why a Conditional Bound Allows Adaptation
An interim review may make it reasonable to change how a trial continues. The continuation procedure might use a different rejection rule or otherwise respond to the interim data. The statistical question is whether such a change can retain the original test’s type I error control: the probability of rejecting a true null hypothesis must not exceed its specified level \(\alpha\).
The key comparison is conditional, not merely overall. For each interim data outcome, compare the adapted procedure’s chance of rejecting under the null with the original procedure’s chance of rejecting, conditional on that same interim outcome. If the adapted chance never exceeds the original conditional chance, averaging over the interim data cannot increase the overall rejection probability. This is a sufficient guarantee; it does not say that every adaptation satisfying it must use the same test or rejection rule as the original.
This definition covers a procedure with interim decisions as well as one that always continues. If the original design has already rejected for a particular interim outcome, its conditional probability of eventual rejection is 1 at that outcome. An adaptation that changes only the continuation must leave decisions already made intact. More generally, the guarantee below compares the conditional probabilities of the complete final rejection events.
The Conditional Error Preservation Result
Suppose the original test has type I error at most \(\alpha\) under \(H_0\). An adapted procedure preserves that control if it satisfies the conditional bound under the null:
The inequality must hold for almost every interim outcome under the null, not just for the average of a selected collection of outcomes. Its justification uses the law of total probability, expressed for conditional expectations by the tower property.
The first and third equalities follow because the expectation of a conditional probability equals the probability of the event. The inequality between them follows from the conditional bound: taking expectations preserves an almost-sure inequality. Thus, a pointwise conditional comparison is enough to establish the unconditional type I error guarantee.
For a composite null, the guarantee must hold under every null distribution relevant to the claim. In particular, if \(P_\theta\) denotes a distribution with parameter \(\theta\in H_0\), require \(P_\theta(R_0)\leq\alpha\) and the conditional inequality \(P_\theta(R_A\mid\mathcal F)\leq P_\theta(R_0\mid\mathcal F)\) almost surely under each such \(P_\theta\). A bound verified under only one null parameter does not by itself establish control over the whole composite null.
Worked Example: Comparing Conditional Errors Across Interim Outcomes
Suppose a hypothetical test has two possible interim outcomes, \(L\) and \(H\). Under \(H_0\), \(P_0(L)=0.40\) and \(P_0(H)=0.60\). The original test’s conditional rejection probabilities are 0.08 after \(L\) and 0.03 after \(H\). An adapted procedure has conditional rejection probabilities 0.07 and 0.02, respectively. Find the original and adapted type I error rates.
The original overall error rate is the probability-weighted average of its conditional errors:
The adapted conditional error is below the original one at each interim outcome: \(0.07\leq0.08\) after \(L\), and \(0.02\leq0.03\) after \(H\). Its overall error rate is
The adaptation therefore has type I error 0.040, or 4.0%, under this null distribution, compared with 5.0% for the original procedure. The reduction is not required: the theorem also allows equality in the conditional bound, which would preserve the original error rate exactly.
What Makes an Adaptation “Arbitrary”
The conditional-bound argument does not require the adapted continuation to have the same form as the original continuation. It requires a valid comparison of their conditional rejection probabilities. The adaptation may use interim information to select among procedures, provided that the selection rule is determined by that information and that the resulting procedure respects the bound for each relevant interim outcome.
Formally, an adaptation rule maps the observed interim information to a continuation procedure. For the conditional argument to apply, this rule must be measurable with respect to \(\mathcal F\): knowing the interim information must determine the selected procedure. The conditional probabilities must also be calculated under the null distribution for the procedure actually selected. A procedure that appears valid when considered before selection may not be valid after selection if its conditional rejection probability was not checked for the selection rule.
Worked Example: Selecting Between Two Continuation Procedures
In a hypothetical trial, the interim information is summarized by outcome \(L\) or \(H\), each occurring with null probability 0.50. The original procedure’s conditional errors are 0.06 after \(L\) and 0.02 after \(H\). Two candidate continuation procedures, A and B, have the following null conditional rejection probabilities:
| Interim outcome | Original conditional error | Procedure A | Procedure B |
|---|---|---|---|
| \(L\) | 0.060 | 0.040 | 0.055 |
| \(H\) | 0.020 | 0.015 | 0.019 |
Consider the rule: choose procedure B after either outcome. This is a measurable rule because the choice is determined by the interim information. After \(L\), procedure B’s conditional error is \(0.055\leq0.060\); after \(H\), it is \(0.019\leq0.020\). Thus the rule respects the original conditional error bound. The original overall type I error is
For the adapted rule, the overall error is
The adapted procedure has type I error 3.70%, below the original 4.00%. Procedure B is more permissive than A at both outcomes, but it still meets the conditional bound. The guarantee depends on the selected procedure’s conditional error, not on whether its rule resembles the original test.
Why an Average Bound Is Not the Same
A common mistake is to check only an overall or average error calculation without ensuring that the proposed conditional bounds have been correctly evaluated. The theorem provides a convenient sufficient condition: control the adapted conditional error at every relevant interim outcome, then average. If the condition fails at some outcomes, the theorem no longer guarantees control. The failure does not prove that the overall error must exceed \(\alpha\), but it means that some other valid argument is needed.
Worked Example: Exceeding the Bound Can Increase Type I Error
Suppose a hypothetical original test has two interim outcomes, each with null probability 0.50, and conditional error 0.05 at each outcome. Its overall type I error is
Now suppose an adapted procedure has conditional rejection probability 0.12 after the first outcome and 0.02 after the second. At the first outcome, it exceeds the original conditional bound \(0.12>0.05\). Its overall type I error is
The adapted test has a 7.0% type I error rate, exceeding the original 5.0% rate. This example shows why one cannot treat the conditional bound as optional. It does not imply that every violation inflates error: a sufficiently small rejection probability elsewhere could still keep the overall rate below 5.0%. It does show that the conditional-preservation theorem would not establish control for this adaptation.
Assumptions, Diagnostics, and Alternatives
Before relying on the result, check the parts that make the conditional comparison meaningful:
- Specify the null distribution. Conditional errors are calculated under \(H_0\). For composite nulls, establish the guarantee for every relevant null parameter, not just a convenient value.
- Use the same interim information. The original and adapted conditional rejection probabilities must be conditioned on the information available when the change is made.
- Account for the selected procedure. If the rule selects a continuation using interim data, evaluate the conditional rejection probability after that selection. Do not substitute the error rate of a procedure averaged over selections that would not occur at the observed interim outcome.
- Check the inequality pointwise. Verify that the adapted conditional error is no larger than the original conditional error for each relevant interim outcome, or establish the corresponding almost-sure inequality for continuous interim data.
- Keep decisions and data definitions consistent. The events compared must represent rejection of the same null hypothesis under the stated design and use the same conditioning information.
A practical diagnostic is to tabulate, plot, or otherwise calculate the original conditional error and the adapted conditional error as functions of the interim data. Look for interim regions where the adapted value exceeds the original one. In a simulation, estimate both quantities within well-defined interim strata under the null; a favorable overall simulated rate alone does not verify the pointwise condition. If conditional probabilities come from a model or approximation, assess whether the model correctly represents the null distribution of the future data given the interim data.
The main alternative is to design and calibrate the adapted procedure directly so its unconditional type I error is controlled. That may be valid, but it requires its own argument and can be difficult when many interim outcomes lead to different procedures. Conditional error preservation instead supplies a transparent sufficient route: the original procedure provides an outcome-specific error budget, and the adaptation stays within it. The route can be conservative because a procedure may exceed the original bound for some interim outcomes yet still have an acceptable overall error rate. Such a procedure is not automatically invalid; it simply needs another proof of control.
The conditional bound is only as reliable as its inputs. A calculation that uses the wrong conditional distribution, overlooks the way a procedure was selected, or addresses only part of a composite null cannot establish the stated guarantee. When the assumptions are not credible, do not report the theorem’s conclusion as if it had been verified.
Check Your Understanding
Answer each question using the conditional error argument. Brief solutions follow the questions.
- Under \(H_0\), an interim outcome \(L\) has probability 0.25 and \(H\) has probability 0.75. The original conditional errors are 0.08 and 0.04. What is the original overall type I error?
- For the setting in Question 1, an adapted rule has conditional errors 0.06 and 0.04. Does it satisfy the conditional bound, and what is its overall type I error?
- Why does a rule that chooses a continuation procedure based on interim data need to be evaluated after applying the selection rule?
- For a composite null, is verifying the conditional inequality under one null parameter always sufficient? Explain.
Solutions. 1. The original error is \((0.25)(0.08)+(0.75)(0.04)=0.020+0.030=0.050\), or 5.0%. 2. Yes: \(0.06\leq0.08\) and \(0.04\leq0.04\). Its overall error is \((0.25)(0.06)+(0.75)(0.04)=0.015+0.030=0.045\), or 4.5%. 3. Selection can change the conditional rejection probability; the relevant bound is for the procedure that is actually selected at that interim outcome. 4. No. Unless the null is simple or additional conditions justify the extension, the guarantee must be checked under every relevant null parameter.