Why Conditional Error Preservation Matters
An interim result may prompt a change to a trial’s planned testing procedure. The statistical question is not simply whether the revised procedure looks reasonable: it is whether the chance of rejecting a true null hypothesis remains controlled. Conditional error provides a way to answer that question by examining the planned test separately for each possible interim result.
The key comparison is conditional. For a given interim result, the revised test must have no greater chance of rejection than the original test had at that same result. Under the assumptions below, this pointwise comparison preserves the original test’s overall null error bound. It is a sufficient condition for preservation, not the only possible way to establish overall error control.
The Preservation Theorem
The comparison applies under each null parameter value, not just under one convenient point of the null. The original test must have null rejection probability at most \(\alpha\), and the modified design must retain the same probability law for the interim information \(Y\). The latter condition ordinarily holds when the change is made after observing \(Y\): changing what happens next does not change the distribution of data already collected. The modified procedure may use a different continuation rule, but its conditional rejection probability must be evaluated under that rule.
“Almost surely” means that the comparison need only hold for interim results that occur with probability one under the specified null value. Values outside the support of \(Y\) do not affect the overall rejection probability. If the null is composite, the condition and conclusion must hold for each of its parameter values.
The justification is the conditional-probability identity: an overall rejection probability is the average of the corresponding conditional rejection probabilities over the distribution of \(Y\). Because the two designs have the same distribution for \(Y\), the pointwise bound can be averaged under that common distribution:
The first and third equalities average conditional rejection probabilities over interim outcomes. The inequality in the middle follows from \(D_\theta(y)\leq C_\theta(y)\) for almost every \(y\). Thus the modified procedure cannot have a larger null rejection probability than the original one. If the original test is conservative, the modified test is bounded by that original rejection probability, which may itself be strictly below \(\alpha\).
Worked Example: A Modified Boundary in a Two-Stage Normal Test
Worked Example: A Modified Boundary in a Two-Stage Normal Test
Consider a hypothetical two-stage test under \(H_0\). Assume the interim statistic \(X\) and the future statistic \(Z\) are independent, each has a standard normal distribution, and the original test rejects when \(0.6X+0.8Z>1.96\). Since \(0.6^2+0.8^2=1\), the combined statistic is standard normal under this null model. The original test’s rejection probability is therefore \(P(N(0,1)>1.96)\), approximately \(0.0250\), so take \(\alpha=0.025\).
Given \(X=x\), the original test rejects when \(Z>(1.96-0.6x)/0.8\). Its conditional error is
where \(\Phi\) is the standard normal cumulative distribution function. Suppose the modified test uses the stricter boundary \(0.6X+0.8Z>2.12\). Its conditional rejection probability is
For every \(x\), the modified threshold for \(Z\) exceeds the original threshold by \((2.12-1.96)/0.8=0.20\):
Because the probability that a standard normal variable exceeds a threshold decreases as the threshold increases, \(D(x)\leq C(x)\) for every \(x\). For example, at \(x=1\), the original threshold is \((1.96-0.6)/0.8=1.70\), giving conditional error approximately \(0.0446\). The modified threshold is \((2.12-0.6)/0.8=1.90\), giving conditional rejection probability approximately \(0.0287\). At this interim result, the modified conditional rejection probability is lower.
The theorem applies because the modification leaves the distribution of \(X\) unchanged and satisfies the conditional comparison everywhere. Indeed, the modified overall rejection probability is \(P(N(0,1)>2.12)\), approximately \(0.0170\), which is below the original \(0.0250\). The conditional comparison guarantees this error bound without relying only on that final calculation.
Worked Example: Checking a Conditional Error Table
Worked Example: Checking a Conditional Error Table
Suppose a hypothetical trial’s interim result falls into one of three categories, \(A\), \(B\), or \(C\). Under a specified null value, the categories have probabilities \(0.50\), \(0.30\), and \(0.20\), respectively. Assume the original test’s conditional errors and the modified test’s conditional rejection probabilities are as follows.
| Interim category | Probability under the null | Original conditional error \(C(y)\) | Modified conditional rejection probability \(D(y)\) |
|---|---|---|---|
| A | 0.50 | 0.040 | 0.030 |
| B | 0.30 | 0.100 | 0.080 |
| C | 0.20 | 0.020 | 0.020 |
The pointwise comparisons are \(0.030\leq0.040\), \(0.080\leq0.100\), and \(0.020\leq0.020\). Thus the modified conditional probability does not exceed the original conditional error in any category. The original overall rejection probability is
The modified overall rejection probability is
If the original test has level \(\alpha=0.06\), its rejection probability \(0.054\) is at most \(\alpha\), and the modified test’s rejection probability \(0.043\) is also at most \(\alpha\). The table demonstrates both the conditional comparison and how its overall consequence is obtained.
Pointwise Preservation and Direct Error Control
The theorem’s condition is sufficient for a particular preservation argument: it bounds the modified conditional rejection probability by the original conditional error at each interim result, then averages that bound. The pointwise condition is not necessary for the modified test to have level at most \(\alpha\). A test may exceed the original conditional error in one region and still have a sufficiently small overall rejection probability.
Worked Example: Overall Control Without Pointwise Preservation
Suppose a hypothetical interim result is \(A\) or \(B\), each with null probability \(0.50\). The original conditional errors are \(C(A)=0.04\) and \(C(B)=0.08\), so the original rejection probability is
Let \(\alpha=0.06\). Now suppose the modified test has \(D(A)=0.10\) and \(D(B)=0.02\). It violates the pointwise comparison in category \(A\), because \(0.10>0.04\), but its overall rejection probability is
The modified test is still controlled at level \(0.06\) for this null value. This is direct average error control under the modified design’s actual null distribution of \(Y\); it does not follow from the conditional error preservation comparison. More generally, directly establishing \(E_\theta[D_\theta(Y)]\leq\alpha\) for every null value is sufficient for level control, even if the pointwise condition fails.
Assumptions, Diagnostics, and Alternatives
A practical application should verify the ingredients of the argument rather than treating the inequality as a stand-alone guarantee. In particular, check that the conditional probabilities refer to the same null value and the same definition of interim information, that the modified probability reflects the procedure actually used after adaptation, and that the interim data retain their original null distribution. For a composite null, a comparison at only one null value does not establish control at all null values.
The unchanged interim distribution matters. For example, suppose the original interim categories \(L\) and \(H\) have null probabilities \(0.90\) and \(0.10\), and conditional errors \(C(L)=0.02\) and \(C(H)=0.30\). The original overall rejection probability is
If \(\alpha=0.05\), the original test meets its bound. Even if a modified procedure has \(D(y)=C(y)\) in both categories, a design that changes the distribution of the interim category to probabilities \(0.50\) and \(0.50\) would have average conditional rejection probability \((0.50)(0.02)+(0.50)(0.30)=0.16\), not \(0.048\). The original preservation argument cannot be applied across these different distributions. When the interim distribution changes, control must instead be established under the modified design’s actual distribution, for example by a direct calculation of its rejection probability.
There are two common routes to error control. Conditional error preservation offers a result that holds separately at each interim outcome, and can therefore be useful when the continuation procedure changes after that outcome. Direct recalibration or direct evaluation of the modified test’s overall rejection probability can also establish level control, but an overall calculation alone does not show that the original conditional error was preserved. Either route needs correct null distributions and must address every null parameter value covered by the claim.
Common Mistakes and Limitations
- Checking only an average against the original test without specifying its distribution. The preservation proof averages both conditional quantities under the same null distribution of \(Y\). An average under some other distribution does not establish the pointwise comparison or its preservation conclusion.
- Claiming the pointwise condition is necessary for all size control. It is sufficient for this preservation argument. A modified test can violate it at some interim outcomes and still have an overall rejection probability no greater than \(\alpha\), as the preceding example shows.
- Comparing the wrong conditional probabilities. The modified probability must correspond to the modified procedure that would actually be run after \(Y=y\), not to a planned rule that is no longer used.
- Ignoring the interim-data law. If the modification changes how \(Y\) is generated, averaging the original conditional error under the original design does not automatically bound the modified test’s rejection probability.
- Checking only one null value. A level-\(\alpha\) conclusion for a composite null requires the relevant bounds for every null parameter value, not merely for a selected value.
The theorem gives a robust sufficient condition, but it does not by itself explain how to construct a modified procedure satisfying the condition, nor does it guarantee that the resulting test has good power. Its conclusion concerns null rejection probabilities. The comparison can also be conservative: a modified test may spend less conditional error than the original test allows at many interim results.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- State the conditional error preservation condition using \(C_\theta(y)\) and \(D_\theta(y)\).
- If the original test’s rejection probability is \(0.03\) and the pointwise conditional comparison holds, what upper bound follows for the modified test’s rejection probability?
- For two equally likely interim categories, the original conditional errors are \(0.02\) and \(0.08\), while the modified conditional rejection probabilities are \(0.05\) and \(0.03\). Does the pointwise preservation condition hold? What is the modified overall rejection probability?
- Why does an average of \(D_\theta(Y)\) under a distribution different from the modified design’s actual null distribution fail to establish the modified test’s rejection probability?
- Can the modified test have controlled overall error if \(D_\theta(y)>C_\theta(y)\) for some interim outcomes? Explain.
Solutions. (1) For every null value \(\theta\), require \(D_\theta(Y)\leq C_\theta(Y)\) almost surely under the null distribution of the interim information, with the original and modified designs sharing that distribution. (2) The theorem gives \(P_\theta(\text{modified test rejects})\leq0.03\), and hence also at most \(\alpha\) if the original test’s level bound is \(\alpha\). (3) The condition fails in the first category because \(0.05>0.02\). The modified overall probability is \((0.50)(0.05)+(0.50)(0.03)=0.04\). (4) The rejection probability is the average under the modified design’s actual distribution of \(Y\); averaging under a different distribution need not give that probability. (5) Yes. The modified test may still have overall rejection probability at most \(\alpha\) if its conditional probabilities, averaged under its actual null distribution, are at most \(\alpha\). That is direct error control, not pointwise conditional error preservation.