Why distinguish exact from conservative preservation?
An adaptive test can change its continuation rule after interim data are observed. The key constraint is not necessarily that the new rule reproduces the original decision boundary. Rather, at each interim result, its probability of ultimately rejecting under the null must be no larger than the original test’s conditional probability of rejection. That original conditional probability is the conditional error.
Spending that conditional error in full preserves the original test’s unconditional type I error. Spending less at some interim results can lower the unconditional error rate. The distinction matters because “the conditional error is preserved” may mean either equality or merely that the new procedure stays within the original conditional budget. Those claims have different consequences.
“Almost surely” means that the equality or inequality holds for all interim outcomes except possibly a set with probability zero under the null. Conditional error is a probability, so each value lies between 0 and 1. It is a budget attached to the observed interim information, not a new type I error level chosen independently at each interim result.
Conditions for the unconditional bound
The comparison is meaningful when the original and adapted procedures are evaluated under the same null hypothesis and the same probability law for the interim information. The conditioning information must include the data used to choose the continuation rule. Conditional probabilities must account for the actual continuation design and its null distribution. If the adapted test uses randomization, that randomization must also be included in the probability calculation.
For a simple null, suppose the original test has type I error \(\alpha_0=P_0(R_0=1)\). The key result follows from the law of total expectation. It does not require the new rule to have the same rejection boundary or the same conditional power under alternatives; the claim here concerns type I error under the stated null.
By definition, \(C_1=E_0(R_1\mid\mathcal{F})\) and \(C_0=E_0(R_0\mid\mathcal{F})\).
The law of total expectation gives \(E_0(R_1)=E_0\{E_0(R_1\mid\mathcal{F})\}=E_0(C_1)\), and likewise \(E_0(R_0)=E_0(C_0)\).
Because \(C_1\le C_0\) almost surely, monotonicity of expectation gives \(E_0(C_1)\le E_0(C_0)\). Since the rejection indicators have expectations equal to their rejection probabilities, \(P_0(R_1=1)\le P_0(R_0=1)\).
The strict inequality also follows from this argument: if \(C_0-C_1\ge0\) everywhere and is positive on a set of positive null probability, then \(E_0(C_0-C_1)>0\). Exact equality of the conditional errors, in contrast, gives equality of the unconditional errors. If the original test’s type I error is at most a target level \(\alpha\), conservative preservation therefore guarantees an adapted type I error at most \(\alpha\); exact preservation retains the original rate, which need not equal \(\alpha\).
Worked calculations with a two-stage normal test
For the next examples, assume a one-sided simple null under which \(Z_1\) and \(Z_2\) are independent standard normal statistics. The original test rejects when their equally weighted combination \(Z=(Z_1+Z_2)/\sqrt{2}\) exceeds 1.96. Under the null, \(Z\sim N(0,1)\), so the original type I error is \(P_0(Z>1.96)\), approximately 0.0250. Independence and the stated standard normal distributions are assumptions of this illustration; a real design must use its own appropriate conditional null distribution.
Worked Example: Finding the Original Conditional Error
Suppose the observed interim statistic is \(Z_1=1.2\). Find the original test’s conditional error and the continuation threshold that achieves it.
The original rejection condition is \((Z_1+Z_2)/\sqrt{2}>1.96\). Given \(Z_1=1.2\), this is equivalent to \(Z_2>1.96\sqrt{2}-1.2\). Under the stated null model, \(Z_2\) remains standard normal conditional on \(Z_1\), by independence. Thus,
Substitution gives \(1.96\sqrt{2}=2.7719\), so the continuation threshold is \(2.7719-1.2=1.5719\). Therefore \(C_0(1.2)=P(Z_2>1.5719)\approx0.05799\), or about 5.799%. This is the conditional error at this interim result; it is not the overall type I error. The original unconditional error remains about 2.500%.
At this interim value, exact preservation requires a modified continuation rule whose conditional rejection probability is 0.05799. For example, rejecting when \(Z_2>1.5719\) does so under the assumed normal model. Other continuation rules could also have the same conditional rejection probability; equality concerns the probability, not a uniquely specified boundary.
Worked Example: Spending Less Than the Conditional Error
At \(Z_1=1.2\), suppose the adapted rule raises the continuation boundary by 0.3, rejecting only when \(Z_2>1.8719\). Calculate the new conditional error and compare it with the original conditional error at this interim result.
The new conditional error is \(C_1(1.2)=P(Z_2>1.5719+0.3)=P(Z_2>1.8719)\approx0.0306\). The original conditional error is approximately 0.05799. Therefore \(C_1(1.2)<C_0(1.2)\): this rule spends less than the available conditional error at this interim value. The conditional error left unused is about \(0.05799-0.0306=0.0274\), or 2.74 percentage points.
That comparison alone does not determine the new global type I error: the rule must be assessed over all interim outcomes, not just \(Z_1=1.2\). To calculate a global rate for a rule that raises the boundary by 0.3 at every interim result, its rejection condition is \(Z_2>1.96\sqrt{2}-Z_1+0.3\), equivalently \(Z_1+Z_2>1.96\sqrt{2}+0.3\). Hence
The adapted global type I error is approximately 1.49%, below the original 2.50%. This calculation uses the same independent standard normal null model throughout and a fixed increase of 0.3, so the rule spends less at every interim result.
Exact and conservative preservation across interim outcomes
The next example shows how conditional comparisons combine into an unconditional error rate when the interim information has only two possible categories. The numbers are hypothetical probabilities under a simple null, chosen to make the expectation calculation transparent.
Worked Example: Averaging Conditional Error Across Two Outcomes
Suppose interim outcome A has null probability 0.4 and outcome B has probability 0.6. The original conditional errors are 0.08 for A and 0.03 for B. An adapted design has conditional errors 0.08 for A and 0.01 for B. Find both global type I error rates and determine whether preservation is exact or conservative.
For the original test, the unconditional error is the probability-weighted average: \(0.4(0.08)+0.6(0.03)=0.032+0.018=0.050\), or 5.00%.
The adapted design equals the original conditional error for A and is below it for B. Its global error is \(0.4(0.08)+0.6(0.01)=0.032+0.006=0.038\), or 3.80%. It is conservatively preserving conditional error: its conditional rejection probability is no greater at either outcome and is strictly smaller at B. Since B has positive null probability, the global error is strictly smaller, by \(0.050-0.038=0.012\), or 1.20 percentage points.
If the adapted design instead had conditional error 0.08 for A and 0.03 for B, both conditional probabilities would match the original, and the weighted average would remain 5.00%. That would be exact preservation.
Diagnostics, alternatives, and limitations
A useful audit is to tabulate or otherwise evaluate \(C_0\) and \(C_1\) over the possible interim information. Check the inequality \(C_1\le C_0\) at every relevant value, or establish it analytically. Then average under the null distribution of the interim information to obtain the global error. Simulations can help find coding errors or regions where the inequality fails, but simulation alone does not prove a pointwise bound.
The main alternatives are exact matching and conservative control. Exact matching uses the full original conditional error at each interim result and retains the original global type I error. Conservative control imposes an inequality, may simplify the adaptation, and can provide a margin below the original error. A direct recalculation of the adapted test’s global null rejection probability is another way to assess size when the full joint distribution is tractable. It can confirm an error rate for a particular design, but the conditional comparison explains why control follows from the original test and can be easier to verify.
The argument has limits. It is only as reliable as the null model, conditioning information, and conditional probability calculations. If the interim information used to select a rule is omitted from \(\mathcal{F}\), the stated conditional comparison may not describe the actual procedure. Likewise, a pointwise inequality verified only at a few interim values does not establish the required almost-sure inequality over the full range. For composite null hypotheses or nuisance parameters, the relevant control must hold appropriately over the null parameter space; that issue requires additional care beyond the simple-null argument here.
Common mistake
A common error is to see that a modified rule spends less conditional error at one observed interim result and conclude that its global type I error is therefore lower. That does not follow: the adapted rule could spend more than the original conditional error at other interim outcomes, offsetting the apparent saving. The correction is to verify \(C_1\le C_0\) almost surely across interim outcomes, then use conditional expectation to obtain the global bound. A strict global reduction requires strict inequality on a set of positive null probability.
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- An original test has type I error 0.04. An adapted rule has conditional error equal to the original conditional error almost surely. What is its unconditional type I error?
- At two interim outcomes with probabilities 0.25 and 0.75, original conditional errors are 0.10 and 0.02. An adapted rule has conditional errors 0.08 and 0.02. Calculate both global errors and classify the preservation.
- For the normal test in the first worked example, write the continuation threshold given \(Z_1=x\). What threshold applies at \(x=0\)?
- A designer verifies the conservative inequality only at the observed interim value in one trial. Is that enough to establish global type I error control for the adaptive procedure? Explain.
Solutions. 1. Exact preservation gives the same unconditional type I error, 0.04. 2. The original error is \(0.25(0.10)+0.75(0.02)=0.025+0.015=0.040\). The adapted error is \(0.25(0.08)+0.75(0.02)=0.020+0.015=0.035\). Since the adapted conditional errors are no greater at either outcome and strictly smaller at the first, this is conservative preservation and the global error falls from 4.00% to 3.50%. 3. The original rejection condition gives \(Z_2>1.96\sqrt{2}-x\). At \(x=0\), the threshold is \(1.96\sqrt{2}\approx2.772\). 4. No. Control requires the conditional inequality to hold almost surely over the interim information under the null, not only at one observed value; the verification must cover the possible interim outcomes or justify the inequality analytically.