The Conditional Error Bound

Why a Bound Is Needed After an Interim

An interim analysis reveals information that may influence what happens next in a clinical trial. The continuation might use a different test, but changing the procedure can also change its probability of rejecting a true null hypothesis. The relevant comparison is not only between the original and adapted procedures’ overall rejection rates. It is between their rejection probabilities conditional on the interim information actually observed.

For a fixed null hypothesis, the original test has a certain amount of rejection probability remaining at each interim state. That amount is its conditional error. An adapted continuation preserves the original test’s error guarantee if, at every interim state apart from a probability-zero set, its conditional rejection probability is no larger than this remaining allowance.

Definition: Let $H_0$ be a null hypothesis, let $\mathcal F_I$ denote the information available at an interim, and let $\phi^0$ be the original test’s rejection indicator. Under $H_0$, the original test’s conditional error is $$ e(Y)=E_0(\phi^0\mid\mathcal F_I), $$ where $Y$ denotes the interim data and $E_0$ is expectation under $H_0$. If the adapted continuation has rejection indicator $\phi^A$, its conditional rejection probability is $q(Y)=E_0(\phi^A\mid\mathcal F_I)$. The conditional error bound is $q(Y)\le e(Y)$ almost surely under $H_0$.

A test may be randomized, in which case its rejection function can take values between 0 and 1; the same definitions apply with $\phi^0$ and $\phi^A$ interpreted as rejection probabilities given the full data. The conditional probabilities include the uncertainty in all future observations and any randomization used by the continuation. They are evaluated under the null distribution appropriate to the test being considered.

The phrase “almost surely” allows the inequality to fail on interim outcomes having probability zero under $H_0$. It does not allow the bound to fail on a possible interim state merely because that state is uncommon. The comparison is pointwise in the interim information, not just an average across interim states.

How the Bound Controls Overall Type I Error

Suppose the original test has Type I error at most $\alpha$ under $H_0$, so $E_0(\phi^0)\le\alpha$. If the adapted continuation satisfies the conditional error bound, then the law of total expectation gives

P_0(\text{adapted test rejects}) =E_0(\phi^A) =E_0\{E_0(\phi^A\mid\mathcal F_I)\} =E_0\{q(Y)\} \le E_0\{e(Y)\} =E_0(\phi^0) \le \alpha.

The key step is the conditional inequality $q(Y)\le e(Y)$; taking expectations preserves that inequality. The equality $E_0\{e(Y)\}=E_0(\phi^0)$ is also an application of the law of total expectation. Thus, the adapted procedure’s unconditional rejection probability under this null cannot exceed that of the original test.

This statement is for the specified null hypothesis and its probability law. If a trial’s error guarantee concerns several null hypotheses or a family of hypotheses, the relevant conditional bound must be formulated for the test and null configuration whose error is being controlled. A bound for one null does not by itself establish control for every other null.

Worked Example: Checking a Bound Across Two Interim States

Suppose a hypothetical test has two possible interim states, $A$ and $B$. Under $H_0$, $P_0(A)=0.40$ and $P_0(B)=0.60$. The original test’s conditional errors are $e(A)=0.08$ and $e(B)=0.03$. An adapted continuation has conditional rejection probabilities $q(A)=0.07$ and $q(B)=0.02$. Check the bound and compare the overall rejection probabilities.

At state $A$, $q(A)=0.07\le 0.08=e(A)$. At state $B$, $q(B)=0.02\le 0.03=e(B)$. The conditional bound holds in both states.

The original test’s overall rejection probability under $H_0$ is $$ P_0(\text{original rejects}) =(0.40)(0.08)+(0.60)(0.03) =0.032+0.018 =0.050. $$

The adapted continuation’s overall rejection probability is $$ P_0(\text{adapted test rejects}) =(0.40)(0.07)+(0.60)(0.02) =0.028+0.012 =0.040. $$

The adapted procedure’s Type I error is $0.040$, or 4.0%, in this hypothetical setting, compared with 5.0% for the original test. The pointwise checks, rather than this aggregate comparison alone, establish that the conditional error bound holds.

Calculating Conditional Error in a Normal Combination Test

A useful calculation arises when an original one-sided test combines interim and future evidence. Assume, under $H_0$, that $Z_1$ and $Z_2$ are independent standard normal statistics from the interim and future stages. Let the fixed information fraction be $t$, with $0<t<1$, and let the original test reject when

Z=\sqrt{t}\,Z_1+\sqrt{1-t}\,Z_2>c,

where $c$ is the prespecified critical value. Under the stated assumptions, $Z\sim N(0,1)$ under $H_0$, because it is a weighted sum of independent standard normal variables with squared weights summing to 1. Once $Z_1=z_1$ is observed, only $Z_2$ remains random. The conditional error is therefore

e(z_1)=P_0\left( Z_2>\frac{c-\sqrt{t}\,z_1}{\sqrt{1-t}} \;\middle|\; Z_1=z_1 \right) =1-\Phi\left(\frac{c-\sqrt{t}\,z_1}{\sqrt{1-t}}\right),

where $\Phi$ is the standard normal cumulative distribution function. Independence matters: it ensures that conditioning on $Z_1$ leaves $Z_2$ standard normal under the null. If that conditional distribution differs, the calculation must use the correct conditional law instead.

Worked Example: A Normal Test at an Observed Interim Value

Consider a hypothetical one-sided combination test with $t=0.50$, critical value $c=1.96$, and observed interim statistic $Z_1=1.20$. Under $H_0$, assume $Z_1$ and $Z_2$ are independent standard normal variables, as in the model above. Find the original test’s conditional error at this interim value, then compare it with two possible continuation rules.

The future-stage threshold for the original test is $$ \frac{1.96-\sqrt{0.50}(1.20)}{\sqrt{0.50}} =\frac{1.96-0.8485281}{0.7071068} \approx 1.5718586. $$

Consequently, $$ e(1.20)=P_0(Z_2>1.5718586) =1-\Phi(1.5718586) \approx 0.0580. $$

At this interim state, the original test has about 5.80% conditional rejection probability remaining under $H_0$. Suppose an adapted continuation rejects if $Z_2>1.60$. Its conditional rejection probability is $$ q(1.20)=1-\Phi(1.60)\approx 0.0548, $$ which is below $0.0580$, so the bound holds at this state. If instead it rejects when $Z_2>1.55$, then $$ q(1.20)=1-\Phi(1.55)\approx 0.0606, $$ which exceeds the conditional error and violates the bound at this state. These comparisons address the observed interim value only; a procedure claiming the bound must satisfy it at all possible interim states, almost surely under $H_0$.

Why an Aggregate Comparison Is Not the Same Guarantee

The conditional bound is a sufficient pointwise condition for preserving the original test’s overall error guarantee. It is stronger than merely checking whether the adapted test’s average rejection probability is below the original test’s average. A procedure could exceed the original conditional error in one interim state and compensate with a smaller rejection probability in another. Such compensation may leave its overall Type I error below the original level, but it does not satisfy the conditional error bound.

Worked Example: An Average Can Hide a Conditional Violation

Suppose, hypothetically, that two interim states are equally likely under $H_0$, and that the original test has conditional error $0.06$ in each state. An adapted continuation has conditional rejection probabilities $0.08$ in state $A$ and $0.02$ in state $B$. Compare the conditional and overall error rates.

In state $A$, the bound fails because $0.08>0.06$. In state $B$, it holds because $0.02\le 0.06$. The original overall rejection probability is $$ (0.50)(0.06)+(0.50)(0.06)=0.060. $$

The adapted overall rejection probability is $$ (0.50)(0.08)+(0.50)(0.02)=0.040+0.010=0.050. $$

Thus the adapted test’s average rejection probability is $0.050$, lower than the original $0.060$, even though the pointwise conditional bound is violated in state $A$. The example shows why an average comparison cannot be substituted for checking the stated conditional condition.

Diagnostics, Alternatives, and Limitations

To assess a proposed continuation, first specify the null distribution and the information being conditioned on. Then calculate the original test’s conditional error as a function of the interim data, calculate the adapted continuation’s conditional rejection probability under the same null, and compare the two at each possible interim state. This comparison is meaningful only when both probabilities use compatible conditioning and the appropriate null law.

Several errors can undermine the check:

Reusing $\alpha$ at the interim. The conditional error is generally not equal to the original nominal level $\alpha$. It depends on the interim data and the original testing rule.
Checking only the observed or typical state. A guarantee requires the inequality almost surely over the interim-state distribution, not merely at one illustrative value.
Comparing unconditional rates only. An acceptable overall average does not establish the pointwise conditional bound, as the preceding example shows.
Using the wrong conditional distribution. Dependence between interim and future statistics, or a misspecified null model, changes the conditional rejection probability and can invalidate the calculation.
Confusing conditional error with conditional power. Conditional error is calculated under the null for the original test’s rejection event. Conditional power concerns rejection under an alternative and answers a different question.

A direct alternative is to analyze the full adapted procedure’s unconditional Type I error under the null. That can establish an error guarantee if the calculation correctly represents the complete procedure. The conditional bound instead supplies a local criterion at every interim state and compares the continuation with the original test’s remaining error allowance. It is sufficient, but not necessary: a continuation can fail the pointwise bound and still have an overall error rate no larger than the original test.

The bound also does not repair an incorrectly specified original test, nor does it automatically guarantee control under null hypotheses not included in the calculation. Its conclusion is only as sound as the null model, interim conditioning, and rejection probabilities used to establish it.

Key takeaway: Under a specified null hypothesis, the adapted continuation preserves the original test’s error guarantee when its conditional rejection probability does not exceed the original test’s conditional error at almost every interim state. The inequality is checked state by state; overall control follows by taking expectations.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

If the original test has conditional errors $0.04$ and $0.10$ in two interim states, may an adapted continuation have conditional rejection probabilities $0.05$ and $0.08$ and still satisfy the bound?
In the normal combination test with $t=0.50$, $c=1.96$, and $Z_1=1.20$, what is the conditional error to three significant figures?
Why does an average adapted rejection probability below the original average not prove the conditional error bound?
What assumption about $Z_1$ and $Z_2$ permits the normal-tail calculation after conditioning on $Z_1$?

Solutions. (1) No. In the first state, $0.05>0.04$, so the bound fails even though the second state satisfies it. (2) The future-stage threshold is $(1.96-\sqrt{0.50}(1.20))/\sqrt{0.50}\approx1.5718586$, giving $1-\Phi(1.5718586)\approx0.0580$, or 5.80%. (3) A smaller average can conceal a state where the adapted conditional rejection probability exceeds the original conditional error; the bound requires the inequality at almost every state. (4) Under the null, $Z_1$ and $Z_2$ are independent standard normal variables, so conditioning on $Z_1$ leaves $Z_2$ standard normal.

What You'll Learn