Conditional Type I Error at an Interim

Why Look at Error Conditional on Interim Data?

A trial’s Type I error is usually described as a probability for the complete design: under a null hypothesis, how often would the design reject? An interim review introduces a different question. Once some data have been observed, what is the probability that the trial will eventually reject, conditional on those data, if the null hypothesis is true?

That probability can vary substantially across interim results. An interim result near a final rejection threshold may leave appreciable chance of eventual rejection, while an interim result far from it may leave very little. Neither conditional probability, on its own, changes the unconditional Type I error of the original design. It describes the remaining rejection risk for a particular interim result under a specified null.

Definition: Let $R$ be the event that a design eventually rejects a null hypothesis $H_0$, and let $I$ denote the interim data. For a fixed parameter value $\theta$ satisfying $H_0$, the conditional Type I error at interim data $I=i$ is $$ \alpha_{\mathrm{cond}}(\theta;i)=P_\theta(R\mid I=i). $$ It is the probability of eventual rejection under that null parameter value, conditional on the interim data. The corresponding unconditional Type I error is $P_\theta(R)$, which averages over the possible interim data as well as future data.

The parameter value matters. For a composite null, different values of $\theta$ in the null parameter space can give different conditional probabilities. Thus, a conditional Type I error is not automatically one number for the whole null hypothesis; state which null parameter value or null model is being used. In a continuous-data model, an exact interim value may have probability zero. The conditional probability is then understood through the model’s regular conditional distribution, defined for almost all interim values.

How the Conditional and Unconditional Probabilities Relate

For a discrete interim record, the unconditional probability is the probability-weighted average of the conditional probabilities. To see why, partition the rejection event according to the possible interim values. The events $R\cap\{I=i\}$ are disjoint, so their probabilities add; and $P_\theta(R\cap\{I=i\})=P_\theta(R\mid I=i)P_\theta(I=i)$. Therefore,

P_\theta(R)=\sum_i P_\theta(R\mid I=i)P_\theta(I=i).

For continuous interim data, the corresponding statement is that the unconditional probability is the expectation of the conditional rejection probability over the distribution of $I$. This averaging identity is a useful check: conditional risks can differ greatly across interim results while averaging to the same unconditional Type I error. It does not say that each conditional risk equals the unconditional one.

Worked Example: A Two-Observation Test

Suppose a hypothetical study observes two independent binary outcomes $X$ and $Y$. Under the null hypothesis, each is Bernoulli with success probability $0.5$. The interim observation is $X$, and the planned rule rejects only if both observations are successes. Find the unconditional and conditional Type I errors.

Under the null, the rejection event is $R=\{X=1,Y=1\}$. Independence gives

$$ P_0(R)=P_0(X=1)P_0(Y=1)=0.5(0.5)=0.25. $$

If the interim value is $X=1$, eventual rejection requires $Y=1$, so $P_0(R\mid X=1)=0.5$. If $X=0$, rejection is impossible, so $P_0(R\mid X=0)=0$. The weighted average checks the unconditional result:

P_0(R\mid X=1)P_0(X=1)+P_0(R\mid X=0)P_0(X=0) =0.5(0.5)+0(0.5)=0.25.

Thus, this design has unconditional Type I error $0.25$, but its conditional error is $0.5$ after a success and $0$ after a failure. The conditional probability is a description of future rejection risk, not a new unconditional error rate.

Calculating Conditional Error from a Normal Interim Statistic

A common model uses independent increments of a test statistic. Suppose under a fixed null hypothesis the interim statistic $Z_1$ and the future increment $Z_2$ are independent, each distributed as $N(0,1)$. The planned final statistic is $Z=(Z_1+Z_2)/\sqrt{2}$, which has distribution $N(0,1)$ under the null. The planned design rejects if $Z>1.96$, giving unconditional one-sided Type I error approximately $0.0250$.

Once $Z_1=z_1$ is observed, only $Z_2$ remains random. The final rejection rule becomes $Z_2>\sqrt{2}(1.96)-z_1$. The conditional Type I error is therefore a standard normal upper-tail probability. The calculation relies on the stated independent-increments null model and on applying the original final decision rule.

Worked Example: Conditional Error After a Favorable Interim Result

In the normal model above, suppose the hypothetical interim value is $z_1=1.50$. Calculate the conditional Type I error and compare it with the planned unconditional value.

The final rule rejects when $(1.50+Z_2)/\sqrt{2}>1.96$. Rearranging gives

Z_2>\sqrt{2}(1.96)-1.50 =2.7719-1.50 =1.2719.

Since $Z_2\sim N(0,1)$ under the null,

P_0(R\mid Z_1=1.50)=P(Z_2>1.2719) =1-\Phi(1.2719)\approx 0.1017.

Conditional on this interim value, the planned trial has about a 10.17% chance of rejecting under the null. The unconditional Type I error remains about 2.50% for the original design, because it also averages over interim values that make rejection much less likely. This is not a contradiction: one is conditional on a particular result, the other averages over all results.

A Discrete Clinical-Outcome Example

The same distinction applies when the interim statistic is a count. The next example uses a simple binomial model so that the conditional probability can be calculated exactly. Its assumptions are independent, identically distributed binary outcomes with the stated null success probability, and a fixed rejection rule applied after the remaining outcomes are observed.

Worked Example: Interim Successes and a Final Count Rule

In a hypothetical study, ten independent binary outcomes have success probability $0.5$ under $H_0$. After the first four outcomes, let $X$ be the number of successes observed; let $Y$ be the number among the remaining six. The planned rule rejects if the total number of successes is at least eight. Find the conditional Type I error after $X=3$, and verify the unconditional Type I error.

Given $X=3$, rejection requires $3+Y\geq 8$, or $Y\geq 5$. Under the null, $Y\sim\operatorname{Binomial}(6,0.5)$, so

P_0(R\mid X=3) =P(Y\geq 5) =\frac{\binom{6}{5}+\binom{6}{6}}{2^6} =\frac{6+1}{64} =0.1094.

Unconditionally, the total number of successes is $\operatorname{Binomial}(10,0.5)$, so

P_0(R) =\frac{\binom{10}{8}+\binom{10}{9}+\binom{10}{10}}{2^{10}} =\frac{45+10+1}{1024} =0.05469.

For an explicit averaging check, $X$ can only permit rejection when $X\geq 2$. The conditional probabilities for $X=2,3,4$ are $1/64,7/64,22/64$, respectively; the first requires $Y=6$, and the last requires $Y\geq4$. Since $P(X=2)=6/16$, $P(X=3)=4/16$, and $P(X=4)=1/16$, the average is

\frac{6}{16}\frac{1}{64} +\frac{4}{16}\frac{7}{64} +\frac{1}{16}\frac{22}{64} =\frac{6+28+22}{1024} =\frac{56}{1024} =0.05469.

The interim-specific risk after three successes is about 10.94%, while the planned unconditional risk is about 5.469%. Neither number is an estimate of the probability that $H_0$ is true.

Conditional Error in a Multiple-Testing Setting

When the rejection event concerns more than one hypothesis, define it explicitly. For example, under a complete null, $R$ may mean that at least one endpoint null is rejected. The conditional Type I error is then the chance of that familywise error event, given the interim data. It is not necessarily the conditional error for each individual endpoint, and those probabilities should not be conflated.

For a simple safeguard, suppose two endpoint tests each have a one-sided null tail probability no greater than $0.0125$. The union bound gives a familywise error no greater than $2(0.0125)=0.0250$. A standard normal cutoff that ensures the individual tail is at most $0.0125$ is $c=\Phi^{-1}(0.9875)\approx 2.241403$. A cutoff of $2.241$ is slightly smaller: its tail probability is approximately $0.012513$, giving a union bound of approximately $0.025026$, which does not establish a $0.0250$ guarantee. This illustrates why an interim-conditional calculation or a rounded threshold must be checked against the precise event and probability being controlled.

Interpretation, Diagnostics, and Limits

Conditional Type I error is useful for describing how much rejection probability remains after a particular interim result. It can also help identify which interim outcomes leave substantial risk under the null. But it is tied to a specified null model, interim information, and eventual decision rule. If any of these are unclear, the conditional probability is not fully defined.

Check the event. State whether $R$ means rejection of one null, any endpoint null, or another specified error event.
Check the null model. Identify the parameter value or distribution under which the probability is calculated, especially for a composite null.
Check what remains random. After conditioning, use the distribution of future data given the interim record, not the unconditional distribution of the full data.
Check the rule. The calculation must reflect the actual planned stopping and rejection rule. A probability calculated for one rule does not automatically describe a different rule.
Check numerical precision. A rounded tail probability slightly above a target does not justify claiming that the target is met.

A frequent mistake is to say, “The trial’s Type I error is 10.17% because the interim statistic was 1.50.” That confuses a conditional probability for one interim value with the unconditional error probability of the design. The correction is to state both quantities and their conditioning: in the normal example, the conditional rejection probability at $Z_1=1.50$ is about 10.17%, whereas the original design’s unconditional one-sided Type I error is about 2.50%.

Conditional Type I error also differs from conditional power. Conditional Type I error calculates future rejection probability under a null hypothesis; conditional power calculates future rejection probability under an alternative hypothesis, with the relevant assumptions specified. Neither quantity alone is the probability that the hypothesis is true. And calculating a conditional error for the original design does not by itself prove that a changed design has controlled Type I error; that requires evaluating the changed rule and its error-control justification.

The main limitation is therefore interpretive: a conditional risk is local to the observed interim information and model. It can be large or small without contradicting the design’s unconditional error rate. Its value is precision about the question being asked, not replacement of unconditional error control.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A design has unconditional Type I error $0.04$. After one interim result, its conditional Type I error is $0.12$. Is this automatically inconsistent? Explain.
In the normal-increment example, calculate the conditional Type I error when $Z_1=0$. Use the original final cutoff $1.96$.
For the binomial example, find the conditional Type I error after $X=2$ successes in the first four outcomes.
Explain why a conditional Type I error for a fixed null parameter may not describe all parameter values in a composite null.

Solutions. (1) No. Conditional error is specific to the observed interim result; the unconditional error averages conditional errors over all possible interim results. (2) Rejection requires $Z_2>\sqrt{2}(1.96)-0=2.7719$, so the conditional error is $1-\Phi(2.7719)\approx0.00279$. (3) Rejection requires $2+Y\geq8$, so $Y=6$; hence $P(Y=6)=1/64=0.01563$. (4) The distribution of future data given the interim record can depend on the parameter. Different parameter values within the null may therefore produce different probabilities of eventual rejection.