Conditioning on Interim Data

Why the Interim State Must Be Specified

A trial can have an overall Type I error rate that is calibrated under a null hypothesis while its error probability varies substantially across interim outcomes. A favorable interim result may leave a relatively large chance of eventual rejection; an unfavorable result may leave very little. To describe that variation, “given the interim” is not precise enough. One must say exactly what interim information is being conditioned on and which null hypothesis is being evaluated.

The distinction matters in adaptive design because an interim review can be represented by more than one summary. A test statistic, the full set of interim outcomes, and a record that includes operational or safety information are not interchangeable conditioning sets. Each can lead to a different conditional distribution for the future data, and therefore a different conditional error probability.

Definition: Let $H_0$ be a null hypothesis, let $D$ denote the data used by the originally planned test, and let $\phi(D)$ be its rejection indicator (equal to 1 for rejection and 0 otherwise). Let $\mathcal{F}_1$ represent the specified information available at the interim. For a particular null parameter value $\theta$, the conditional Type I error is $$ C_\theta(\mathcal{F}_1)=E_\theta\!\left[\phi(D)\mid\mathcal{F}_1\right] =P_\theta(\text{reject }H_0\mid\mathcal{F}_1). $$ After observing an interim state $f$, its value is $c_\theta(f)$. If the test is randomized, $\phi(D)$ may take values between 0 and 1, and the same conditional-expectation definition applies.

For a simple null hypothesis, there is one null probability model and hence one conditional error function. For a composite null, such as $H_0:\theta\leq 0$, the conditional error can vary with $\theta$ even when the interim state is fixed. In that case, a conditional error statement must identify the null value being used, or specify a rule such as taking the largest conditional error over the null parameter space. Reporting only “the conditional error” can conceal this dependence.

What Counts as Interim Information?

A formal way to specify interim information is to name the random quantities it contains. For example, if $Y_1$ is the interim statistic, conditioning on $Y_1$ means conditioning on the information generated by $Y_1$. If the full interim dataset contains additional observed quantities, conditioning on that dataset is a richer conditioning set. The resulting conditional probability need not equal the probability given $Y_1$ alone.

A reduction to a statistic is justified when the relevant conditional distribution of future data, and the planned rejection rule, depend on the interim data only through that statistic. Otherwise, omitted information may matter. In particular, conditioning on the interim statistic while ignoring other observed variables that predict future outcomes can give the wrong conditional error for the actual interim state.

The conditional probability is also tied to the null model. Under a simple null, that model specifies how future data behave after conditioning. Under a composite null, each parameter value can imply a different conditional law. It is not generally correct to substitute the alternative model, an estimate of the treatment effect, or an unspecified “null” distribution without explaining the choice.

Worked Example: Conditional Error From Normal Increments

Suppose, hypothetically, that an original one-sided test uses two independent standardized increments, $Y_1$ at the interim and $Y_2$ after the interim. Under parameter value $\theta$, assume $Y_1$ and $Y_2$ are independent and each has distribution $N(\theta,1)$. The planned test rejects $H_0:\theta\leq 0$ if $(Y_1+Y_2)/\sqrt{2}>1.645$. Under the boundary null $\theta=0$, this test has one-sided size approximately 0.05. If the observed interim value is $Y_1=1.2$, calculate the conditional error under $\theta=0$.

The conditioning information is specifically $Y_1$, and the observed state is $Y_1=1.2$. Independence implies that, given this interim value, $Y_2$ still has distribution $N(0,1)$ under the boundary null. The rejection condition becomes

\frac{1.2+Y_2}{\sqrt{2}}>1.645 \quad\Longleftrightarrow\quad Y_2>1.645\sqrt{2}-1.2=1.126

Therefore,

P_0(\text{reject}\mid Y_1=1.2) =1-\Phi(1.126)\approx 0.130

The conditional error is about 13.0% at this interim state, although the unconditional size under the boundary null is about 5.0%. This is not a contradiction: an unconditional rate averages over possible interim states, whereas this value describes one specified state.

Deriving the Conditional Probability

The normal calculation follows directly from the definition. Under a specified null parameter, write the planned rejection event as a condition on interim data and future data. Then hold the interim information fixed and calculate the probability of the remaining condition using the conditional distribution of the future data.

Write the planned rejection event.
For the normal example, rejection is $(Y_1+Y_2)/\sqrt{2}>1.645$.

Condition on the stated information.
Set the observed interim value to $Y_1=y$. The rejection event becomes $Y_2>1.645\sqrt{2}-y$.

Use the future-data distribution under the null.
Under $\theta=0$, independence gives $Y_2\mid Y_1=y\sim N(0,1)$, so the conditional error is $1-\Phi(1.645\sqrt{2}-y)$.

This derivation depends on the model assumptions, not merely on algebra: the increments must have the stated distributions and be independent under the null. If the future-data distribution depends on the interim data, the conditional distribution must instead include that dependence. A calculation that uses an unconditional future-data distribution when the conditional distribution differs is not a valid conditional error calculation.

Worked Example: Exact Conditional Error for a Binomial Test

Consider a hypothetical trial with 20 binary outcomes, of which the first 10 are observed at the interim. Under a simple null, let each outcome be independent with success probability $p=0.5$. The original test rejects if at least 15 of the 20 outcomes are successes. If 7 successes have been observed among the first 10, find the conditional error.

The remaining 10 outcomes are independent of the observed interim outcomes and have a $\operatorname{Binomial}(10,0.5)$ distribution under the null. To reach at least 15 total successes after observing 7, at least 8 of the remaining 10 must be successes. Thus,

P_{0.5}(\text{reject}\mid X_1=7) =P\{\operatorname{Binomial}(10,0.5)\geq 8\} =\frac{\binom{10}{8}+\binom{10}{9}+\binom{10}{10}}{2^{10}}

Substituting the binomial coefficients gives

\frac{45+10+1}{1024} =\frac{56}{1024} =0.05469

So the conditional error at this interim state is about 5.47%. The originally planned test has unconditional null rejection probability $$ \frac{\binom{20}{15}+\binom{20}{16}+\binom{20}{17}+\binom{20}{18}+\binom{20}{19}+\binom{20}{20}}{2^{20}} =\frac{21700}{1048576}\approx 0.02069. $$

The difference is possible because the test is conservative overall, but the observed 7 successes place the trial in a state from which rejection is more likely than it is before observing any data. The calculation is exact for the stated independent Bernoulli model; it does not require a normal approximation.

Why the Null Hypothesis Is Part of the Answer

Conditioning specifies the observed information; it does not remove the need to specify the probability model. If the null is composite, the same observed interim state can have different conditional error probabilities at different null parameter values. This is especially important when the test statistic is discrete or when future outcomes depend strongly on the null parameter.

Worked Example: One Interim State, Two Null Values

Use the binomial test above, but now consider the composite null $H_0:p\leq 0.5$. Keep the observed interim count at 7 successes and compare $p=0.5$ with $p=0.4$. At $p=0.4$, the remaining 10 outcomes have distribution $\operatorname{Binomial}(10,0.4)$, and at least 8 further successes are still required.

At the boundary $p=0.5$, the conditional error was $56/1024=0.05469$. At $p=0.4$, it is

\begin{aligned} P_{0.4}(\text{reject}\mid X_1=7) &=\binom{10}{8}(0.4)^8(0.6)^2 +\binom{10}{9}(0.4)^9(0.6) +(0.4)^{10}\\ &=45(0.4)^8(0.6)^2+10(0.4)^9(0.6)+(0.4)^{10}\\ &\approx 0.01229. \end{aligned}

Thus, for the same observed interim count, the conditional error is about 5.47% at $p=0.5$ and 1.23% at $p=0.4$. The null hypothesis contains both values, but it does not assign a single conditional probability to the state unless an additional convention is specified. For this increasing-tail test, the boundary $p=0.5$ gives the larger conditional error: each term in the upper tail for the remaining count increases as $p$ increases from 0.4 to 0.5.

Interpreting and Checking a Conditional Error

A conditional error is the probability of the planned rejection event, conditional on specified interim information and evaluated under a specified null model. It is not the probability that the null hypothesis is true, a posterior probability of treatment benefit, or a p-value. A p-value is calculated from observed data under a null model; conditional error is a probability of a future or final rejection event given an interim state.

For intersection hypotheses in a closed testing procedure, the same principle applies to the local test of each intersection null. One must identify both the intersection being tested and the interim information being conditioned on. The conditional error for one intersection test need not equal that for another, even when the observed interim data are identical, because the null models or rejection rules can differ.

Useful diagnostics are straightforward. Check that the interim information is explicitly named; verify that the stated conditional distribution of future data follows from the model; and confirm that the rejection boundary is the one in the original test. If conditioning on a statistic rather than the full interim dataset, justify why the omitted information does not alter the relevant conditional law or decision rule. For composite nulls, calculate or bound the conditional error across the null parameter values rather than silently treating one value as representative.

Quantity	What it conditions on	What it answers
Unconditional Type I error	No interim state is fixed	How often the test rejects over repeated trials under the null
Conditional Type I error	A specified interim information set and observed state	How likely the planned test is to reject from that state under the null
p-value	The observed test statistic, evaluated under a null distribution	How extreme the observed statistic is under the null model
Posterior probability	Observed data and a prior distribution	How probability is assigned to parameters or hypotheses under a Bayesian model

The conditional error framework has limits. It is only as reliable as the model for future data and the definition of the interim information. Sparse discrete data can produce jumps in conditional error as the observed count changes; nuisance parameters may make a single value inadequate; and model misspecification can make the calculated future-data distribution inaccurate. These are reasons to state the assumptions and conditioning set, not reasons to treat conditional error as an unconditional guarantee.

Common Mistakes

Leaving the conditioning set vague. “Given the interim” does not identify whether the conditioning information is a statistic or the full interim history. Name it precisely.
Using the wrong future-data law. The conditional distribution must be derived under the null and given the interim information, not copied from an unrelated unconditional calculation.
Treating a composite null as one model. State the parameter value or explain how the conditional error is handled across the null space.
Calling conditional error a probability that the null is true. It is a rejection probability under a specified null model, conditional on the observed state.
Assuming every conditional value equals the overall error rate. The conditional probability can vary by interim state even when the overall test has a fixed Type I error rate.

Key takeaway: A conditional Type I error is defined only after specifying the interim information, the observed state, the planned rejection rule, and the null probability model. For a composite null, the parameter value—or the method for accounting for all null values—must also be made explicit.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In the normal-increment example, what is the conditional error at $Y_1=0$ under $\theta=0$?
In the binomial example, if the interim count is 4, what is the conditional error under $p=0.5$?
Why can the conditional error at an observed state differ from the unconditional Type I error?
For a composite null $p\leq0.5$, why is the conditional error at $p=0.4$ not automatically the conditional error for the whole null?

Solutions. (1) The future increment must exceed $1.645\sqrt{2}=2.326$, so the conditional error is $1-\Phi(2.326)\approx0.0100$. (2) With 4 interim successes, at least 11 additional successes would be needed to reach 15, which is impossible in the 10 remaining observations; the conditional error is 0. (3) The unconditional rate averages over all possible interim states, while the conditional error fixes one state, whose rejection probability can be higher or lower. (4) The composite null includes other parameter values, and the future-data distribution—and thus the conditional error—can differ across them. A value for the whole null requires an explicit rule for accounting for that variation.

What You'll Learn