Why Condition on Interim Data?
An interim analysis divides a trial into information available so far and outcomes that may still be observed. The original trial’s type I error rate summarizes the probability of eventual rejection before any data are seen. After the interim data are observed, however, the probability of rejection under the null can differ across interim outcomes. The conditional error function describes that variation.
This quantity is useful because it characterizes the remaining null rejection probability at a particular interim result, according to a specified test and probability model. It is not itself an observed p-value, nor is it the probability that the null hypothesis is true. Its definition requires a fixed rejection rule and a specified null distribution for the trial data.
The word eventually matters. The event \(R\) includes the entire planned decision process, including any interim stopping decisions and the rule for reaching a final decision. Thus \(C_0(i)\) is a probability under the null model for future outcomes, conditional on the interim information; it is not a count of how much evidence the interim data have already accumulated.
For a simple null hypothesis, \(P_0\) refers to one specified null distribution. For a composite null, such as a set of parameter values, the conditional probability can depend on the particular null parameter value. One can write \(C_{\theta}(I)=P_{\theta}(R\mid I)\) for each \(\theta\) in the null parameter space. A claim about error control for a composite null must specify whether it holds pointwise for each such \(\theta\) or uses a worst-case bound. A single conditional error number is not automatically valid for every null parameter value.
Relation to Unconditional Type I Error
The unconditional type I error is the null probability of rejecting \(H_0\), calculated before observing the interim data. If the original test has exact size \(\alpha\) under a simple null, then \(P_0(R)=\alpha\). Conditional errors need not equal \(\alpha\) for every interim outcome. Instead, their average over the null distribution of interim data equals the unconditional rejection probability.
This identity is the law of total expectation applied to the indicator of rejection. Since \(1_R\) equals 1 when rejection occurs and 0 otherwise, \(E_0(1_R)=P_0(R)\). Conditioning first on \(I\) gives
If the original procedure has type I error at most \(\alpha\), then the null expectation of its conditional error function is at most \(\alpha\). This statement does not mean that every interim result has conditional error at most \(\alpha\); some results can have a larger conditional error and others a smaller one. The unconditional rate averages over how often those results occur under the null.
Worked Example: Conditional Errors Average to the Type I Error
Suppose a fully specified hypothetical two-stage test has two possible interim categories, \(G\) and \(B\). Under the simple null, \(P_0(G)=0.40\) and \(P_0(B)=0.60\). The planned final rule has conditional rejection probabilities \(P_0(R\mid G)=0.02\) and \(P_0(R\mid B)=0.07\). These probabilities include the possibility of rejection at any later decision point.
The conditional error function takes the values \(C_0(G)=0.02\) and \(C_0(B)=0.07\). By partitioning on the interim category,
Thus the unconditional type I error is \(0.050\), even though the conditional error is \(0.02\) for one interim category and \(0.07\) for the other. The calculation assumes that the two categories exhaust the interim outcomes and that the stated conditional probabilities come from the null model and the specified original rule. It does not establish error control for any different rule.
Calculating a Conditional Error from Future Outcomes
Once the interim result is fixed, conditional error is calculated by identifying which possible future outcomes lead to rejection and summing their probabilities under the null, conditional on that interim result. The future outcomes need not be independent of the interim data. The relevant probabilities are conditional probabilities given those data.
Worked Example: A Rejection Region for a Future Count
Consider a hypothetical trial after an interim result \(I=i\). Under the null, the number \(K\) of responses among the remaining participants has the following conditional distribution: \(P_0(K=0\mid I=i)=0.50\), \(P_0(K=1\mid I=i)=0.30\), and \(P_0(K=2\mid I=i)=0.20\). Suppose the specified final rule rejects \(H_0\) exactly when \(K=2\).
The rejection event, conditional on this interim result, is \(R=\{K=2\}\). Therefore,
The conditional error is \(0.20\), or 20%, under the stated null model and rule. The probabilities sum to \(0.50+0.30+0.20=1.00\), and the calculation uses the distribution conditional on \(I=i\), not the marginal distribution of \(K\) before interim data are observed. If the rule instead rejected for \(K\geq 1\), the conditional error would be \(0.30+0.20=0.50\). The value depends on both the conditional null distribution and the rejection rule.
Conditional Error Is Not Conditional Power
The conditioning information may be the same in two calculations while the probability model differs. Conditional error is calculated under the null. Conditional power is the probability of eventual rejection conditional on the interim data under a specified alternative model or effect. Neither quantity is the probability that its corresponding hypothesis is true.
Worked Example: Null Error and Alternative Power at the Same Interim Result
Suppose that for a particular hypothetical interim result \(I=i\), the specified procedure has null rejection probability \(P_0(R\mid I=i)=0.03\). Under a prespecified alternative model representing a treatment benefit of interest, suppose its conditional rejection probability is \(P_1(R\mid I=i)=0.68\).
Under the null, the conditional error is
Under the alternative model, conditional power is
These are not competing estimates of one probability: they use different distributions. The first describes the chance of a false rejection under the null, given the interim result; the second describes the chance of rejection if the specified alternative model holds, given that same result. To calculate either value, the relevant conditional distribution of future data and the complete rejection rule must be specified. The numerical values here are hypothetical inputs, not results inferred from the interim result alone.
What the Function Does—and Does Not—Represent
A conditional error function is a property of a test procedure together with its null model and conditioning information. It can be useful as a benchmark for understanding how much null rejection probability remains after an interim outcome. It does not, by itself, prescribe an adaptation or prove that a modified procedure will control type I error.
The following quantities answer different questions:
| Quantity | Probability statement | Interpretation |
|---|---|---|
| Unconditional type I error | \(P_0(R)\) | Null probability of rejection before conditioning on interim data |
| Conditional error | \(P_0(R\mid I=i)\) | Null probability of eventual rejection for a specified interim result |
| Conditional power | \(P_1(R\mid I=i)\) | Alternative-model probability of eventual rejection for a specified interim result |
| Interim p-value | A tail probability based on an interim statistic under a null reference distribution | A measure of how extreme the interim statistic is under that reference distribution, not the probability of eventual rejection |
In particular, an interim p-value cannot generally be substituted for conditional error. A p-value is defined using a statistic and its reference distribution; conditional error is defined using the full eventual rejection event, the null distribution of future data conditional on interim information, and the planned decision rule. Likewise, a small conditional error does not imply low conditional power, and a large conditional power does not imply an inflated null error. The quantities describe different operating characteristics.
Assumptions, Diagnostics, and Limitations
Interpreting \(C_0(i)\) requires several pieces to be clear. The null model must be specified, the interim information being conditioned on must be defined, and the original decision rule must be complete enough to determine whether eventual rejection occurs. The conditional distribution of future data must also be appropriate for the observed interim data. If the design uses randomization, the probability calculation must account for the stated randomization mechanism.
A useful diagnostic is to check the calculation from both directions: verify that the conditional probabilities of all possible future outcomes sum to one, and verify that the event \(R\) includes exactly the outcomes that the planned rule rejects. For a small set of interim categories, also check that weighting their conditional errors by their null probabilities reproduces the unconditional null rejection probability. A mismatch can reveal a missing rejection path, an omitted interim category, or use of marginal rather than conditional probabilities.
The function can be difficult to summarize with one number when the null is composite, when the interim data are continuous, or when the conditional distribution depends on nuisance parameters. In those settings, the value may vary over interim outcomes and null parameter values. Conditional probabilities are also only defined up to sets of interim outcomes having null probability zero; a value assigned at an impossible interim result has no effect on the unconditional type I error. These details limit what can be concluded from a conditional error value without its model and conditioning context.
Common Mistake
Wrong reasoning: “The original test has type I error \(0.05\), so the conditional error must be \(0.05\) after every interim result.” This confuses an average with each conditional value. The law of total expectation gives \(P_0(R)=E_0\{C_0(I)\}\); it does not require \(C_0(i)\) to be constant. For example, in the first worked example, conditional errors of \(0.02\) and \(0.07\) average to \(0.05\) using the null probabilities of the interim categories.
A related error is to call conditional error “the chance the null is true after seeing the interim data.” It is instead a probability of a future decision under a specified null model, conditional on the interim data. It does not assign probability to the truth of a hypothesis.
Check Your Understanding
Answer each question using the definitions and probability statements above.
- Under a simple null, two interim categories have probabilities \(0.25\) and \(0.75\), with conditional errors \(0.04\) and \(0.08\). What is the unconditional type I error?
- For a fixed interim result, the conditional null probabilities of future outcomes \(A,B,C\) are \(0.20, 0.50, 0.30\), and the procedure rejects for \(A\) or \(C\). Find the conditional error.
- At a specified interim result, conditional error is \(0.02\) and conditional power under a prespecified alternative is \(0.60\). Which probability uses the null model, and what does the other probability describe?
- Why does an unconditional type I error of \(0.05\) not imply a conditional error of \(0.05\) for every interim result?
Solutions. (1) \(P_0(R)=(0.25)(0.04)+(0.75)(0.08)=0.01+0.06=0.07\), so the unconditional type I error is \(0.07\). (2) The rejection event is \(A\cup C\), so the conditional error is \(0.20+0.30=0.50\). (3) The conditional error \(0.02\) uses the null model and is the conditional probability of eventual rejection under that model. The conditional power \(0.60\) uses the specified alternative and is its conditional probability of eventual rejection. (4) The unconditional rate is the null expectation of the conditional error function. Different interim outcomes can have different conditional errors while their null-probability-weighted average equals \(0.05\).