Exact Conditional Error for Discrete Endpoints

Why Conditional Error Needs Special Care for Discrete Data

When interim results influence a trial’s continuation, sample size, or test statistic, the final decision rule may differ from the one planned initially. The relevant error-control question is not merely whether the new rule has a small average rejection probability. It is whether, after each possible interim result, the continuation rule spends no more null rejection probability than the original test had available at that result. With discrete endpoints, those available probabilities often jump in sizable increments.

This tutorial develops that calculation for binomial and Fisher exact tests. The focus is the conditional rejection probability under the original test, not a general procedure for selecting adaptations. The argument assumes that the interim data are recorded before the future outcomes used in the continuation, and that the probability model for those future outcomes is specified under the null.

Definition: Let $D_1=d_1$ denote an observed interim result, and let $R_0$ be the rejection event of the original, fixed-design test. The conditional error at null parameter value $\theta$ and interim result $d_1$ is $$ CE_\theta(d_1)=P_\theta(R_0\mid D_1=d_1). $$ It is the original test’s conditional probability of rejection, not the probability that the null hypothesis is true.

For a simple null, $\theta$ is fixed, so a continuation test can be compared directly with $CE_\theta(d_1)$. For a composite null, the comparison must hold at every null value relevant to the model. In particular, a single scalar allowance that is required to work uniformly cannot exceed the infimum of the conditional errors over those values. A supremum would be too permissive: it could exceed the original conditional error at some null values.

Binomial Continuation Probabilities

Suppose a binary endpoint is observed independently for $n_1$ participants at interim and $n_2$ further participants if the trial continues. Under a simple null, assume each participant has the same response probability $p_0$, with independent Bernoulli outcomes across participants and no change in endpoint definition or ascertainment. If $x_1$ responses have occurred by interim, the number $Y$ of responses among the future $n_2$ participants satisfies $Y\sim\operatorname{Bin}(n_2,p_0)$, independently of the interim outcomes.

If the original rule rejects when the total number of responses is at least $c$, then the conditional error is the probability that the future count supplies enough responses to reach that boundary:

CE_{p_0}(x_1)=P_{p_0}(x_1+Y\ge c) =\sum_{y=\max(0,c-x_1)}^{n_2} \binom{n_2}{y}p_0^y(1-p_0)^{n_2-y}.

This is an exact tail probability under the stated model. It is not generally equal to the original test’s nominal significance level: conditioning on a particular interim count changes the probability of reaching the final boundary.

Worked Example: Binomial Conditional Error From an Interim Count

Consider a hypothetical trial with 5 interim participants and 5 future participants. Under the simple null, let $p_0=0.30$. The original test rejects if at least 5 of the 10 participants respond. Suppose 2 responses have been observed at interim. Calculate the original test’s conditional error.

The total must reach 5, so at least $5-2=3$ of the 5 future participants must respond. Under the independent Bernoulli model, $Y\sim\operatorname{Bin}(5,0.30)$, and

CE_{0.30}(2)=P(Y\ge 3) =\binom{5}{3}(0.30)^3(0.70)^2 +\binom{5}{4}(0.30)^4(0.70) +(0.30)^5.

The three terms are $10(0.027)(0.49)=0.1323$, $5(0.0081)(0.70)=0.02835$, and $0.00243$. Therefore,

CE_{0.30}(2)=0.1323+0.02835+0.00243=0.16308.

At this interim count, the original test has a conditional rejection probability of 0.1631 under the null response rate of 0.30. This is the relevant allowance for a continuation rule at this state, not automatically the original test’s unconditional Type I error level.

Fisher Exact Tests After Interim Counts

For two independent groups with a binary endpoint, Fisher’s exact test conditions on the final group sizes and the total number of successes. Under the null hypothesis of equal response probabilities, conditional on those margins, the number of successes in one group has a hypergeometric distribution. A one-sided Fisher test for higher response in group A rejects when its upper-tail probability is at most the chosen test level.

At interim, however, the future total number of successes is not yet known. Conditional on the interim counts, the exact conditional error is obtained by summing over possible future outcomes under the null, applying the original Fisher decision to each resulting final table, and adding the probabilities of the outcomes that would reject. Under a common null response probability $p$, each future participant’s outcome is Bernoulli with probability $p$, independently of the other future outcomes. The conditional error can therefore depend on the nuisance parameter $p$, even though the Fisher test itself conditions on final margins.

Worked Example: Fisher Exact Conditional Error by Enumeration

In a hypothetical two-stage study, one participant per group has been observed at interim: group A has 1 success and group B has 0. One further participant per group will be observed. Under the null, suppose all outcomes are independent Bernoulli with a common response probability $p$. The original one-sided Fisher test rejects at level 0.20 when the conditional upper-tail p-value is at most 0.20. Find its conditional error given the interim counts.

Let $U$ and $V$ denote the future success indicators in groups A and B, respectively. To identify the final rejection event, consider the four possible pairs. If $(U,V)=(1,0)$, the final table has 2 successes in A and 0 in B. Given 2 successes among 4 participants, the Fisher upper-tail probability of placing both in A is $1/\binom{4}{2}=1/6$, which is at most 0.20. For $(U,V)=(0,0)$, the final table is (1,0), with upper-tail probability $2/4=0.50$. For $(0,1)$, it is (1,1), with upper-tail probability $5/6$. For $(1,1)$, it is (2,1), with upper-tail probability $1/2$. Thus only $(U,V)=(1,0)$ leads to rejection.

The model gives $P_p(U=1)=p$ and $P_p(V=0)=1-p$, independently. Therefore,

$$ CE_p(1,0)=P_p(U=1,V=0)=p(1-p). $$

For example, at $p=0.40$, the conditional error is $0.40(0.60)=0.2400$. At $p=0.50$, it is $0.2500$. This enumeration makes clear both which future outcomes reject and why the conditional error can vary with a nuisance parameter under the composite null.

Discrete Jumps and Randomized Boundary Decisions

A discrete continuation statistic cannot usually attain every desired rejection probability with a nonrandomized cutoff. For example, the probability can fall from one tail area to a substantially smaller tail area when the critical count increases by one. If a test is permitted to randomize, it can use a boundary count with a specified probability. The randomization must be independent of the endpoint data and specified as part of the decision rule.

Let $Y$ be a discrete continuation statistic and let $a$ be the allowed conditional rejection probability under a simple null. Choose a boundary value $k$ such that $P(Y>k)\le a\le P(Y\ge k)$. Reject when $Y>k$, and reject with probability $\gamma$ when $Y=k$, where

\gamma=\frac{a-P(Y>k)}{P(Y=k)}.

Since $0\le a-P(Y>k)\le P(Y=k)$, this gives $0\le\gamma\le1$. The resulting rejection probability is $P(Y>k)+\gamma P(Y=k)=a$. If randomization is not acceptable, a nonrandomized test can use a cutoff with rejection probability no greater than $a$, but it may leave part of the allowance unused.

Worked Example: Randomizing at a Binomial Boundary

Suppose a hypothetical continuation test has $Y\sim\operatorname{Bin}(5,0.30)$ under a simple null and is allowed conditional rejection probability $a=0.0200$. Find a randomized upper-tail rule that uses this allowance exactly.

The upper tail above 4 is $P(Y>4)=P(Y=5)=(0.30)^5=0.00243$. The probability at the boundary 4 is $P(Y=4)=\binom{5}{4}(0.30)^4(0.70)=0.02835$. Thus $k=4$ brackets the allowance, since $0.00243\le0.0200\le0.03078$. The boundary randomization probability is

\gamma=\frac{0.0200-0.00243}{0.02835} =\frac{0.01757}{0.02835} \approx 0.6198.

The rule always rejects at $Y=5$, rejects at $Y=4$ with probability approximately 0.6198, and otherwise does not reject. Using the unrounded ratio, its null rejection probability is $0.00243+\left(0.01757/0.02835\right)(0.02835)=0.0200$. The example illustrates how a randomized boundary fills a discrete probability gap; it does not imply that randomization is required in a clinical trial.

Constraining an Adapted Continuation Test

Suppose the original test has conditional error $CE_\theta(d_1)$ at interim state $d_1$, and the adapted continuation rule has conditional rejection probability $q_\theta(d_1)$. A sufficient preservation condition is the pointwise inequality

q_\theta(d_1)\le CE_\theta(d_1) \quad\text{for every relevant null }\theta \text{ and every interim state }d_1.

This condition has a direct justification. Under any fixed null value $\theta$, average the adapted conditional rejection probability over the distribution of interim results. The corresponding average for the original test is its original rejection probability:

P_\theta(\text{adapted rejection}) =\sum_{d_1}P_\theta(D_1=d_1)q_\theta(d_1) \le \sum_{d_1}P_\theta(D_1=d_1)CE_\theta(d_1) =P_\theta(R_0).

The last equality follows by partitioning the original rejection event according to the possible interim states. Thus the adapted test does not exceed the original test’s rejection probability under any null value for which the inequality holds. If the original test has Type I error at most $\alpha$ throughout the null, the adapted test also has Type I error at most $\alpha$. The argument requires that the conditional probabilities correctly reflect the null model and the actual continuation rule.

When a scalar allowance is needed for a composite null, a sufficient choice at a given state is no larger than $\inf_{\theta\in\Theta_0} CE_\theta(d_1)$, where $\Theta_0$ is the relevant null parameter set. Often it is better to retain the pointwise inequality, because the infimum may be very small or zero even when useful pointwise constraints are available. Using the supremum instead would not establish the required inequality at every null value.

Worked Example: Why a Supremum Is Not a Safe Uniform Allowance

In the Fisher example, $CE_p(1,0)=p(1-p)$ for $0\le p\le1$. Its supremum is 0.25, attained at $p=0.50$, while its infimum is 0, attained at the endpoints. Suppose an adapted continuation rule has conditional rejection probability $q_p(1,0)=0.20$ for every $p$.

A cap of 0.25 based on the supremum would appear to permit this rule. But at $p=0.10$, the original conditional error is $0.10(0.90)=0.0900$, so $0.20\le0.0900$ is false. At $p=0.50$, the pointwise comparison does hold, since $0.20\le0.25$. The supremum therefore does not supply a uniform allowance.

A scalar allowance valid at every $p\in[0,1]$ cannot exceed the infimum, which is zero. That conservative scalar fact does not itself rule out all useful adaptations: it indicates that the pointwise condition must be examined if a nontrivial rule is sought. For this proposed constant $q_p=0.20$, the pointwise condition fails for null values near either endpoint.

Diagnostics, Alternatives, and Limitations

A conditional-error calculation is only as reliable as its conditioning and probability model. Before using it, verify the following:

Check the interim state. The interim count, group allocation, and stopping or continuation status must be defined unambiguously. Conditioning on a different selection event can give a different conditional probability.
Check the future-outcome model. For the binomial calculation, future outcomes must have the assumed response probability and independence structure under the null. For Fisher enumeration, include every possible future table and evaluate the original test on the resulting final table.
Check the rejection boundary. Discrete tail probabilities must include or exclude the boundary as specified. A one-count error can materially change conditional error.
Check nuisance parameters pointwise. A bound at one null value does not imply a bound across a composite null. Evaluate the inequality over the relevant null parameter space or justify a valid uniform allowance.

An asymptotic normal approximation can be easier to calculate, but it may misrepresent tail probabilities when counts are small or the distribution is highly discrete. Exact enumeration avoids that approximation under the specified model, at the cost of conservatism for nonrandomized tests and potentially cumbersome calculations. Randomization can remove conservatism at a chosen boundary, but it introduces a randomized decision that may be operationally unattractive. Fisher’s conditioning handles nuisance response probability for its fixed-table test, whereas a conditional-error calculation given interim counts may still depend on that probability. Neither exactness nor conditioning repairs a misspecified model, changed endpoint definition, or unaccounted adaptation.

Common mistake: “The adapted conditional rejection probability is below the largest conditional error over the null, so it is safe.” This is wrong when conditional error varies with a nuisance parameter: the largest value can exceed the original allowance at other null values. The correction is to require $q_\theta(d_1)\le CE_\theta(d_1)$ pointwise, or to use a scalar allowance no larger than the infimum over the relevant null values.

Check Your Understanding

Use the definitions and methods above. Each solution follows the question.

For $Y\sim\operatorname{Bin}(4,0.25)$, an original rule rejects if $Y\ge3$. Calculate its rejection probability.
Solution: $P(Y\ge3)=\binom{4}{3}(0.25)^3(0.75)+(0.25)^4=0.046875+0.00390625=0.05078125$, or approximately 0.05078.
In the Fisher example, what is the conditional error at $p=0.20$?
Solution: The rejection event is $U=1,V=0$, so $CE_{0.20}=0.20(0.80)=0.1600$.
For the randomized-binomial example, what is the rejection probability if the test rejects at $Y=5$ and at $Y=4$ with probability 0.50?
Solution: It is $P(Y=5)+0.50P(Y=4)=0.00243+0.50(0.02835)=0.016605$, below the allowance 0.0200.
Why does $q_\theta(d_1)\le CE_\theta(d_1)$ preserve the original rejection bound?
Solution: Multiplying each pointwise inequality by $P_\theta(D_1=d_1)$ and summing over interim states bounds the adapted rejection probability by the original rejection probability through the law of total probability.
If a conditional error ranges from 0.04 to 0.18 over the null parameter values, is 0.18 a valid uniform scalar cap?
Solution: Not in general. A uniform cap must be no greater than the infimum, 0.04, unless the pointwise condition is retained and justified separately.