Why Discreteness Changes Conditional Error
For a continuous test statistic, a critical boundary can often be placed so that a tail probability equals a chosen significance level. A discrete statistic has only particular attainable values. Its tail probability therefore jumps as the boundary moves from one value to the next. This matters when calculating conditional error: after interim data are observed, the remaining outcomes may have only a few possible values, and their probability masses must be counted explicitly.
The calculation is not a new test of the interim data. It asks how likely the original planned final rejection rule would be to reject, under the null, given the interim information already observed. This conditional probability is the error budget associated with that interim history.
A calculation requires a specified null distribution, a fixed original rejection rule, and a clear statement of what is conditioned on. In the examples below, stagewise observations are independent under the null, the interim count is the first-stage count, and the final statistic is the sum of the stage counts. The endpoint is binary, so each stage count has a binomial distribution. These assumptions give a simple conditional distribution for the future count; in another design, the appropriate conditional distribution must be derived from that design’s model.
For a composite null, conditional error is generally a function of the null parameter as well as the interim history. A conditional calculation at one parameter value does not by itself establish control over the entire null. The nuisance-parameter treatment must match the inferential requirement, as in the earlier material on composite nulls and nuisance parameters.
Summing Over Attainable Outcomes
Suppose the future statistic \(Y\), conditional on interim information \(H_1=h\) and under the null, can take values in a finite set \(\mathcal{Y}(h)\). If the original final rule rejects for outcomes in a set \(\mathcal{R}(h)\), then, without randomization,
If rejection is randomized at an attainable boundary \(y=b\), let \(\gamma\) be the probability of rejecting when \(Y=b\), using a random draw independent of the trial data. If outcomes above the boundary always reject and outcomes below it never reject, then
This expression makes the jump explicit. Moving the cutoff past \(b\) changes the rejection probability by the entire mass \(P(Y=b\mid h,H_0)\). Randomization can include a chosen fraction of that mass rather than including or excluding it all.
Worked Example: Calibrating a Randomized Binomial Boundary
Consider a hypothetical two-stage trial with binary outcomes. Under the simple null, each participant has response probability \(p=0.5\), and all responses are independent. There are four participants per stage. Let \(X\) be the first-stage number of responses and \(Y\) the second-stage number, so \(X+Y\sim\operatorname{Bin}(8,0.5)\) under \(H_0\). Suppose the planned one-sided rule rejects for \(X+Y\geq 7\), and rejects with probability \(\gamma\) if \(X+Y=6\). Choose \(\gamma\) to give overall type I error \(0.10\).
The attainable total counts are integers, so the upper-tail probability jumps between cutoffs. Under \(H_0\),
The rejection probability is \(9/256+\gamma(28/256)\). Set this equal to \(0.10\) and solve:
Thus, when the total is six, the rule rejects with probability about \(59.29\%\). The resulting null rejection probability is exactly \(0.10\), subject to the stated model. The randomization draw must be independent of the participant outcomes.
To calculate conditional error, condition on \(X=x\). Under the stated independent binomial model, \(Y\sim\operatorname{Bin}(4,0.5)\) after observing any \(x\). The original rule rejects if \(Y\geq 7-x\), with randomization when \(Y=6-x\). Its conditional error is therefore
where probabilities of impossible values of \(Y\) are zero. For example, if \(x=3\), rejection is certain when \(Y=4\), and is randomized when \(Y=3\). Since \(P(Y=4)=1/16\) and \(P(Y=3)=4/16\),
This conditional error exceeds \(0.10\), even though the overall type I error is \(0.10\). A particular interim history can leave a larger chance of rejection than the design’s average chance across all interim histories.
Check the Conditional Calculation Against the Design
A useful justification follows directly from conditioning. The interim count \(X\) has a discrete distribution under \(H_0\), and the events \(X=x\) partition the sample space. Therefore the original marginal rejection probability must equal the probability-weighted average of the conditional errors:
This is the law of total probability applied to rejection. It is also a practical check: if the conditional errors are computed from the original rule and the conditional null distributions, their weighted average must recover the design’s original null rejection probability.
Worked Example: Verifying the Conditional Errors
Continue the eight-participant example. Under \(H_0\), \(X\sim\operatorname{Bin}(4,0.5)\), so its probabilities for \(x=0,1,2,3,4\) are \(1/16,4/16,6/16,4/16,1/16\). The conditional errors from the original randomized rule are
For \(x=2\), only \(Y=4\) reaches the randomized total-count boundary, giving \(CE(2)=(83/140)(1/16)=83/2240\). For \(x=4\), \(Y=3\) or \(4\) always rejects, and \(Y=2\) is the randomized boundary, giving \(CE(4)=5/16+(83/140)(6/16)\).
Weight these values by the probabilities of the interim counts. Keeping the fractional terms makes the check exact:
The conditional calculation recovers the calibrated marginal error. A mismatch would indicate an error in the conditional support, the boundary probability, the randomization probability, or the weighting distribution.
Nonrandomized Cutoffs and Their Trade-Off
Randomization is not always required or desirable. A nonrandomized rule can use an attainable cutoff and retain a type I error no larger than the target. It will generally be conservative when no cutoff gives exactly the target. The distinction is especially clear in the eight-participant example.
Worked Example: A Conservative Nonrandomized Rule
Suppose instead that the rule rejects only when \(X+Y\geq 7\). Its overall type I error is
This is below \(0.10\), but it does not attain \(0.10\). Given \(X=3\), rejection requires \(Y=4\), so \(CE(3)=1/16=0.06250\). Given \(X=4\), rejection requires \(Y\geq3\), so \(CE(4)=(4+1)/16=0.3125\). The other interim counts have conditional error zero. The weighted average is
The nonrandomized version controls error more conservatively, while the randomized boundary uses part of the probability mass at six to reach the specified level. The choice is a design decision; a nominal level does not itself require randomization.
| Approach | How the boundary is handled | Main implication |
|---|---|---|
| Nonrandomized discrete cutoff | Reject or do not reject at each attainable value | Simple to implement, but the actual error can fall below the target because of a tail-probability jump |
| Randomized boundary | Reject with a specified probability at one boundary value | Can attain a target exactly under the specified null model, but requires a randomization mechanism |
| Continuous approximation | Approximate a discrete tail by a continuous distribution | May be useful for rough calculations, but does not replace summing exact attainable probabilities when exact conditional error is required |
Diagnostics, Assumptions, and Limitations
Before using a conditional error, check that the calculation matches the planned design. In particular, verify the following:
- Conditioning information: State exactly what was observed. If the interim information includes more than a count, the conditional distribution must reflect that additional information.
- Future support: List the attainable values of the future statistic given the observed history. Do not include impossible values or omit possible ones.
- Null model: Use the null distribution of future outcomes conditional on the interim information. Independence of stages and a common response probability were assumptions in the binomial examples, not universal facts.
- Original rule: Include every part of the planned rejection rule, including strict versus inclusive inequalities and any randomization already specified.
- Numerical check: Average the conditional errors over the null distribution of the interim data. The result should equal the original marginal rejection probability.
A common mistake is to see \(CE(3)\approx0.2107\) in the randomized example and conclude that the design violates a \(0.10\) significance level. That reasoning treats a conditional probability at one interim history as though it were the unconditional type I error. The correction is to distinguish the two quantities: the conditional error describes the rejection chance for a specified history under the null, while the marginal error averages those conditional chances over all histories under the null.
Another pitfall is to approximate a discrete tail with a smooth curve and then treat the result as exact. When a boundary probability is appreciable, that approximation can materially change the conditional error. For an exact calculation, sum the null probabilities at each attainable future value and apply the boundary rule directly.
Randomization attains a chosen rejection probability under the null model used to calculate it; it does not remove model uncertainty or guarantee the same level under every nuisance-parameter value. Nor does a conditional error calculation alone determine which adaptation to make. It supplies a probability tied to the original rule and a specified interim history. Its validity depends on the assumptions, conditioning set, and rejection rule being stated correctly.
Check Your Understanding
Answer each question using the definitions and calculations above. Full solutions follow the questions.
- In the eight-participant example, what is the conditional error when \(X=1\), and why?
- For that example, why is \(CE(4)\) allowed to exceed the marginal significance level \(0.10\)?
- If a discrete boundary has upper-tail probability \(0.018\) and boundary mass \(0.012\), what boundary randomization probability gives a target error of \(0.024\), assuming the upper tail is the only other rejection probability?
- What weighted-average identity should hold between conditional errors and the original marginal rejection probability?
Solutions. (1) \(CE(1)=0\): even the largest future count \(Y=4\) gives total \(X+Y=5\), below both the randomized boundary at six and the always-reject region at seven or more. (2) A conditional error is computed for one interim history, not averaged over histories. The overall error is \(0.10\) because the conditional errors, including the larger value at \(X=4\), are weighted by their null probabilities. (3) Solve \(0.018+\gamma(0.012)=0.024\), giving \(\gamma=(0.024-0.018)/0.012=0.5\). (4) The identity is \(P(R\mid H_0)=\sum_x P(X=x\mid H_0)CE(x)\); it follows by partitioning the rejection probability over the possible interim values.