Rejection Probability and Early Stopping

Two Probabilities That Answer Different Questions

In a monitored trial, an interim analysis can lead to rejection and an early stop, or it can lead to continued follow-up and another analysis. Two questions therefore need separate answers: How often would the trial reject at any scheduled look? And how often would it stop before the planned final look? They are related, but they are not interchangeable.

The distinction also depends on the scenario being considered. Under the trial-wise null hypothesis, an early efficacy rejection is a false rejection. Under a specified alternative, the same kind of early rejection is evidence in favor of the alternative, and its probability is part of the trial's power. Neither probability alone describes every way a trial might stop; here, stopping means stopping for efficacy under a prespecified first-rejection rule.

Write \(P_0\) for probability under the trial-wise null hypothesis \(H_0\), and \(P_\theta\) for probability under a specified alternative with treatment effect \(\theta\). The corresponding quantities have different interpretations:

• \(P_0(R_{\mathrm{any}})\) is the overall false-rejection probability, or type I error, for the complete monitoring rule.
• \(P_\theta(R_{\mathrm{any}})\) is the rejection probability under that alternative, commonly called power for that alternative.
• \(P_0(E)\) is the probability of stopping early for a false efficacy rejection under the null.
• \(P_\theta(E)\) is the probability of stopping early for efficacy under the alternative.

These are probabilities over repeated hypothetical trials following the specified model and monitoring plan. They are not probabilities that \(H_0\) is true, nor do they describe the chance that an individual trial's conclusion is correct.

How the Probabilities Fit Together

Under the first-rejection rule, the events \(R_1,\ldots,R_K\) cannot occur together: a trial that rejects at look 1 has already stopped and cannot first reject at look 2. They form disjoint paths to rejection. Early rejection paths are only some of those paths; the final-look path also contributes to any-look rejection.

Taking probabilities under either scenario gives the basic decomposition:

Thus early efficacy stopping probability is no greater than any-look rejection probability. Under \(H_0\), the gap \(P_0(R_{\mathrm{any}})-P_0(E)\) is the probability of a false rejection that first occurs at the final look. Under an alternative, the corresponding gap is the probability of rejecting at the final look after no earlier rejection.

This decomposition assumes a fixed schedule of \(K\) looks, a prespecified rejection rule at each look, and stopping at the first rejection. It also assumes that the probability model used for \(P_0\) or \(P_\theta\) correctly represents the trial under that scenario. It does not count stopping for safety, administrative reasons, or futility. If those stopping routes are part of a real protocol, they must be defined and their probabilities reported separately.

Worked Null and Alternative Scenarios

The examples use hypothetical probability summaries for a two-look design. The path probabilities are assumed to have been obtained under the stated model and prespecified rule. A two-look design makes the distinction visible without requiring a particular test statistic or boundary calculation.

Derivation and Practical Checks

The key probability identity follows by partitioning all paths. Let \(R_k\) denote first rejection at look \(k\). Because the trial stops on first rejection, for \(j\ne k\), \(R_j\cap R_k=\varnothing\). The event of any rejection is the union of these mutually exclusive events. Therefore, by additivity of probability,

The same derivation applies under \(P_0\) and \(P_\theta\); only the probability distribution changes. This identity does not require the test statistics at different looks to be independent. It requires that the first-rejection paths are defined consistently and are mutually exclusive. In cumulative monitoring, the test statistics are generally dependent, so path probabilities must reflect that dependence; simply treating per-look results as independent can give incorrect probabilities.

In practice, check a probability summary in three ways. First, confirm that each trial contributes to one and only one path: early rejection, later first rejection, or no rejection. Second, verify that the path probabilities sum to one, allowing for rounding. Third, confirm the scenario attached to every number: a null probability concerns false rejection, while an alternative probability concerns power or stopping behavior at that effect.

Choosing What to Report

A final-look-only analysis asks whether the planned final test rejects. A sequential any-look summary asks whether any prespecified look rejects, and so includes early as well as final-look paths. The early stopping probability is more operational: it quantifies how frequently the trial would end before reaching the planned final analysis because the efficacy rule was crossed. These summaries are complementary, not competing definitions of one probability.

For a complete description of an efficacy-only monitoring rule, report at least the null any-look rejection probability and, when early stopping is relevant, the null early-rejection probability. Under a scientifically meaningful alternative, report power and early efficacy stopping probability separately. If useful, also report the conditional fraction of rejections that occur early, explicitly naming its denominator.

• Null scenario: early efficacy stopping is a subset of false rejections; it is not the whole type I error unless final-look rejection is impossible.
• Alternative scenario: early efficacy stopping is part of power, not a substitute for power.
• Additional stopping routes: safety, futility, or administrative stopping require their own event definitions; they cannot be silently included in the efficacy stopping probability.

A useful diagnostic is an impossible ordering. If a report gives an early efficacy stopping probability greater than the any-look rejection probability under the same scenario and rule, at least one definition, denominator, or calculation is wrong. Likewise, if a purported conditional percentage \(P(E\mid R_{\mathrm{any}})\) exceeds 1 or is calculated using all trials rather than rejecting trials, it is not the stated conditional probability.

Common Mistake

Wrong reasoning: “Only 1.8% of trials stopped early under the null, so the overall false-positive probability is 1.8%.” This ignores trials that continued past the interim look and falsely rejected at the final look.

Correction: Add the probabilities of all mutually exclusive first-rejection paths to obtain the any-look error. In the null example, that is \(0.018+0.037=0.055\). The early stopping probability remains 0.018, because it counts only the interim rejection path. Always label the scenario, event, and denominator before interpreting a reported percentage.