Deriving Spending Boundaries by Conditional-Error Recursion

Why Conditional Calibration Is Needed

A multi-stage trial gives repeated opportunities to reject one or more treatment hypotheses. A boundary calibrated as though it were the only boundary in the trial will generally not control the probability of at least one false rejection over all stages. The relevant probability is cumulative: it includes every rejection opportunity, across arms and looks.

A spending schedule specifies how much of the familywise error budget may have been used by each look. To turn that schedule into boundaries, it is useful to ask a conditional question at every stage: among trials that have not rejected previously, what probability of rejection can be spent now? The answer is not usually the increment in cumulative spending itself. Previous non-rejections have already reduced the set of trials still at risk.

The recursion below calibrates the familywise rejection event for a specified null configuration and a specified joint distribution of the stagewise statistics. It does not, by itself, establish strong familywise error control over every possible pattern of true and false hypotheses; that requires the appropriate intersection-testing argument.

Cumulative Spending and Conditional Hazards

Definition: Let \(S_j\) denote the event that no hypothesis has been rejected through stage \(j\), with \(S_0\) denoting the event that the trial has not yet begun. Let \(A_j\) be the prespecified cumulative familywise error spending by stage \(j\), where \(A_0=0\) and \(0\leq A_1\leq\cdots\leq A_K=\alpha\). The conditional stage-\(j\) rejection probability, or hazard, is \(q_j=P(S_{j-1}\cap S_j^c\mid S_{j-1})\), evaluated under the calibration null.

The cumulative spending target means \(P(S_j)=1-A_j\). The trials in \(S_{j-1}\) are precisely the trials still eligible to reject at stage \(j\). Since the stage-\(j\) rejection event among those trials is \(S_{j-1}\cap S_j^c\),

q_j =\frac{P(S_{j-1}\cap S_j^c)}{P(S_{j-1})} =\frac{A_j-A_{j-1}}{1-A_{j-1}} =1-\frac{1-A_j}{1-A_{j-1}}.

Thus the increment \(A_j-A_{j-1}\) is an unconditional increment in error probability, whereas \(q_j\) is a probability conditional on having survived all earlier looks. The denominator is the probability of still being at risk. Equivalently, the recursion can be written as

P(S_j)=P(S_{j-1})(1-q_j),\qquad 1-A_j=(1-A_{j-1})(1-q_j).

These identities are exact for a prespecified cumulative schedule. They do not assume that stagewise statistics are independent. Dependence matters when translating \(q_j\) into a boundary: the distribution of the stage-\(j\) statistic among survivors may differ from its unconditional distribution.

Calibrating Boundaries from the Joint Null Law

Suppose stage \(j\) has statistics \(Z_{1j},\ldots,Z_{mj}\) for the arms currently tested, and let \(c_j\) be a common upper boundary. For illustration, rejection at that look occurs if at least one statistic crosses it. Write \(C_j\) for the stage-\(j\) non-rejection region, so in this example \(C_j=\{Z_{1j}<c_j,\ldots,Z_{mj}<c_j\}\). The correct calibration equation is

P(S_j) =\int_{S_{j-1}} P\!\left((Z_{1j},\ldots,Z_{mj})\in C_j \mid H_{j-1}=h\right) \,dP(h) =1-A_j.

Here \(H_{j-1}\) denotes the data and decisions observed through stage \(j-1\), and the integral averages the stage-\(j\) conditional survival probability over histories that have not rejected. A boundary can be found by solving this equation, often using numerical integration or simulation from the specified joint null model. Arm-specific boundaries replace \(C_j\) with the corresponding joint non-rejection region.

This equation also identifies the assumptions behind a calibration. The spending schedule and look times must be specified; the null distribution used for calibration must correctly represent the stagewise statistics and their dependence; and the rejection rule and any arm-selection or dropping rules must be included in the event being calibrated. If the actual design uses a different joint law or a different decision rule, the calculated boundaries need not attain the stated spending targets.

Worked Example: Three Looks with Two Independent Arms

Consider a hypothetical design with two arms and three looks. Under the calibration null, suppose the two arm statistics are independent standard normal variables at each look, and the pairs of statistics are independent across looks. At each look, reject if either arm statistic reaches a common boundary. These independence assumptions make the example simple; they are not required for the general recursion. Let cumulative spending be \(A_1=0.010\), \(A_2=0.030\), and \(A_3=0.050\).

First calculate the conditional rejection probabilities:

q_1=0.010,\qquad q_2=\frac{0.030-0.010}{1-0.010}=0.02020,\qquad q_3=\frac{0.050-0.030}{1-0.030}=0.02062.

At each look the arm statistics are independent standard normal variables. Therefore the probability that neither statistic crosses \(c_j\) is \(\Phi(c_j)^2\), where \(\Phi\) is the standard normal cumulative distribution function. Setting the stagewise rejection probability to \(q_j\) gives \(1-\Phi(c_j)^2=q_j\), so

c_j=\Phi^{-1}\!\left(\sqrt{1-q_j}\right).

Substitution gives \(c_1=\Phi^{-1}(\sqrt{0.990})\approx2.576\), \(c_2=\Phi^{-1}(\sqrt{0.97980})\approx2.321\), and \(c_3=\Phi^{-1}(\sqrt{0.97938})\approx2.313\). The boundary at each look is applied to each arm’s standard normal statistic.

To verify the target, the probability of surviving all three looks is \(0.990(1-0.02020)(1-0.02062)=0.9500\), to rounding. Hence the probability of at least one rejection is \(1-0.9500=0.0500\), the specified final spending. The calculation uses the conditional hazards, not the raw increments \(0.010,0.020,0.020\) as if each were the conditional rejection probability.

Why Dependence Changes the Boundary

The recursion for \(q_j\) depends only on the spending schedule. The boundary solving the calibration equation also depends on the joint distribution. For a common boundary across \(m\) arms, the stagewise rejection probability is \(P(\max_i Z_{ij}\geq c_j\mid S_{j-1})\), not generally \(m\{1-\Phi(c_j)\}\). That sum is a Bonferroni upper bound, not an exact probability except in special cases.

Worked Example: Perfectly Correlated Arm Statistics

Suppose, hypothetically, that at one look two arm statistics have standard normal marginal distributions and are perfectly positively correlated: \(Z_{1}=Z_{2}\) under the null. The target conditional rejection probability is \(q=0.020\). Since the two statistics are identical, rejecting if either crosses \(c\) is just the event \(Z_1\geq c\). Thus \(1-\Phi(c)=0.020\), giving \(c=\Phi^{-1}(0.980)\approx2.054\).

If instead the two statistics were independent, the same familywise stagewise probability would require \(1-\Phi(c)^2=0.020\), giving \(c=\Phi^{-1}(\sqrt{0.980})\approx2.326\). The different boundaries reflect different joint null laws, even though the arm-wise marginal distributions are unchanged. Calibrating by marginal distributions alone cannot distinguish these cases.

Conditional Error Along Observed Histories

There are two related but distinct conditional quantities. The hazard \(q_j\) above averages over all histories that survived to stage \(j\). A conditional error calculation can instead be specific to an observed history \(h\): it is the probability, under the null, of a future rejection given that history. Such history-specific probabilities matter when future testing or arm selection is adapted to interim data. Matching an average hazard does not automatically preserve the allowed error probability separately for every possible history.

The following small hypothetical example makes the averaging explicit. Suppose stage 1 has spent \(A_1=0.020\), and the target after stage 2 is \(A_2=0.050\). The required stage-2 hazard is

q_2=\frac{0.050-0.020}{1-0.020}=0.03061.

Worked Example: Averaging Over Surviving Interim Histories

Suppose that, conditional on no stage-1 rejection, two types of interim history, \(A\) and \(B\), are equally likely. At stage 2 a binary score is either 0 or 1, and the rule rejects when the score is 1. Under the null, suppose the score equals 1 with probability \(0.050\) after history \(A\), and probability \(0.01122\) after history \(B\). These are hypothetical conditional distributions; the rejection probabilities are defined directly by the score rule.

The stage-2 rejection probability among survivors is the average of the two history-specific probabilities: \(q_2=0.5(0.050)+0.5(0.01122)=0.03061\). The cumulative probability of rejection by stage 2 is therefore \(A_1+(1-A_1)q_2=0.020+0.980(0.03061)=0.0500\), up to rounding.

The example also shows why an average-spending calculation is not a substitute for history-specific conditional-error control when a procedure changes after observing interim data. The allowed future rejection probability may differ across histories. If a redesign is made, its conditional rejection probability must be checked against the relevant history-specific allowance; matching the overall average alone does not establish that condition.

Calibration Workflow and Failure Modes

State the calibration target.
Specify the null configuration, the family of rejection events being controlled, the looks, and cumulative spending values \(A_j\).

Convert spending to hazards.
For each look calculate \(q_j=(A_j-A_{j-1})/(1-A_{j-1})\).

Use the joint null distribution.
Write the probability of surviving through look \(j\), conditional on the earlier survival region, as a function of the proposed boundary.

Solve and verify.
Choose the boundary so that \(P(S_j)=1-A_j\), then check the implied cumulative rejection probability at every look and at the final look.

When the stagewise statistics are independent across looks, the survival probability can factor into stagewise terms. With correlated looks, it generally cannot: the earlier non-rejection event changes the distribution of later statistics. A one-look marginal quantile used repeatedly may therefore spend too much or too little error. Similarly, calibrating each arm separately and then reporting the intended familywise level ignores the probability of rejecting on another arm.

Common alternatives have different roles. A Bonferroni allocation assigns per-test error levels whose sum is bounded by the target; it can be simple and valid but may be conservative because it does not use the joint dependence structure. Direct joint calibration can use that structure for less conservative boundaries, but relies on a credible joint null model. A prespecified cumulative spending function determines when error is spent; conditional-error recursion is the calculation that translates its cumulative targets into stagewise probabilities and boundaries. Neither alternative removes the need to model the actual decision rule, including arm selection and dropping.

A useful diagnostic is to simulate or numerically integrate the complete null decision process and estimate \(P(S_j^c)\) at every look, not just at the final look. Check that these estimates agree with \(A_j\) within numerical error, that all specified dependence and adaptation are represented, and that increasing a rejection boundary cannot increase rejection probability. A mismatch may indicate an incorrect survival region, a mistaken conditional denominator, or a joint distribution that does not match the design.

Common Mistake and Limitations

Common mistake: Treating \(A_j-A_{j-1}\) as the conditional rejection probability at stage \(j\). The increment is unconditional; only a fraction \(1-A_{j-1}\) of trials have survived to that look. The correct conditional probability is \((A_j-A_{j-1})/(1-A_{j-1})\).

The recursion precisely allocates error for the probability model and rejection event used to calibrate it. If that model is misspecified, or if an unplanned adaptation changes the null distribution or rejection rule, the nominal spending schedule need not be achieved. Also, controlling the global-null probability of any rejection is not automatically the same as controlling familywise error under every configuration of true and false hypotheses. The relevant family of null hypotheses and the procedure’s guarantees must be addressed separately.

Check Your Understanding

Try each question, then compare your reasoning with the solutions.

If \(A_{j-1}=0.040\) and \(A_j=0.055\), what conditional rejection probability is required at stage \(j\)?
Why can two designs with the same marginal standard normal arm statistics require different common boundaries?
For two independent standard normal arm statistics, write the equation that determines a common boundary for target stagewise rejection probability \(q\).
What does a calibration based only on an average conditional hazard fail to show when decisions adapt to interim histories?

Solutions. 1. \(q_j=(0.055-0.040)/(1-0.040)=0.015625\), or about \(0.01563\). 2. The probability of at least one crossing depends on the joint distribution, including the dependence between arms; identical marginals do not imply identical joint crossing probabilities. 3. Independence gives \(1-\Phi(c)^2=q\), so \(c=\Phi^{-1}(\sqrt{1-q})\). 4. It does not establish that the future rejection probability is within its allowable conditional error for each particular surviving history; the average can conceal larger probabilities in some histories.