Eliminating Multiple Arms at One Interim

Why Simultaneous Elimination Needs a Prespecified Rule

At an interim analysis in a multi-arm trial, several experimental arms may look unpromising at the same time. Applying each arm’s elimination criterion together is straightforward only if the decision set and the consequences of eliminating all experimental arms have been defined in advance. Otherwise, a team may make a series of ad hoc choices that depend on the order in which arms are reviewed.

The central issue is not that several arms are assessed at one look; it is that separate arm-level decisions can have a joint consequence. In particular, if every experimental arm meets its dropping threshold, the trial must have a prespecified response. It may stop experimental recruitment, or a rule may protect one arm from elimination. A protected arm is not thereby shown to be promising.

Definition: Suppose there are $K$ experimental arms, indexed by $j=1,\ldots,K$, and a control arm indexed by $0$. Let $D$ denote the data available at the interim under the prespecified analysis. For each experimental arm, define $B_j$ as the event that the arm is clinically promising according to the protocol’s criterion, and let $q_j=P(B_j\mid D)$. For a prespecified threshold $\gamma$, the raw simultaneous elimination set is $$ S(D)=\{j\in\{1,\ldots,K\}:q_j<\gamma\}. $$ Every arm in $S(D)$ meets the dropping rule at this interim. The control arm is not included in this set.

The event $B_j$ must have a clinically meaningful definition. For example, if $\theta_j$ and $\theta_0$ are the response probabilities for experimental arm $j$ and control, respectively, and $\delta$ is a prespecified clinically important improvement, one possible definition is $B_j=\{\theta_j-\theta_0>\delta\}$. Then $q_j$ is the posterior probability that arm $j$ exceeds that improvement margin. The threshold $\gamma$ specifies how small this probability must be before the arm meets the elimination criterion.

This is one form of an arm-specific rule, not the only one. The earlier material on posterior futility, posterior superiority, predictive probability, and clinical margins covers other possible quantities and thresholds. The simultaneous procedure is the same in structure: calculate the specified arm-level quantity for every eligible arm using the interim analysis, apply its prespecified criterion, and form the set of arms that meet it. Do not switch definitions or thresholds between arms after seeing their results.

Applying the Rule at One Interim

A simultaneous decision uses the same interim data definition, model, and decision time for all arms. The posterior calculations may be dependent. For example, arms compared with a shared control typically have posterior probabilities linked through uncertainty about the control response. Dependence does not prevent defining $S(D)$: each $q_j$ is calculated from its marginal posterior probability under the joint analysis.

The rule $q_j<\gamma$ uses a strict inequality. An arm with $q_j=\gamma$ does not meet that rule and is retained unless another prespecified criterion applies. If a protocol instead uses $q_j\leq\gamma$, it must state that boundary convention in advance.

Fix the interim analysis.
Specify which data, model, clinical event, and posterior summary determine each $q_j$. Apply the same planned analysis to every experimental arm.

Evaluate arms separately.
Compare each arm’s prespecified posterior quantity with the common threshold, or with its arm-specific threshold if the protocol explicitly uses different thresholds.

Form the set before acting.
List all arms meeting the criterion together. Do not eliminate one arm and then silently recompute the remaining arms under a different rule at the same interim.

Apply the all-arms policy if needed.
If every experimental arm is in the raw elimination set, follow the prospectively specified trial-level rule rather than inventing a survivor or changing the threshold.

Worked Example: Applying One Threshold Across Four Arms

Consider a hypothetical platform with four experimental arms and a control arm. At an interim, 40 participants have an observed binary response in each arm. The control has 20 responses. The experimental arms have 8, 10, 12, and 7 responses, respectively. The protocol defines $B_j$ as an improvement in response probability exceeding a clinical margin and sets $\gamma=0.10$. Under the prespecified Bayesian model, suppose the posterior summaries are $q_1=0.04$, $q_2=0.08$, $q_3=0.12$, and $q_4=0.03$. These are hypothetical model outputs.

The rule is $q_j<0.10$. Substitution gives $0.04<0.10$, $0.08<0.10$, $0.12\not<0.10$, and $0.03<0.10$. Therefore, $$ S(D)=\{1,2,4\}. $$

Arms 1, 2, and 4 meet the elimination criterion; arm 3 does not. This conclusion concerns the stated posterior rule, not a claim that arm 3 is certainly effective. Its posterior probability of meeting the clinical promise definition is 0.12, which is above the dropping threshold but still modest. The control remains outside the experimental elimination set.

What the Simultaneous Decisions Imply Jointly

An arm-level threshold does not directly specify the posterior probability that at least one of several dropped arms is promising. Define $B_j$ as above. Conditional on the interim data, the probability of dropping at least one promising arm among the set $S(D)$ is $$ P\!\left(\bigcup_{j\in S(D)}B_j\mid D\right). $$ By the union bound, this probability is no greater than the sum of the marginal probabilities $q_j$ for the dropped arms. The bound does not require independence.

P\!\left(\bigcup_{j\in S(D)}B_j\mid D\right) \leq \sum_{j\in S(D)}P(B_j\mid D) =\sum_{j\in S(D)}q_j.

If $S(D)$ is nonempty and every dropped arm satisfies $q_j<\gamma$, then the sum is strictly less than $|S(D)|\gamma$. For any $S(D)$, including the empty set, the valid bound is

P\!\left(\bigcup_{j\in S(D)}B_j\mid D\right) \leq \sum_{j\in S(D)}q_j \leq |S(D)|\gamma.

When $S(D)$ is empty, the union is the empty event and its probability is zero; both sums are also zero. Thus the weak bound holds, while a strict claim of $0<0$ would not. This joint bound is a posterior statement conditional on the observed interim data. It is not, by itself, a frequentist familywise type I error rate.

Worked Example: Bounding the Posterior Risk Across Dropped Arms

In a hypothetical five-arm trial, the interim posterior probabilities for the clinical promise events are $0.03, 0.08, 0.14, 0.02,$ and $0.25$. The common dropping threshold is $\gamma=0.10$, so arms 1, 2, and 4 are dropped. The posterior probability that at least one of these dropped arms is promising is bounded by

P(B_1\cup B_2\cup B_4\mid D) \leq q_1+q_2+q_4 =0.03+0.08+0.02 =0.13.

There are three dropped arms, so the threshold-based bound is also $3(0.10)=0.30$; the actual marginal-probability sum gives the tighter bound of 0.13. The exact joint probability cannot be obtained from these three marginal probabilities alone because the promise events may be dependent. In particular, the calculation does not assert that the probability is 0.13.

A union bound can be conservative, especially when many arms are dropped or when their promise events overlap strongly. If the decision requires direct control of a joint posterior quantity, the protocol can define a joint rule, such as evaluating the posterior probability that any arm in a proposed elimination set is promising. That is a different decision rule from applying a common marginal threshold separately to each arm.

When Every Experimental Arm Meets the Threshold

If $q_j<\gamma$ for every $j=1,\ldots,K$, then the raw set is $S(D)=\{1,\ldots,K\}$. There is no experimental arm that passes the arm-level retention criterion. A protocol must state what follows. Two broad policies are possible, and neither is universally correct:

Stop experimental recruitment under the trial-level rule. The protocol may specify that meeting the criterion for every experimental arm ends recruitment to all experimental arms, with any further trial disposition handled according to its stopping plan.
Protect one experimental arm by a prespecified constraint. If the design requires at least one experimental arm to remain, the protocol may protect the arm with the largest $q_j$, using a prespecified tie-break rule. This arm is the least weak by that particular posterior summary, but it still met the raw dropping threshold.

The second option is a constraint on the action, not a revision of the evidence. If arm $j^\star$ has the largest $q_j$ but $q_{j^\star}<\gamma$, reporting it as “retained because it passed the futility rule” is incorrect. It was protected despite failing that rule. The protocol should explain why such protection is required and what action is intended if all arms meet the threshold. Any control-arm retention requirement remains separate from this experimental-arm rule.

Worked Example: Handling the All-Arms Case

Suppose a hypothetical trial has three experimental arms. At an interim, each has 30 observed binary outcomes; the control has 16 responses and the experimental arms have 9, 10, and 11 responses. The prespecified posterior analysis gives $q_1=0.03$, $q_2=0.06$, and $q_3=0.09$, with $\gamma=0.10$. Thus $$ S(D)=\{1,2,3\}, $$ because $0.03<0.10$, $0.06<0.10$, and $0.09<0.10$.

Suppose the protocol requires at least one experimental arm to remain and says to protect the arm with the largest $q_j$. Here $j^\star=3$, since $0.09$ is the largest of the three values. Arms 1 and 2 are eliminated, and arm 3 is protected from elimination by the constraint. Its posterior probability remains $0.09$; it has not crossed the $0.10$ threshold and is not evidence of clinical promise under the defined rule. If instead the protocol specified stopping experimental recruitment when all arms meet the criterion, the appropriate action would be that planned stop—not selecting arm 3 after seeing the results.

Justification, Diagnostics, and Alternatives

The set-based procedure avoids an order effect. Let $d_j(D)$ equal 1 when $q_j<\gamma$ and 0 otherwise. The raw set is the collection of arms with $d_j(D)=1$. Since each indicator is determined by the same data and rule, the set does not depend on whether the analyst checks arm 1 before arm 2 or in the reverse order. By contrast, if an analyst removes an arm, changes the model or threshold, and then reevaluates other arms at the same interim, the resulting decisions are no longer the stated simultaneous rule.

Useful checks before acting include confirming that all arms use the planned interim data and model; verifying the direction and strictness of every threshold; reviewing whether the clinical margin is the same as specified; and checking that shared-control dependence is represented in the posterior analysis. Also inspect sensitivity to reasonable prior choices and to small changes in $\gamma$. If a small change flips several decisions, the arm-level classifications are threshold-sensitive and should be reported as such, not presented as a sharp distinction in biological truth.

Simultaneous marginal thresholds are easy to communicate and permit arm-specific interpretation. Sequential elimination followed by reassessment can answer a different question, because later decisions may use a changed set of active arms or a different analysis. A joint posterior decision rule directly evaluates a collective event, while a decision rule based on expected utility can weigh the consequences of different actions. These alternatives may be preferable when the cost of one false elimination is especially high, but they require their own prespecified definitions and assumptions.

No posterior rule can compensate for a poorly specified model, an unsuitable clinical margin, or an outcome definition that does not match the decision. A low $q_j$ means that the model and observed data assign low posterior probability to the defined promise event; it does not prove that the arm is ineffective or harmful. Conversely, an arm not meeting the elimination criterion is not thereby established as beneficial. These distinctions matter particularly when all arms fall below the threshold.

Common pitfall: “Every arm has a small posterior probability of being promising, so at least one must be retained as the best arm.” This reasoning confuses ranking with evidence of promise. The arm with the largest $q_j$ can still be below $\gamma$. Retain it only if a prespecified all-arms policy requires a protected survivor; otherwise follow the planned trial-level rule.

Check Your Understanding

Answer each question using the stated rule, then compare your reasoning with the solutions.

With $\gamma=0.10$, an interim analysis gives $q_1=0.10$, $q_2=0.07$, and $q_3=0.13$. Which arms meet the rule $q_j<\gamma$?
Three arms have $q_1=0.02$, $q_2=0.05$, and $q_3=0.08$, with $\gamma=0.10$. A protocol requires one protected experimental arm, chosen by largest $q_j$. Which arm is protected, and what can be concluded about its status under the raw rule?
If $S(D)$ is empty, what is $P(\bigcup_{j\in S(D)}B_j\mid D)$, and what is the valid union-bound statement?
Three dropped arms have $q$-values 0.01, 0.04, and 0.06. Give the union-bound calculation for the posterior probability that at least one dropped arm is promising. Is the bound a frequentist familywise error rate?

Solutions. (1) Only arm 2 meets the strict rule, because $0.07<0.10$; arm 1 is exactly at the threshold and arm 3 is above it. (2) Arm 3 is protected because $0.08$ is largest. It still meets the raw dropping rule because $0.08<0.10$; protection does not make it pass. (3) The union is empty, so its probability is zero. The valid bound is $0\leq 0\leq |S(D)|\gamma=0$; a strict inequality is not valid. (4) The bound is $P(\bigcup B_j\mid D)\leq0.01+0.04+0.06=0.11$. It is a posterior probability bound conditional on the interim data, not a frequentist familywise error rate.