Minimum Numbers of Arms to Continue

Why Require a Minimum Number of Arms to Continue?

At an interim analysis, an experimental arm may continue only if its interim score meets a prespecified cutoff. This threshold rule is transparent, but it can leave very few arms for the remaining stages. If the design requires a minimum number of experimental options to remain under evaluation, the continuation policy must say what happens when too few arms meet the cutoff.

A minimum-continuation constraint answers that operational question. It does not change the evidence from the interim data, and it does not make a below-cutoff arm satisfy the cutoff. Instead, when the usual rule selects fewer than the required minimum, a prespecified ranking identifies additional arms to carry forward. The distinction between meeting the evidence criterion and being retained to satisfy the quota must remain clear.

This definition assumes that a larger score consistently represents a more favorable interim result. The score might be a standardized selection score or another measure established by the design; the minimum rule does not prescribe how to construct it. The cutoff, score direction, minimum \(m\), and ranking convention must be specified before using the rule. The scores must be available for all arms being ranked and comparable for the intended ranking.

The rule also presumes that \(1 \leq m \leq q\). If \(m>q\), the minimum is impossible to meet because fewer than \(m\) experimental arms are available. That is a design-feasibility problem, not something the interim data can resolve. If two arms have identical scores at the boundary of selection, the design needs a prespecified tie-handling convention; without one, the selected set may not be uniquely determined.

Writing the Rule as a Selection Set

Let \(E=\{i:S_i\geq c\}\) be the set of qualifying arms, and let \(r=|E|\) be the number of qualifying arms. The retained set \(R\) can be described in two cases. When \(r\geq m\), the threshold rule alone selects all qualifying arms. When \(r<m\), the retained set contains \(E\) and the \(m-r\) highest-ranked arms among those not in \(E\).

Here \(B_{m-|E|}\) denotes the set of the \(m-|E|\) highest-ranked nonqualifying arms. In either case, \(R\) contains at least \(m\) arms. It may contain more than \(m\): if several arms meet the cutoff, the rule retains all of them rather than discarding qualifying arms to impose an upper limit. Thus, this is a minimum, not an exact continuation count.

Why the Fill Rule Has This Form

When fewer than \(m\) arms qualify, any retained set that includes all qualifiers needs at least \(m-r\) additional arms. Choosing exactly that many additional arms makes the final set contain exactly \(m\) arms in this case. To favor the strongest available interim results among the nonqualifiers, the rule chooses the \(m-r\) largest scores below the cutoff.

This gives a simple justification for the ranking step. Sort all \(q\) scores from largest to smallest, using the prespecified tie convention if needed. When \(r<m\), all \(r\) scores at or above the cutoff are among the leading scores, and the next \(m-r\) largest scores complete the first \(m\) positions. Any alternative set of \(m\) arms that keeps all qualifiers but substitutes a lower-scoring nonqualifier for a higher-scoring one has not followed the ranking principle. The fill operation therefore honors both parts of the policy: keep every qualifying arm, then select the strongest available arms needed to meet the minimum.

This justification concerns the deterministic selection rule, not the probability that an arm is truly effective. The ranking depends on observed scores, which can be noisy. Assessing how often the rule retains a promising arm or advances a less promising one requires evaluating the rule under plausible joint distributions of the interim scores, including their dependence. A shared control can induce dependence between arm-specific scores; that dependence does not prevent the ranking rule from being applied, but it matters when evaluating its operating characteristics.

Assumptions, Diagnostics, and Limitations

The rule is mechanical, but its meaning depends on several design choices. Before an interim decision, check that:

• The minimum \(m\) is feasible for the number of experimental arms available at that look.
• The score is defined for every arm, points in the intended direction, and can support the planned ranking.
• The cutoff and ranking rule are prespecified, including how exact ties are handled.
• The decision procedure states whether all cutoff qualifiers continue when their number exceeds \(m\). The rule defined here retains all qualifiers.
• Records distinguish qualification under the cutoff from retention solely to satisfy the minimum.

Potential warning signs include an undocumented change in score direction, a missing score for an arm, or a selection record that retains fewer than \(m\) arms even though \(q\geq m\). Another diagnostic is to reconstruct the decision from the recorded scores, cutoff, and tie convention. If the resulting set differs from the recorded set, either the rule was not applied as written or some design specification is missing.

A minimum constraint necessarily weakens the threshold-only rule’s ability to exclude all low-scoring arms: if too few qualify, at least some nonqualifiers continue. That is the intended trade-off, not a statistical contradiction. It can preserve options for later evaluation, but it can also use later-stage resources on arms with weaker interim scores. The minimum should therefore be chosen as part of the design, with its resource and selection consequences assessed in advance.

The main alternatives answer different questions. A threshold-only rule retains exactly the qualifying arms and may retain fewer than \(m\). A fixed top-\(k\) rule retains a specified number based solely on rank, whether or not those arms meet a cutoff. The minimum rule combines these ideas: it retains all qualifiers and uses ranking only when the qualifying set is too small. It is not equivalent to lowering the cutoff until \(m\) arms qualify, because lowering the cutoff changes the qualification criterion itself.

Common Mistake

Incorrect reasoning: “The minimum is three, so the three retained arms all passed the interim cutoff.” This confuses the operational requirement with the evidence criterion. If only one arm qualifies, two additional arms may be retained despite falling below the cutoff.

Correction: Report the two statuses separately. State which arms met \(S_i\geq c\), which additional arms were retained by rank to satisfy \(m\), and how many arms continue. The minimum rule determines continuation; it does not change observed scores or establish that a nonqualifying arm met the cutoff.