From Interim Results to a Set of Continuing Arms
At an interim look, a MAMS simulation has to do more than calculate a statistic for each treatment: it must translate the prespecified rule into a reproducible decision about which treatment arms remain active. The rule is applied to each simulated trial replicate, using that replicate’s interim data snapshot and the thresholds specified for that look. A small coding difference at this step can change the simulated trial path, so the decision and the updated set of arms should be explicit and auditable.
This tutorial focuses on implementing that update. It does not determine how the interim statistic or threshold should be chosen. Treat both as inputs supplied by the design: the statistic is computed from the interim snapshot according to the protocol, and the continuation threshold is the prespecified value for the relevant arm and look.
The inequality is part of the rule, not a programming detail. If the protocol instead specifies a strict comparison, such as \(Z_{jk}>c_{jk}\), implement that comparison exactly. Likewise, a statistic for which smaller values are favorable must be oriented or compared according to its actual prespecified rule; do not assume that every interim statistic uses a larger-is-better convention.
The active set is carried forward from one look to the next. Therefore, the rule is applied only to arms active immediately before the current look. An arm not in \(A_k\) cannot be reintroduced merely because a later value would have passed the threshold. A control arm may be handled separately, depending on the protocol; this tutorial’s update concerns treatment arms subject to the dropping rule.
Representing and Updating the Rule
A reliable implementation keeps the decision inputs aligned by arm and look. For each active arm at look \(k\), the simulation needs the interim statistic \(Z_{jk}\), the threshold \(c_{jk}\), and the prescribed boundary convention. Thresholds can be common across arms at a look or vary by arm and look. In either case, the lookup must use the correct labels, not merely assume that two arrays happen to be in the same order.
Use \(A_k\) from the previous update. At the first interim look, initialize it from the protocol’s set of treatment arms.
Retrieve that arm’s statistic from the current interim snapshot and its threshold for look \(k\). Confirm that the statistic and threshold refer to the same arm and look.
For a larger-is-better rule with inclusive boundary, retain \(j\) if \(Z_{jk}\geq c_{jk}\). Record the decision for every arm in \(A_k\).
Set \(A_{k+1}\) equal to the arms retained at this look. Preserve the decision record and the updated set for later simulation steps.
In code, this is a logical filter: for each arm in the current set, evaluate one Boolean condition and keep the arm if it is true. The essential state transition can also be written using an indicator \(I_{jk}\), which equals 1 when arm \(j\) continues and 0 otherwise:
This expression establishes a useful invariant: \(A_{k+1}\subseteq A_k\). To see why, membership in \(A_{k+1}\) requires membership in \(A_k\) as well as passing the rule. Thus, an arm can be removed but cannot reappear through this update. The invariant is an immediate check on the implementation: if an arm absent from \(A_k\) appears in \(A_{k+1}\), the update is not using the current active set correctly.
Worked Examples
Worked Example: Applying a Common Threshold
Consider one hypothetical simulated trial at an interim look. Three treatment arms, A, B, and C, are active. The protocol uses a larger-is-better statistic and specifies continuation when \(Z_{jk}\geq 1.50\). The snapshot contains \(Z_{Ak}=1.72\), \(Z_{Bk}=1.50\), and \(Z_{Ck}=0.94\). Find the decisions and the next active set.
For arm A, \(1.72\geq1.50\), so A continues. For arm B, \(1.50\geq1.50\), so B also continues because equality meets the inclusive threshold. For arm C, \(0.94<1.50\), so C is dropped.
The decisions are therefore A: continue, B: continue, and C: drop, giving \(A_{k+1}=\{\mathrm{A},\mathrm{B}\}\). The calculation uses the statistic and threshold as supplied; it does not treat passing this continuation rule as a declaration of treatment efficacy.
Worked Example: Arm-Specific Thresholds and Persistent Dropping
In a second hypothetical simulation, arms D, E, and F are active at look 1. Their continuation thresholds are \(c_{\mathrm{D},1}=1.20\), \(c_{\mathrm{E},1}=1.40\), and \(c_{\mathrm{F},1}=1.10\). The corresponding statistics are 1.35, 1.32, and 1.10. The rule is \(Z_{jk}\geq c_{jk}\). At look 2, the snapshot has statistics 2.00 for D and 1.80 for F, and both have a threshold of 1.00. What arms remain after each look?
At look 1, D passes because \(1.35\geq1.20\). E fails because \(1.32<1.40\). F passes because \(1.10\geq1.10\). Thus, \(A_2=\{\mathrm{D},\mathrm{F}\}\); E is no longer active.
At look 2, apply the rule only to D and F. D passes because \(2.00\geq1.00\), and F passes because \(1.80\geq1.00\). Hence \(A_3=\{\mathrm{D},\mathrm{F}\}\). E is not reconsidered at look 2, even though its hypothetical statistic there is not provided: it was dropped at look 1 and cannot re-enter under this state update.
Worked Example: Empty Active Set
Suppose a hypothetical trial has active arms G and H before a look. Both have threshold 0.80, and their statistics are \(Z_{\mathrm{G}k}=0.61\) and \(Z_{\mathrm{H}k}=0.79\). Under the inclusive larger-is-better rule, \(0.61<0.80\) and \(0.79<0.80\), so neither arm passes. The next treatment active set is \(A_{k+1}=\varnothing\).
The update should record that both arms were dropped and that no treatment arms remain. What the trial simulation does next—such as stopping, following a protocol-defined no-arm pathway, or continuing observations for another purpose—is a separate design rule. The arm-dropping code should not invent a terminal outcome or silently restore an arm.
Checks, Failure Modes, and Alternatives
Before running many simulated replicates, test the rule on small, deliberately chosen inputs. Include a statistic above threshold, one below threshold, and one exactly at threshold. Also test that a previously dropped arm stays absent and that the implementation can represent an empty active set. These are direct checks of the comparison and state update, not evidence that the design’s statistical assumptions are valid.
- Check the orientation. Verify that a larger value really favors continuation for the statistic being used. If smaller values are favorable, encode the appropriate inequality rather than reusing a larger-is-better comparison.
- Check the boundary. Test equality explicitly. An inclusive rule uses \(\geq\); a strict rule uses \(>\). Do not let an undocumented default decide.
- Check arm and look labels. A threshold for one arm or look must not be matched to another because of a sorting or indexing mismatch. A useful validation is to fail clearly if any active arm lacks a statistic or threshold.
- Handle missing values deliberately. A missing statistic is not automatically evidence to drop or continue. Follow a prespecified missing-data rule; if none is implemented, flag the replicate as an error rather than silently treating a missing comparison as false or true.
- Preserve state. Filter the current active set, not the full original list of arms at every look. Otherwise, a dropped arm can be accidentally reconsidered.
The main alternative to a direct arm-by-arm Boolean filter is to encode decisions through a more general decision function—for example, a function that accepts arm-specific criteria or additional prespecified conditions. This can be appropriate when the protocol truly has such conditions, but it makes the logic harder to inspect. A simple filter is easier to test when the rule is just a statistic-threshold comparison. Either approach must implement the same protocol rule and return a clearly defined active set.
This update does not establish type I error control, power, or the validity of the thresholds. Those depend on the design and its operating characteristics. It also does not decide whether an arm has met a terminal success criterion. Continuation means only that the arm remains eligible for the next stage under the specified interim rule. Keeping these meanings separate avoids coding a dropping rule as though it were a final treatment decision.
Check Your Understanding
Answer each question using the inclusive rule \(Z_{jk}\geq c_{jk}\), unless the question says otherwise.
- Two active arms have statistics 1.10 and 1.30 and a common threshold of 1.20. Which arms continue?
- Why can an arm that was dropped at look 1 not be added to the active set at look 2 by the update defined here?
- A protocol specifies continuation only when \(Z_{jk}>c_{jk}\). What happens when the statistic equals the threshold?
- At a look, all currently active arms have statistics below their thresholds. What is the updated treatment active set, and what should the arm-dropping code avoid deciding on its own?
Solutions. (1) The arm with statistic 1.10 fails because \(1.10<1.20\); the arm with statistic 1.30 passes because \(1.30\geq1.20\). Only the second arm continues. (2) The next set is defined as a subset of the current set, \(A_{k+1}=\{j\in A_k:Z_{jk}\geq c_{jk}\}\). An arm absent from \(A_k\) is not eligible for membership in \(A_{k+1}\). (3) It does not continue, because equality does not satisfy the strict inequality \(Z_{jk}>c_{jk}\). (4) The updated treatment set is empty. The dropping code should not invent a terminal trial outcome or restore an arm; the protocol determines any subsequent pathway.