From a Sequence of Decisions to One Event
In a multi-arm, multi-stage design, the final set of arms depends on a sequence of decisions. An arm may pass an interim threshold, lose a comparison with another arm, or be dropped for futility; later decisions are made only for arms that remain. To study treatment-effect estimation after selection, we first need to say precisely which sequence of decisions occurred. The natural representation is a joint event defined by all the relevant interim statistics.
An informal description such as “A and C continued, then A was selected” is not yet a mathematical event. It must specify the rule at every stage, the observed statistics that determine each decision, and what happens at equality. Those details matter even if ties are unlikely under a continuous model: the event must match the design rule, not just the outcomes that are most common.
For example, write \(\mathcal{A}_j\) for the set of arms active after stage \(j\), with \(\mathcal{A}_0\) the initial set. At stage \(j\), a deterministic rule \(\delta_j\) uses the available statistics and the earlier decisions to produce \(\mathcal{A}_j\), or to stop the trial. If \(\pi=(\mathcal{A}_1,\ldots,\mathcal{A}_J)\) is a path that reaches stage \(J\), then its event has the general form
Each condition in this intersection represents one decision along the path. A stopped path has no conditions on statistics from stages it never reaches. One can formally include statistics that would have been collected later in the full vector \(Z\), but the path event does not constrain those unused values.
Why the Joint-Event Representation Works
A deterministic design rule maps each possible vector of interim statistics to one path. The event for a path is simply the set of statistic vectors that map to it. In mathematical terms, if \(G(Z)\) is the path produced by the rule, then \(E_\pi=\{G(Z)=\pi\}\). This is the preimage of the path under the decision rule. The definition gives a direct way to check whether an event is complete: a statistic vector belongs to it exactly when applying the stated rules stage by stage produces the specified path.
If \(Z\) has joint density \(f_Z\), the probability of that path is
where \(\mathbf{1}\{z\in E_\pi\}\) equals 1 when \(z\) is in the path event and 0 otherwise. The event is determined by the design rule; its probability additionally depends on the joint distribution of the interim statistics. In MAMS settings, statistics may be correlated, for example because treatment contrasts share a control group. Therefore, multiplying marginal probabilities is justified only when the relevant independence assumptions hold.
A useful derivation follows directly from the rule. At stage 1, the observed statistics must fall in the region that produces \(\mathcal{A}_1\). Among those outcomes, the stage 2 statistics must fall in the region that produces \(\mathcal{A}_2\). Continuing in this way, the outcome produces the entire specified path if and only if it satisfies every stage-specific condition. “If and only if” gives the intersection above: omitting any condition admits outcomes that take a different path, while adding a condition not required by the rule wrongly excludes possible outcomes on the path.
Worked Example: Threshold Screening Followed by a Tied Comparison
Suppose three arms, A, B, and C, are screened at stage 1. Each arm is retained if its statistic \(Z_{1i}\) is at least 1. If exactly two arms remain, stage 2 compares their statistics \(Q_i\) and selects the larger. If they tie, the rule selects A over C, A over B, and B over C. Encode the path in which A and C pass stage 1, B is dropped, and A is selected at stage 2. A hypothetical observed vector is \(Z_{1A}=1.24\), \(Z_{1B}=0.62\), \(Z_{1C}=1.08\), \(Q_A=1.15\), and \(Q_C=0.91\).
The first-stage conditions are \(Z_{1A}\geq 1\), \(Z_{1B}<1\), and \(Z_{1C}\geq 1\). Because the tie rule selects A over C, the stage 2 condition for selecting A is \(Q_A\geq Q_C\), not \(Q_A>Q_C\). The complete path event is therefore
For the given hypothetical values, \(1.24\geq1\), \(0.62<1\), and \(1.08\geq1\), so the stage 1 conditions hold. Also, \(1.15\geq0.91\), so A is selected at stage 2. If \(Q_A=Q_C\), the outcome would still belong to this event because the specified tie rule selects A. A strict inequality would fail to encode that allowed outcome.
Rank-Based Retention Requires the Tie Rule
A rank rule is another common way to retain a fixed number of arms. The event must encode the ordering induced by both the statistics and the tie-breaking convention. Consider three arms with statistics \(Z_A,Z_B,Z_C\). Suppose the two largest are retained, and exact ties are ordered A before B before C. The path that retains A and C and drops B requires both A and C to outrank B under that rule.
Worked Example: Retaining Two Arms by Rank
Under the rule just described, A outranks B if \(Z_A>Z_B\), or if \(Z_A=Z_B\) and A wins the tie. Since A has priority over B, this condition is \(Z_A\geq Z_B\). C outranks B only if \(Z_C>Z_B\): when those statistics are equal, B has priority over C. Thus the complete event for retaining A and C is
For hypothetical values \(Z_A=0.90\), \(Z_B=0.70\), and \(Z_C=0.74\), both conditions hold, so A and C are retained. If instead all three statistics equal 0.70, the first condition holds but the second fails. The rule then retains A and B, as required by the stated priority order. The strict inequality for C is essential here; replacing it with \(Z_C\geq Z_B\) would include a path the rule does not produce.
This example also illustrates why one should not encode ranks with strict inequalities by default. When a tie can occur and the rule resolves it deterministically, the event must include exactly the equality cases that the rule assigns to the path. Under some continuous distributions a particular equality has probability zero, but that does not change the definition of the design event.
Encoding Decisions Across Three Stages
A path can combine several types of decisions. For each stage, write only the conditions needed to produce the specified next active set or stopping decision. Do not add conditions on an arm’s statistic after that arm has been dropped: the rule no longer uses that statistic along this path.
Worked Example: A Three-Stage Retention Path and Its Probability
Consider two arms, A and B, and the following hypothetical rule. At stage 1, retain an arm if its statistic \(Z_{1i}\) is at least 0.5. At stage 2, among the active arms, retain an arm if its statistic \(W_{2i}\) is at least 0. At stage 3, select the sole remaining arm if its statistic \(V_{3i}\) is at least 1.2; otherwise stop without selecting an arm. Encode the path in which both arms pass stage 1, A is dropped at stage 2, and B is selected at stage 3. The hypothetical observations are \(Z_{1A}=0.80\), \(Z_{1B}=0.60\), \(W_{2A}=-0.20\), \(W_{2B}=0.40\), and \(V_{3B}=1.40\).
Both first-stage values are at least 0.5. At stage 2, \(W_{2A}<0\) and \(W_{2B}\geq0\), so only B remains. At stage 3, \(V_{3B}\geq1.2\), so B is selected. The path event is
For a probability calculation, make the additional simplifying assumption that these five statistics are mutually independent standard normal random variables. This is an explicit working model for the calculation, not a requirement for defining the event. With \(\Phi\) denoting the standard normal cumulative distribution function, independence gives
Thus, under this particular independent-normal model, the probability of this exact path is about 0.274%. If the statistics were correlated, the event would be unchanged but this product calculation would generally not be valid; its probability would need to use their joint distribution. No \(W_{2A}\) or \(V_{3A}\) condition is added beyond the stated stage 2 condition: A is dropped before any later decision involving A.
Checks, Alternatives, and Limitations
A path event is more detailed than a statement about one selected arm. Several different paths may end with the same arm; the event that A is selected is the union of the path events that end in A. Conversely, a complete path records the intermediate decisions as well as the endpoint. These are different events and should not be substituted for each other when specifying which selection rule was followed.
A verbal decision tree is helpful for designing and reviewing a procedure, but the intersection of statistic constraints is more precise for probability calculations. A list of separate marginal pass probabilities is not a substitute for the joint event, particularly when arms share a control or when decisions at later stages depend on earlier retention. The decision-tree description and the event representation are complementary: the tree clarifies the rule, and the event states exactly which region of the statistic space produces the path.
Before using a path event, check the following:
- Completeness: Does each stage’s condition reproduce the stated active set or stopping decision? Have all necessary arm failures as well as passes been included?
- Boundary consistency: Does every weak or strict inequality agree with the threshold convention and tie rule?
- Reachability: Does the event describe a logically possible sequence? For instance, it cannot impose a stage 3 decision on a path that stops at stage 1.
- Distributional assumptions: Is a probability calculation using a joint model that reflects dependence among the interim statistics? If not, the event may still be right while the probability is wrong.
The representation does not, by itself, provide a bias correction, an estimator, or a probability model for the statistics. It identifies the selection event. Its probability and its implications for an estimator require additional assumptions and calculations. In particular, replacing a multi-stage path with only its final winner loses information about which arms passed or failed earlier.
Common Mistake
A frequent error is to write only the conditions for arms that pass, or to use strict inequalities throughout. For example, someone might encode “A selected over C” as \(Q_A>Q_C\) even when the declared tie rule gives A priority. That reasoning excludes the outcome \(Q_A=Q_C\), although the procedure selects A there. The correction is to state the tie rule first and then translate it literally: with A preferred to C, the condition is \(Q_A\geq Q_C\). Similarly, a complete stage 1 path must say which other arms failed their retention criteria, not just name the arms that passed.
Check Your Understanding
For each question, first write the event using the stated decision rule. Solutions are included below.
- Two arms, A and B, are screened using \(Z_A\geq1\) and \(Z_B\geq1\). Both pass. At the next stage the larger of \(Q_A,Q_B\) is selected, with ties going to B. Write the event for the path in which both pass and B is selected.
- Three arms are ranked by \(Z\), retaining exactly one, with ties resolved in the order C, B, A. Write the event that A is retained.
- At stage 1, arm A continues if \(Z_A\geq0\). If it continues, it is selected at stage 2 if \(Q_A\geq1.5\). Write the event that A continues but is not selected.
- Why is it generally invalid to calculate a multi-arm path probability by multiplying the marginal probabilities of each arm’s retention event?
Solutions. (1) The event is \(\{Z_A\geq1,\ Z_B\geq1\}\cap\{Q_B\geq Q_A\}\), since B wins equality. (2) A is last in the tie priority, so it must strictly exceed both other statistics: \(\{Z_A>Z_B,\ Z_A>Z_C\}\). (3) The event is \(\{Z_A\geq0,\ Q_A<1.5\}\); the second condition describes failure of the stage 2 selection threshold. (4) Arm statistics can be dependent, for example through a shared control group. The probability of an intersection is not generally the product of marginal probabilities unless the relevant events are independent.