Why the Conditioning Event Matters
In a multi-arm multi-stage design, the arm ultimately selected is only part of the selection history. At interim analyses, some arms may be retained and others dropped; later decisions then occur among the arms still eligible. Two trials can therefore select the same final arm after following different retention paths. If an estimate is examined after selection, the distribution being described depends on whether we condition only on the winner or also on the decisions that led to that winner.
Winner-only conditioning answers a marginal question: what is the distribution of a specified estimate when arm \(j\) is the final selection, averaging over all paths that end with \(j\)? Full-path conditioning answers a more detailed question: what is that distribution when arm \(j\) is selected and the sequence of retention decisions is specified? These are different conditional distributions, even though the selected arm is the same.
Suppose \(T\) is the estimate or other statistic whose post-selection distribution is of interest. It must be the same well-defined quantity across the paths being compared. If the design uses different estimators on different paths, define \(T\) as the actual reported output of that path-dependent rule; do not silently treat distinct statistics as one common estimate. The selection events themselves are determined by the design and observed data, so conditioning on them generally changes the distribution of \(T\).
The Winner-Only Distribution Is a Mixture
For a fixed selected arm \(j\), let \(\mathcal{P}_j\) be the set of complete paths ending with \(j\). These paths are mutually exclusive and, conditional on \(W=j\), exhaustive. Define their conditional weights by \(w_p=P(P=p\mid W=j)\). Then the winner-only distribution is the weighted mixture of the path-conditioned distributions:
This follows by partitioning the event \(\{T\le t,W=j\}\) into the disjoint events \(\{T\le t,P=p\}\), summing their probabilities, and dividing by \(P(W=j)\). Equivalently, the conditional density, when densities exist, is the same weighted sum of the path-conditioned densities. The weights are conditional path probabilities, not the unconditional probabilities of the paths.
The corresponding mean and variance make the distinction especially clear. Write \(\mu_p=E(T\mid P=p)\) and \(\sigma_p^2=\operatorname{Var}(T\mid P=p)\). Then:
For the variance identity, condition first on the path. The conditional variance decomposition gives \(\operatorname{Var}(T\mid W=j)=E\{\operatorname{Var}(T\mid P)\mid W=j\}+\operatorname{Var}\{E(T\mid P)\mid W=j\}\). The first term is the weighted average of the within-path variances; the second is the weighted variation among path-specific means. Thus, winner-only variance includes both sources of variation. In particular, it can exceed every path-specific variance when path means differ.
Worked Example: Combining Two Paths for One Winner
Consider a hypothetical design in which arm A is the winner on two possible paths, \(p_1\) and \(p_2\). Suppose that, conditional on selecting A, the path probabilities are \(P(p_1\mid W=A)=0.25\) and \(P(p_2\mid W=A)=0.75\). Let \(T\) be the reported effect estimate in percentage points, with \(T\mid p_1\sim N(1,1)\) and \(T\mid p_2\sim N(3,1)\), where the second argument denotes variance.
The winner-conditioned mean is the weighted average of the path means:
Each path has variance \(1\;(\text{percentage points})^2\). The between-path contribution is \(0.25(1-2.5)^2+0.75(3-2.5)^2=0.25(2.25)+0.75(0.25)=0.75\). Therefore:
The standard deviation is \(\sqrt{1.75}=1.323\) percentage points. Neither path has this distribution: the winner-only distribution is a mixture of two normal distributions, not generally a single normal distribution. It combines both within-path variation and the difference between the path-specific means.
Worked Conditional Moments From Path Probabilities
When path probabilities are initially available without conditioning on the winner, first add the probabilities of the paths that end with the specified winner. Dividing each path probability by that total gives the mixture weights. This is an important bookkeeping step: using unconditional path probabilities directly in the mixture does not produce a conditional distribution.
Worked Example: Recovering the Winner-Conditioned Mean and Variance
Suppose a hypothetical MAMS design has two paths ending with arm B. Their unconditional probabilities are \(P(p_1)=0.18\) and \(P(p_2)=0.12\). Conditional on these paths, a common effect estimate \(T\), in percentage points, has means and variances \((1.2,0.64)\) on \(p_1\) and \((2.4,1.44)\) on \(p_2\). Assume these stated path-conditioned moments come from the design’s joint model for the estimates and selection decisions.
The probability of selecting B is \(0.18+0.12=0.30\). Hence the path weights given this winner are \(w_1=0.18/0.30=0.60\) and \(w_2=0.12/0.30=0.40\). The winner-conditioned mean is
The weighted within-path variance is \(0.60(0.64)+0.40(1.44)=0.384+0.576=0.960\;(\text{percentage points})^2\). The between-path contribution is \(0.60(1.2-1.68)^2+0.40(2.4-1.68)^2=0.60(0.2304)+0.40(0.5184)=0.3456\;(\text{percentage points})^2\). Thus:
The variance is rounded from \(1.3056\), and the corresponding standard deviation is \(\sqrt{1.3056}=1.143\) percentage points. If one instead reports the path-conditioned results, the means are 1.2 and 2.4 percentage points and the variances are 0.64 and 1.44, respectively. Each answers a more specific question than the winner-conditioned summary.
Distributional Features Can Also Differ
A mixture relationship applies to probabilities and quantiles as well as to means and variances. In general, a winner-conditioned quantile is not the weighted average of the path-specific quantiles. For example, a tail probability is obtained by averaging the path-specific tail probabilities using the conditional path weights. This can matter when the question concerns how often the selected estimate exceeds a practically important threshold.
Worked Example: A Threshold Probability Under Two Conditioning Choices
Suppose that, conditional on arm C winning, two paths have weights 0.70 and 0.30. For a hypothetical estimate \(T\) in percentage points, assume \(T\mid p_1\sim N(0.8,0.25)\) and \(T\mid p_2\sim N(2.0,0.25)\). Find the probability that \(T\) exceeds 1.5 percentage points under winner-only conditioning, and compare it with the path-conditioned probabilities.
Each path has standard deviation \(\sqrt{0.25}=0.5\) percentage points. Under \(p_1\), the standardized threshold is \((1.5-0.8)/0.5=1.4\), so \(P(T>1.5\mid p_1)=1-\Phi(1.4)\approx0.0808\). Under \(p_2\), it is \((1.5-2.0)/0.5=-1\), so \(P(T>1.5\mid p_2)=1-\Phi(-1)\approx0.8413\), where \(\Phi\) is the standard normal cumulative distribution function.
Averaging with the winner-conditioned path weights gives
Thus the winner-conditioned probability is about 30.9%, while the probabilities conditional on the individual paths are about 8.08% and 84.1%. The large difference between the path-specific probabilities is concealed by the single marginal summary. This calculation describes the assumed conditional distributions; it does not imply that the threshold is a decision rule in the design.
Assumptions, Diagnostics, and Practical Choice
The mixture identities are probability results, not special properties of normal estimates. They hold when the paths form a mutually exclusive, exhaustive partition of the outcomes conditional on \(W=j\), the event \(W=j\) has positive probability, and the relevant conditional moments exist for the mean and variance calculations. Normality is not required. If path-conditioned distributions are normal, their mixture can still be non-normal unless the component distributions coincide in a suitable way.
For a design-based calculation, the joint model for interim and later data must reflect the design’s sampling structure, including any dependence among treatment-control contrasts and any dependence across stages. The selection path must be generated from the actual retention rules. Conditioning on an incorrectly coded event, or using probabilities for paths that do not include every possible route to the winner, produces incorrect mixture weights and can omit part of the winner-conditioned distribution.
Useful checks include verifying that the path events are disjoint, that their probabilities sum to \(P(W=j)\), and that the conditional weights sum to one. Also inspect path-specific probabilities and moments: very rare paths may yield unstable estimates in simulation, while materially different path means signal that the winner-only distribution may be broad or asymmetric. A single normal approximation should not be assumed merely because every path-specific distribution appears approximately normal.
| Conditioning choice | Question answered | Main use and limitation |
|---|---|---|
| Winner only: \(W=j\) | What is the distribution of \(T\) among outcomes selecting arm \(j\), averaged over the paths leading to \(j\)? | Summarizes selection by the final arm; can hide important path-to-path differences. |
| Full path: \(P=p\) | What is the distribution of \(T\) among outcomes following this particular sequence of decisions? | Reveals path-specific behavior; may concern a rare event and is less broadly representative of all outcomes selecting \(j\). |
| Unconditional: no selection event | What is the distribution of \(T\) across all outcomes generated by the design? | Describes the design before conditioning; it does not answer a post-selection distribution question. |
Winner-only conditioning is appropriate when the target question concerns selection of arm \(j\) while intentionally averaging over how that outcome arose. Full-path conditioning is appropriate when the particular interim history matters, for example when checking whether the distribution differs across routes to the same winner. Neither conditioning choice is universally preferable: it depends on the question being asked. Conditioning on more detail does not automatically make an estimate less biased or more accurate; it changes the event that defines the distribution.
Common Mistakes
- Using the wrong weights. The mixture weights are \(P(P=p\mid W=j)\), not \(P(P=p)\). Convert unconditional path probabilities by dividing by their sum over paths ending with \(j\).
- Replacing a mixture with one component. The distribution conditional on \(W=j\) is not the distribution conditional on an arbitrarily chosen path to \(j\), unless the other paths have zero conditional weight or the relevant distributions coincide.
- Averaging variances but omitting between-path variation. The winner-conditioned variance includes both the weighted within-path variances and the variance of path means.
- Assuming a normal winner-conditioned distribution. A mixture of normal distributions is not generally normal. Check the mixture or justify an approximation for the quantity being reported.
- Comparing different statistics as if they were one. If the reported estimator changes with the path, specify the path-dependent output before comparing its conditional distributions.
Check Your Understanding
Answer each question using the distinction between the winner event and the complete path.
- Two paths ending with arm D have unconditional probabilities 0.10 and 0.15. What are their conditional weights given \(W=D\), assuming these are the only paths that select D?
- For those two paths, suppose the conditional means of \(T\) are 2 and 5. What is \(E(T\mid W=D)\)?
- Why can the variance conditional on \(W=D\) exceed both path-conditioned variances?
- If a path-conditioned estimate is approximately normal on every path, must the winner-conditioned estimate be normal? Explain.
Solutions. (1) The probability of \(W=D\) is \(0.10+0.15=0.25\), so the weights are \(0.10/0.25=0.40\) and \(0.15/0.25=0.60\). (2) The winner-conditioned mean is \(0.40(2)+0.60(5)=0.8+3.0=3.8\), in the units of \(T\). (3) Its variance adds the weighted variance of the path-specific means to the weighted within-path variances; the between-path term is positive whenever the path means differ. (4) No. The winner-conditioned distribution is a mixture of the path-conditioned distributions, and a mixture of normal distributions is not generally normal.