Computing Decision Probabilities With MCMC

From Posterior Draws to Decision Probabilities

A multi-arm Bayesian decision often asks about an event involving several unknown treatment effects: for example, whether an arm improves on control by a clinically meaningful amount, whether every selected arm meets a success criterion, or which arm has the best outcome. When the posterior distribution is represented by Markov chain Monte Carlo (MCMC) draws, these probabilities can be estimated directly by checking whether the event occurs in each draw and averaging the resulting zero-one indicators.

This calculation is simple, but its validity depends on using draws from the relevant joint posterior distribution and defining the event precisely. If the control response is uncertain, for instance, each draw's treatment response must be compared with the control response from that same draw. Comparing marginal summaries separately can discard posterior dependence and answer a different question.

Definition: Let $\theta_0$ denote the control-arm parameter and $\theta_j$ the parameter for treatment arm $j$, with larger values defined to represent better outcomes. For a specified posterior event $E$, its posterior probability is $P(E\mid\text{data})$. Given $M$ retained joint posterior draws, the MCMC indicator estimate is $$ \widehat{P}(E\mid\text{data})=\frac{1}{M}\sum_{m=1}^{M} I\{E\text{ holds for draw }m\}, $$ where $I\{\cdot\}$ equals 1 when the event holds and 0 otherwise. The estimate is the proportion of retained draws in which the event occurs.

Assumptions and Calculation

The posterior probability is conditional on the observed data and the specified likelihood and prior model. The MCMC estimate approximates that probability when the retained draws adequately represent the target posterior distribution. In practice, this requires an appropriate model, a well-defined posterior, and chains that have been run and assessed sufficiently for the retained draws to be useful. A raw collection of simulation output is not automatically a representative posterior sample.

For a superiority event, define the contrast in draw $m$ as $\theta_j^{(m)}-\theta_0^{(m)}$. If the decision requires an improvement greater than a margin $\delta$, the event is $\theta_j-\theta_0>\delta$. The margin and direction must be specified before counting. If smaller values are better, reverse the contrast or state the event using the appropriate direction. For a binary outcome, the parameters might be response probabilities; for a continuous outcome, they might be means. Their interpretation and units depend on the model.

The indicator average follows from the expectation identity $P(E\mid\text{data})=E[I\{E\}\mid\text{data}]$. Replacing the posterior expectation with an average over representative draws gives the estimator. This is why the calculation needs no special closed-form distribution for the contrast: each joint draw is evaluated against the event definition.

E[I\{E\}\mid\text{data}] =1\cdot P(E\mid\text{data})+0\cdot P(E^c\mid\text{data}) =P(E\mid\text{data}).

This identity also explains why the event must be evaluated draw by draw. In general, the probability that a contrast exceeds a margin is not determined by whether the posterior mean contrast exceeds that margin. Nor can a joint event usually be reconstructed by multiplying separate arm probabilities: treatment effects may be dependent, especially when arms share a control or are linked by a hierarchical model.

Specify the posterior event.
State the arms involved, the direction of benefit, any clinical margin, and whether the rule requires one, some, or all arms to succeed.

Evaluate each joint draw.
Use parameters from the same retained draw, including the corresponding control value when the control response is uncertain.

Average the indicators.
Divide the number of draws satisfying the event by the total number of retained draws, then interpret the result in the context of the specified model and decision rule.

Superiority Probability for One Arm

A superiority probability can refer to any stated threshold, not only a positive contrast. A threshold of zero asks whether the treatment parameter exceeds control; a positive threshold asks whether it exceeds control by more than that amount. The latter can encode a minimum clinically relevant improvement. The probability is posterior evidence about the parameter event, not the probability that the null hypothesis is true and not a frequentist significance level.

Worked Example: Improvement Beyond a Clinical Margin

Suppose a hypothetical binary-outcome analysis produces eight retained joint posterior draws of response probabilities. Higher response probability is better. The decision event for treatment A is an improvement over control greater than 0.05, or five percentage points.

Draw	Control response probability	Treatment A response probability	Difference	Event: difference > 0.05?
1	0.30	0.40	0.10	Yes
2	0.34	0.37	0.03	No
3	0.28	0.35	0.07	Yes
4	0.31	0.39	0.08	Yes
5	0.36	0.39	0.03	No
6	0.33	0.42	0.09	Yes
7	0.29	0.34	0.05	No
8	0.32	0.38	0.06	Yes

For example, draw 1 has difference $0.40-0.30=0.10$, so it satisfies the event. Draw 7 has difference $0.34-0.29=0.05$, which equals, but does not exceed, the strict threshold. Five of the eight draws satisfy the event:

\widehat{P}(\theta_A-\theta_0>0.05\mid\text{data}) =\frac{5}{8}=0.625.

In this hypothetical calculation, the estimated posterior probability that treatment A improves response probability over control by more than five percentage points is 0.625. The eight draws are deliberately few for transparent arithmetic; this proportion alone does not establish that an actual MCMC estimate is precise.

Joint-Success Probabilities Across Arms

A joint-success event combines criteria for multiple arms within the same posterior draw. For example, a protocol might define success as all three active arms exceeding control by a prespecified margin. Another rule might require at least two of the three arms to meet that criterion. These are different events, so the rule must be stated rather than inferred from the word “joint.”

When arms share a control, or when the model uses hierarchical shrinkage, their posterior parameters may be dependent. Evaluating all criteria within each joint draw retains that dependence. Multiplying separate marginal superiority probabilities would generally not give the probability that all arms succeed; such a product would require the relevant success indicators to be independent.

Worked Example: All Three Arms Must Meet the Margin

In a hypothetical trial with three active treatments, let success require each active-arm response probability to exceed the control response probability by more than 0.03. The table gives eight retained joint draws. “All succeed?” applies the stated rule to the three differences in that row.

Draw	Control	A	B	C	Differences A, B, C	All succeed?
1	0.30	0.38	0.35	0.34	0.08, 0.05, 0.04	Yes
2	0.32	0.40	0.34	0.37	0.08, 0.02, 0.05	No
3	0.29	0.34	0.36	0.35	0.05, 0.07, 0.06	Yes
4	0.31	0.35	0.36	0.33	0.04, 0.05, 0.02	No
5	0.34	0.39	0.38	0.38	0.05, 0.04, 0.04	Yes
6	0.30	0.33	0.32	0.36	0.03, 0.02, 0.06	No
7	0.33	0.37	0.36	0.37	0.04, 0.03, 0.04	No
8	0.28	0.35	0.34	0.32	0.07, 0.06, 0.04	Yes

Draw 1 satisfies all three conditions because each difference is strictly greater than 0.03. Draw 6 fails because the A difference equals 0.03 and the B difference is below 0.03. The event occurs in draws 1, 3, 5, and 8, so

\widehat{P}(\theta_A-\theta_0>0.03,\, \theta_B-\theta_0>0.03,\, \theta_C-\theta_0>0.03\mid\text{data}) =\frac{4}{8}=0.500.

Thus, the estimated posterior probability that all three arms exceed control by more than three percentage points is 0.500. If the decision instead required at least two arms to succeed, seven draws would qualify: draws 1, 2, 3, 4, 5, 7, and 8. That alternative event would have estimate $7/8=0.875$. The difference illustrates why “joint success” must be defined explicitly.

Posterior Ranking Probabilities

Ranking probabilities summarize how often each arm occupies a specified position in the posterior draws. With higher values defined as better, rank 1 is the arm with the largest parameter in that draw. The probability that arm $j$ ranks first is estimated by counting draws in which it has the largest value and dividing by $M$. The same procedure gives probabilities for second place or any other rank.

Rankings must be calculated within draws. Sorting posterior means only produces one ordering of point estimates; it does not describe how uncertain the ordering is. If exact ties are possible, the tie rule should be stated—for example, assigning tied arms equal fractional credit for the first rank. With continuous posterior draws, exact ties may be rare, but rounded output can create artificial ties.

Worked Example: Probability Each Arm Ranks First

Consider four hypothetical arms: control, A, B, and C. For each of eight retained draws, rank the response probabilities from largest to smallest. There are no ties in these draws.

Draw	Control	A	B	C	Arm ranked first
1	0.30	0.42	0.38	0.34	A
2	0.32	0.40	0.43	0.37	B
3	0.31	0.35	0.33	0.39	C
4	0.34	0.36	0.41	0.38	B
5	0.29	0.32	0.37	0.40	C
6	0.33	0.39	0.35	0.42	C
7	0.30	0.41	0.44	0.36	B
8	0.35	0.38	0.40	0.37	B

Arm B is first in draws 2, 4, 7, and 8; C is first in draws 3, 5, and 6; A is first in draw 1; and control is never first. Therefore, the estimated first-rank probabilities are

\widehat{P}(A\text{ ranks first}\mid\text{data})=\frac{1}{8}=0.125,\qquad \widehat{P}(B\text{ ranks first}\mid\text{data})=\frac{4}{8}=0.500, $$ $$ \widehat{P}(C\text{ ranks first}\mid\text{data})=\frac{3}{8}=0.375,\qquad \widehat{P}(\text{control ranks first}\mid\text{data})=\frac{0}{8}=0.

The four estimates sum to 1 because exactly one arm ranks first in each draw and there are no ties. In this small illustrative sample, B ranks first most often, but that does not imply certainty that B is best. Ranking probability also does not by itself establish that the difference between the top-ranked arm and other arms is clinically important.

Checks, Alternatives, and Limitations

Before interpreting an indicator estimate, confirm that the stored quantities are the parameters named in the decision event, that benefit is oriented correctly, and that the margin uses the intended scale. For example, a difference of 0.05 between response probabilities means five percentage points, not a five percent relative increase. Also verify that comparisons involving an uncertain control use the control value from the same draw.

A simple indicator average is often preferable to evaluating a complicated posterior integral by hand, especially for events involving several arms. If a posterior probability is available analytically or by accurate numerical integration, that can serve as an alternative calculation or a check. The choice does not change the event being estimated. In either approach, a model that poorly represents the data or the dependence between arms can yield a misleading decision probability.

MCMC draws are generally dependent, so the number of retained draws is not the same as the number of independent observations. A proportion such as 0.625 is an estimate, not an exact posterior probability; with few or highly dependent draws it may be imprecise. Appropriate chain assessment is therefore necessary before relying on the summary. Detailed assessment near decision boundaries and formal quantification of simulation error require additional checks beyond the indicator calculation itself.

Common pitfall: A reader sees that treatment A has a larger posterior mean than control and reports a high probability of clinically meaningful superiority. That reasoning is incorrect: the means do not determine the probability that the contrast exceeds a threshold. The correction is to calculate the contrast in each joint posterior draw, apply the specified threshold to that draw, and average the resulting indicators.

Check Your Understanding

Answer each question using the event-indicator approach.

In 20 retained draws, an arm exceeds control by more than a prespecified margin in 13 draws. What is the indicator estimate, and what does it describe?
Two arms each have marginal superiority probability 0.80. Is the probability that both succeed necessarily 0.64? Explain.
Across 50 draws, arm A ranks first 18 times, arm B 20 times, and control 12 times. What is each arm’s estimated first-rank probability?
Why must a treatment and uncertain control be compared using values from the same posterior draw?

Solutions. (1) The estimate is $13/20=0.65$, an MCMC estimate of the posterior probability that the arm’s contrast exceeds the specified margin, conditional on the data and model. (2) No. Multiplication to obtain $0.80\times0.80=0.64$ requires independence of the two success events, which need not hold; count the joint successes in the joint draws. (3) The estimates are $18/50=0.36$ for A, $20/50=0.40$ for B, and $12/50=0.24$ for control. They sum to 1 because each draw has exactly one first-ranked arm. (4) The paired values preserve the posterior dependence between treatment and control. Combining values from different draws can change the distribution of their contrast and therefore estimate a different probability.