Worked Binary-Outcome Decisions With Three Arms

Why Work Through the Three-Arm Calculation?

With two treatments and one control, it is straightforward to calculate a posterior probability for each treatment separately. The decisions can become less straightforward when the treatments share the same control: their benefit events are generally dependent, even when the arm-specific response rates have independent posterior distributions. A decision about both treatments succeeding, or at least one succeeding, therefore needs a joint calculation rather than an informal combination of separate probabilities.

This tutorial works from observed binary response counts to posterior decisions under a beta-binomial model. All data below are hypothetical. The calculations illustrate how the chosen prior, benefit margin, joint event, and decision threshold fit together; they do not prescribe universal values for a clinical trial.

Model and Posterior Updating

Let \(k\in\{0,A,B\}\) index control and two treatment arms. In arm \(k\), let \(n_k\) be the number of participants with observed outcomes and \(y_k\) the number who respond. The model assumes that, conditional on the arm response probability \(p_k\), the responses are independent Bernoulli outcomes with common probability \(p_k\) within that arm. Thus \(Y_k\mid p_k\sim\operatorname{Binomial}(n_k,p_k)\). The arm-specific likelihoods are independent conditional on the three response probabilities.

Assume independent beta priors, \(p_k\sim\operatorname{Beta}(a_k,b_k)\), with \(a_k,b_k>0\). The beta prior is conjugate to the binomial likelihood: after observing \(y_k\) responses and \(n_k-y_k\) nonresponses, the posterior is

p_k\mid y_k,n_k\sim \operatorname{Beta}(a_k+y_k,\ b_k+n_k-y_k).

This update treats the response probability as a fixed but uncertain quantity and describes uncertainty about it through the posterior distribution. Under independent priors and the arm-wise likelihood above, the three arm-specific posterior distributions are independent. That independence does not make the two treatment-versus-control contrasts independent: both contrasts contain the same uncertain \(p_0\).

Definition: For a specified benefit margin \(\Delta\geq0\), treatment \(j\) has a clinically relevant benefit over control when \(p_j-p_0>\Delta\). Its posterior benefit probability is \(P(p_j-p_0>\Delta\mid\text{observed data})\). A joint success event, such as both treatments exceeding control by \(\Delta\), is an event involving both contrasts and must be evaluated using their joint posterior distribution.

Posterior Probability for One Treatment

Write \(f_k(x)\) and \(F_k(x)\) for the posterior density and cumulative distribution function of \(p_k\). If the posterior distributions are independent across arms, condition on \(p_0=x\). The treatment response probability must then exceed \(x+\Delta\). For \(0\leq\Delta\leq1\), this gives

P(p_j-p_0>\Delta\mid\text{data}) =\int_0^{1-\Delta} f_0(x)\{1-F_j(x+\Delta)\}\,dx.

The upper limit matters: for \(x>1-\Delta\), even the largest possible value \(p_j=1\) cannot exceed \(x+\Delta\). For \(\Delta>1\), the event is impossible because probabilities lie in \([0,1]\), so its posterior probability is zero. The formula is not used with a negative integration limit. For \(\Delta=1\), the strict event is also impossible, and the integral has zero length.

Worked Example: Updating Three Arms and Comparing Each Treatment

Suppose a hypothetical trial observes one participant per arm. Control has 0 responses out of 1, treatment A has 1 out of 1, and treatment B has 0 out of 1. Use independent uniform priors, \(p_k\sim\operatorname{Beta}(1,1)\). Find the posteriors, posterior means, and the probabilities that each treatment’s response rate exceeds control’s when \(\Delta=0\).

For control, \(y_0=0\) and \(n_0-y_0=1\), so the posterior is \(\operatorname{Beta}(1+0,1+1)=\operatorname{Beta}(1,2)\). For A it is \(\operatorname{Beta}(1+1,1+0)=\operatorname{Beta}(2,1)\), and for B it is \(\operatorname{Beta}(1,2)\). The posterior means, calculated as \(a/(a+b)\), are \(1/3\) for control, \(2/3\) for A, and \(1/3\) for B.

The densities are \(f_0(x)=2(1-x)\) and \(f_A(x)=2x\), for \(0\leq x\leq1\). The cumulative distribution function for A is \(F_A(x)=x^2\), so its survival probability at \(x\) is \(1-x^2\). Therefore,

P(p_A>p_0\mid\text{data}) =\int_0^1 2(1-x)(1-x^2)\,dx =2\left(1-\frac12-\frac13+\frac14\right) =\frac56\approx0.8333.

Control and B have identical independent posterior distributions. By symmetry, \(P(p_B>p_0\mid\text{data})=1/2=0.5000\). These are probabilities about the unknown response rates conditional on the model, priors, and observed data; they are not frequentist significance levels.

Joint Events With a Shared Control

For two treatments to both exceed control by \(\Delta\), both treatment probabilities must exceed the same value \(p_0+\Delta\). Conditional on \(p_0=x\), the treatment posteriors are independent under the stated model, so their conditional survival probabilities multiply. For \(0\leq\Delta\leq1\),

P(p_A-p_0>\Delta,\ p_B-p_0>\Delta\mid\text{data}) =\int_0^{1-\Delta} f_0(x) \{1-F_A(x+\Delta)\}\{1-F_B(x+\Delta)\}\,dx.

The unconditional benefit events are generally dependent because a higher value of the shared \(p_0\) makes both treatment-versus-control comparisons harder to satisfy. To obtain the probability that at least one treatment succeeds, use inclusion-exclusion: add the two individual probabilities and subtract the probability that both succeed. This subtraction prevents the overlap from being counted twice.

P(E_A\cup E_B\mid\text{data}) =P(E_A\mid\text{data})+P(E_B\mid\text{data}) -P(E_A\cap E_B\mid\text{data}),

where \(E_j=\{p_j-p_0>\Delta\}\). This identity holds for any two events; it does not assume that the events are independent.

Worked Example: Both Treatments or at Least One Treatment Succeeds

Use the posteriors in the preceding hypothetical example, and keep \(\Delta=0\). Calculate the posterior probability that both treatments exceed control, then the probability that at least one does.

For A, \(1-F_A(x)=1-x^2\). For B, whose posterior is \(\operatorname{Beta}(1,2)\), \(F_B(x)=2x-x^2\), so \(1-F_B(x)=(1-x)^2\). The shared-control density is \(f_0(x)=2(1-x)\). Thus the joint probability is

\begin{aligned} P(E_A\cap E_B\mid\text{data}) &=\int_0^1 2(1-x)(1-x^2)(1-x)^2\,dx\\ &=2\int_0^1 (1-x)^3(1-x^2)\,dx\\ &=2\left\{\frac14-\frac1{60}\right\} =\frac7{15}\approx0.4667. \end{aligned}

Here \(\int_0^1(1-x)^3dx=1/4\), and \(\int_0^1x^2(1-x)^3dx=1/60\). Using the individual probabilities from the previous example,

P(E_A\cup E_B\mid\text{data}) =\frac56+\frac12-\frac7{15} =\frac{13}{15}\approx0.8667.

The joint probability \(0.4667\) is not the product \((5/6)(1/2)\approx0.4167\). Multiplying the marginal probabilities would incorrectly treat the two benefit events as independent and ignore their shared control rate.

From a Benefit Margin to a Decision

A decision rule needs both a definition of success and a threshold for acting on its posterior probability. For example, a protocol might define an individual treatment as promising when \(P(p_j-p_0>\Delta\mid\text{data})\geq c\), where \(c\) is a pre-specified probability cutoff. A joint rule can instead require a specified probability for both treatments succeeding, or for at least one succeeding. These are different scientific objectives, and their cutoffs need not be the same.

The margin \(\Delta\) is on the response-probability scale. If responses are coded as 0 or 1, a margin of \(0.20\) means an absolute increase of 20 percentage points in response probability, not a 20% relative increase. A posterior cutoff such as \(c=0.65\) is a decision convention, not a consequence of the beta-binomial model.

Worked Example: A Threshold-Based Decision With a 20-Point Margin

Continue with control and treatment A having posteriors \(\operatorname{Beta}(1,2)\) and \(\operatorname{Beta}(2,1)\), respectively. Suppose the hypothetical protocol defines a promising treatment as one with at least 0.65 posterior probability of improving response by more than \(\Delta=0.20\). Determine whether A and B meet that rule.

For A, the posterior survival probability at \(z\) is \(1-z^2\). The control density is \(2(1-x)\), and the integrand is zero when \(x>0.80\), since a response probability cannot exceed 1. Hence

\begin{aligned} P(p_A-p_0>0.20\mid\text{data}) &=2\int_0^{0.80}(1-x)\{1-(x+0.20)^2\}\,dx\\ &=2\int_{0.20}^{1}(1.20-y)(1-y^2)\,dy\\ &=2(0.3328)=0.6656. \end{aligned}

For B, the posterior survival probability at \(z\) is \((1-z)^2\). Thus

\begin{aligned} P(p_B-p_0>0.20\mid\text{data}) &=2\int_0^{0.80}(1-x)(0.80-x)^2\,dx\\ &=2\int_{0.20}^{1}u(u-0.20)^2\,du\\ &=2(0.1365333)\approx0.2731. \end{aligned}

Since \(0.6656\geq0.65\), A meets the stated criterion. Since \(0.2731<0.65\), B does not. These decisions apply only to the specified margin and cutoff; a different margin or threshold could produce a different conclusion.

Justification, Diagnostics, and Limitations

The integral formulas follow by conditioning on the control response probability. For example, let \(E_j=\{p_j>p_0+\Delta\}\). Given \(p_0=x\), the probability of \(E_j\) is \(1-F_j(x+\Delta)\), provided \(x+\Delta\leq1\), and is zero otherwise. Averaging that conditional probability over the posterior density of \(p_0\) gives the single-treatment formula. For the joint event, conditional independence of \(p_A\) and \(p_B\) gives the product of their conditional survival probabilities. Integrating over \(p_0\) then accounts for the shared uncertainty that couples the contrasts.

With beta posteriors, these one-dimensional integrals can be evaluated analytically in special cases or by reliable numerical integration. The beta density and cumulative distribution function are continuous on the interior of their support, so standard quadrature is a natural option; an independent Monte Carlo calculation can also check a result. If using numerical methods, verify integration limits, numerical precision, and that a reported probability lies between 0 and 1. Near a decision cutoff, numerical error can change a classification, so the calculation should be checked at greater precision before applying the rule.

Several checks are especially useful:

Check support and limits. Every response probability lies between 0 and 1. For \(\Delta>1\), the benefit event is impossible; for \(0\leq\Delta\leq1\), values \(p_0>1-\Delta\) cannot contribute.
Check event ordering. Increasing \(\Delta\) makes a benefit event more demanding, so its posterior probability cannot increase. The probability for both treatments to succeed cannot exceed either individual success probability.
Check overlap. The probability that at least one succeeds must be at least as large as either individual probability and no larger than their sum or 1. Do not multiply the two marginal probabilities unless independence of the benefit events has actually been established.
Check the model and prior. Strong prior influence with few observations is possible, not a computational error. Consider whether independent priors and a common response probability within each arm are defensible for the intended analysis.

A plug-in comparison of observed response proportions is simpler, but it does not quantify posterior uncertainty or answer a probability question about the unknown rates. Separate treatment-versus-control calculations answer individual questions but not, by themselves, a joint question. Simulation or MCMC can handle more elaborate posterior models, while this conjugate beta-binomial setup permits direct integration and a transparent calculation. The method’s simplicity is also a limitation: it does not automatically account for covariates, overdispersion, missing outcomes, or prior dependence across arms. Those features require a model that represents them.

Common pitfall: Treating the two treatment-versus-control success events as independent because the arm-specific posteriors are independent. The contrasts share \(p_0\), so the events are generally dependent. Calculate a joint probability by conditioning on \(p_0\) and integrating, or use a valid joint posterior simulation; do not multiply marginal probabilities without justification.

Check Your Understanding

Use the model and definitions in this tutorial. Each answer is followed by a brief solution.

With a \(\operatorname{Beta}(2,3)\) prior and 4 responses among 10 participants, what is the posterior distribution for that arm?
Why is the joint probability that both treatments exceed control not generally the product of their separate posterior benefit probabilities?
For a nonnegative benefit margin \(\Delta=1.2\), what is \(P(p_j-p_0>\Delta\mid\text{data})\), and why?
If a treatment’s posterior probability of exceeding control by a specified margin is 0.72 and the pre-specified cutoff is 0.70, what decision follows under the stated threshold rule?

Solutions. (1) The posterior is \(\operatorname{Beta}(2+4,3+10-4)=\operatorname{Beta}(6,9)\). (2) Both contrasts contain the same uncertain control response probability; conditioning on it creates a joint event probability that must be averaged over its posterior, rather than obtained by multiplying marginal probabilities. (3) The probability is zero because \(p_j-p_0\leq1\) for response probabilities in \([0,1]\), so a difference greater than 1.2 is impossible. (4) The treatment meets the rule because \(0.72\geq0.70\); this conclusion is conditional on the specified model, data, margin, and cutoff.