Allocation Odds From Two Posterior Best Probabilities

Why Express Allocation as Odds?

In two-arm Thompson sampling, the probability of assigning the next participant to an arm is tied to the posterior probability that the arm is best. Expressing the two assignment probabilities as odds gives a compact comparison: an allocation odds of 3:1 means that, conditional on the current trial history and the stated randomization rule, arm 1 is assigned three times as often as arm 2 in repeated assignments from that same state. It does not mean that arm 1 has three times the response rate.

The odds are useful for comparing allocation chances and for checking calculations. Their meaning depends on the posterior state and the assignment rule: the quantities below concern the next participant, conditional on the information currently available, not a fixed proportion guaranteed over a realized trial.

Definition and Assumptions

Definition: Let \(\theta_1\) and \(\theta_2\) denote the unknown response parameters for arms 1 and 2, with a joint posterior distribution conditional on the observed trial history \(H\). Define \(q_1=P(\theta_1>\theta_2\mid H)\) and \(q_2=P(\theta_2>\theta_1\mid H)\). Under two-arm Thompson sampling, with one posterior draw per arm and assignment to the arm with the larger draw, the probabilities of assigning the next participant to arms 1 and 2 are \(q_1\) and \(q_2\), respectively, provided the posterior tie probability is zero. The allocation odds for arm 1 versus arm 2 are therefore \(q_1/q_2\).

This definition conditions on the trial history \(H\), including the data and the model and prior used to obtain the posterior. The posterior must be a meaningful representation of uncertainty for the response parameters, and the allocation rule must actually assign the participant to the arm with the larger posterior draw. The posterior distributions do not need to be independent for the identity: the probabilities are calculated from their joint posterior. In common continuous-response models, the probability of an exact tie is zero.

When there is no posterior tie probability, \(q_1+q_2=1\). Thus, for two arms, the posterior odds that arm 1 is superior to arm 2 and the odds of assignment to arm 1 rather than arm 2 are the same:

\frac{P(\text{assign arm 1}\mid H)}{P(\text{assign arm 2}\mid H)} = \frac{q_1}{q_2} = \frac{P(\theta_1>\theta_2\mid H)}{P(\theta_2>\theta_1\mid H)}.

An odds value can also be converted back to an assignment probability. If \(O=q_1/q_2\), and the two assignment probabilities sum to one, then \(q_1=O/(1+O)\) and \(q_2=1/(1+O)\). This conversion is a useful check: odds of 3:1 correspond to probability \(3/(3+1)=0.75\), not probability 3.

Deriving the Two-Arm Identity

The justification follows directly from the sampling and assignment steps. Conditional on \(H\), draw \((\tilde{\theta}_1,\tilde{\theta}_2)\) from the joint posterior, then assign the next participant to the arm with the larger draw. With zero probability of a tie, the event of assigning to arm 1 is exactly the event \(\tilde{\theta}_1>\tilde{\theta}_2\). The probability of that event is \(q_1\); the complementary event has probability \(q_2\). Dividing these probabilities gives the allocation odds.

\begin{aligned} P(\text{assign arm 1}\mid H) &=P(\tilde{\theta}_1>\tilde{\theta}_2\mid H)=q_1,\\ P(\text{assign arm 2}\mid H) &=P(\tilde{\theta}_2>\tilde{\theta}_1\mid H)=q_2,\\ q_1+q_2&=1,\\ \text{allocation odds}_{1:2} &=\frac{P(\text{assign arm 1}\mid H)} {P(\text{assign arm 2}\mid H)} =\frac{q_1}{q_2}. \end{aligned}

The equality is a statement about probabilities conditional on the same posterior state. It does not say that the eventual observed assignment counts in a finite trial must equal these odds: the posterior state may change as responses accrue, and the realized assignments are random.

Worked Example: Converting Best-Arm Probabilities to Odds

Suppose a hypothetical two-arm trial has posterior best-arm probabilities \(q_1=0.72\) and \(q_2=0.28\), with no posterior tie probability. Assume the next participant is assigned by Thompson sampling from this posterior state.

The allocation odds for arm 1 versus arm 2 are \(q_1/q_2=0.72/0.28=72/28=18/7\approx2.571\). Thus the odds are about 2.571:1 in favor of assigning to arm 1. Equivalently, for every assignment to arm 2, the rule assigns to arm 1 about 2.571 times, in repeated randomizations from this fixed state.

The probabilities provide a check: \(18/(18+7)=18/25=0.72\) for arm 1, and \(7/25=0.28\) for arm 2. Under the two-arm, no-tie assumptions, the same odds are the posterior odds that arm 1 is superior to arm 2.

Obtaining the Odds From Posterior Distributions

The best-arm probabilities can be calculated analytically or numerically from the posterior distributions. The odds calculation comes after those probabilities have been established: divide the probability that arm 1 is best by the probability that arm 2 is best. The following example shows the entire calculation for simple continuous posteriors, including the probability integral.

Worked Example: Exact Calculation With Beta Posteriors

Suppose, hypothetically, that the current response-rate posteriors are \(\theta_1\mid H\sim\text{Beta}(2,1)\) and \(\theta_2\mid H\sim\text{Beta}(1,2)\). These continuous distributions give zero probability to an exact tie. Their densities on \(0<x<1\) are \(f_1(x)=2x\) and \(f_2(y)=2(1-y)\). The distribution function for \(\theta_2\) is \(F_2(x)=2x-x^2\).

Conditioning on a value \(x\) for \(\theta_1\), the probability that \(\theta_2<x\) is \(F_2(x)\). Therefore,

\begin{aligned} q_1 &=P(\theta_1>\theta_2\mid H) =\int_0^1 f_1(x)F_2(x)\,dx\\ &=\int_0^1 2x(2x-x^2)\,dx =\int_0^1(4x^2-2x^3)\,dx\\ &=\left[\frac{4x^3}{3}-\frac{x^4}{2}\right]_0^1 =\frac{4}{3}-\frac{1}{2} =\frac{5}{6}. \end{aligned}

Since there are two arms and no ties, \(q_2=1-q_1=1/6\). The allocation odds are \((5/6)/(1/6)=5\), or 5:1 for arm 1. This is also the posterior odds that \(\theta_1>\theta_2\).

The posterior means are \(E(\theta_1\mid H)=2/3\) and \(E(\theta_2\mid H)=1/3\), so their ratio is 2, not 5. That difference is not a contradiction: the odds describe the probability that a posterior draw for arm 1 exceeds a draw for arm 2, whereas the ratio of posterior means compares average response rates.

Odds Estimated by Posterior Simulation

When the posterior best-arm probabilities are estimated by Monte Carlo simulation, use the simulated best-arm counts to estimate \(q_1\) and \(q_2\), then form their ratio. For two arms with no ties, if \(M\) independent posterior draws produce \(B_1\) draws in which arm 1 is best and \(B_2=M-B_1\) in which arm 2 is best, then \(\hat{q}_1=B_1/M\), \(\hat{q}_2=B_2/M\), and the estimated odds simplify to \(B_1/B_2\). These are simulation estimates of probabilities conditional on the model and current data, not additional observed trial responses.

Worked Example: Odds From Simulated Best-Arm Counts

In a hypothetical computation, \(M=10{,}000\) independent joint posterior draws are made from the current two-arm posterior. Arm 1 has the larger draw in \(B_1=7{,}350\) simulations, and arm 2 has the larger draw in \(B_2=2{,}650\). Assume no draws are ties.

The estimated probabilities are \(\hat{q}_1=7{,}350/10{,}000=0.735\) and \(\hat{q}_2=2{,}650/10{,}000=0.265\). Thus the estimated allocation odds are \(0.735/0.265=7{,}350/2{,}650\approx2.774\), or about 2.774:1 for arm 1.

The Monte Carlo standard error of \(\hat{q}_1\), treating the best-arm indicators across independent posterior draws as Bernoulli observations, is approximately

SE(\hat{q}_1) \approx \sqrt{\frac{\hat{q}_1(1-\hat{q}_1)}{M}} = \sqrt{\frac{0.735(0.265)}{10{,}000}} \approx 0.004413.

This indicates the approximate simulation variability in the estimated probability, not uncertainty in the unknown response rates themselves. Because the odds are a ratio, their simulation error is better assessed on the log-odds scale. Here the estimated log odds are \(\log(7{,}350/2{,}650)\approx1.020\), with approximate standard error \(\sqrt{1/7{,}350+1/2{,}650}\approx0.02266\). Both calculations rely on independent posterior draws and would need reconsideration if the simulation draws were dependent.

Interpretation, Diagnostics, and Alternatives

Allocation odds summarize a relative chance of assignment, conditional on the current history and the specified Thompson-sampling rule. With two arms and no ties, they also equal the posterior odds that arm 1 is superior to arm 2. They do not describe the odds of an individual responding or a causal effect, and they are not a measure of the response-rate ratio or effect magnitude. For example, the exact Beta calculation above gives superiority odds of 5:1 even though the posterior-mean response-rate ratio is 2.

Before interpreting a reported odds value, check the source probabilities and the rule that generated them. The calculation assumes the same history and posterior state for both probabilities, exactly one of the two arms is selected, and assignment is determined by which posterior draw is larger. If the probabilities are rounded, their ratio is approximate; if they are Monte Carlo estimates, simulation error can be material when one best-arm probability is small. An estimated zero denominator makes the raw ratio undefined, so it should not be reported as infinite evidence from finite simulations without addressing simulation precision.

A posterior tie probability also matters. If \(P(\theta_1=\theta_2\mid H)>0\), then \(q_1+q_2\) as defined above is less than one, and \(q_1/q_2\) need not equal the odds of assignment: the treatment of tied draws affects assignment probabilities. The simple identity here is for zero tie probability. Similarly, an allocation rule that modifies Thompson-sampling probabilities—for example, by imposing minimum assignment probabilities—does not in general retain the identity between assignment odds and posterior best-arm odds.

The main alternatives answer different questions. Fixed randomization uses prespecified allocation probabilities rather than current posterior best-arm probabilities. Greedy allocation selects the arm with the largest chosen point estimate, rather than randomizing in proportion to the posterior probability of being best. Comparing posterior mean response rates describes average responses under the posterior, not the chance that a posterior draw for one arm exceeds a draw for the other. These summaries should not be substituted for one another.

Common mistake: “The odds are 3:1, so the response rate on arm 1 is three times the response rate on arm 2.” This confuses allocation odds (and, with two arms and no ties, posterior odds of superiority) with a response-rate ratio. The correction is to interpret 3:1 as a ratio of conditional assignment probabilities; assess response-rate levels or ratios separately using the posterior response distributions.

The odds are a concise description of a single allocation decision under a stated posterior state. They do not summarize how allocations will evolve as new responses change that state, nor do they by themselves establish that the posterior model or the resulting trial design is appropriate.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

If \(q_1=0.60\) and \(q_2=0.40\), what are the allocation odds for arm 1 versus arm 2, assuming no ties?
If the two-arm allocation odds are 3:2 and there is no tie probability, what are the assignment probabilities?
If \(P(\theta_1>\theta_2\mid H)=0.45\), \(P(\theta_2>\theta_1\mid H)=0.45\), and the remaining probability is a tie, can the allocation odds be obtained by simply dividing the two displayed probabilities? Explain.
Explain why allocation odds of 5:1 do not imply that one arm's response rate is five times the other's.

Solutions. (1) The odds are \(0.60/0.40=1.5\), or 1.5:1 for arm 1. (2) The probabilities are \(3/(3+2)=0.60\) for arm 1 and \(2/(3+2)=0.40\) for arm 2. (3) No. The ratio \(0.45/0.45=1\) is the ratio of the two strict-superiority probabilities, but it is not necessarily the assignment odds because the tie probability is positive and how ties are assigned affects the assignment probabilities. (4) The 5:1 quantity compares the probabilities of assignment to the arms under Thompson sampling; with two arms and no ties, it also compares posterior probabilities of superiority. A response-rate ratio instead compares response-rate parameters or summaries of their posterior distributions.