From Binary Responses to a Randomization Probability
In a two-arm trial with a binary outcome, each treatment’s unknown response rate can be represented by a probability between zero and one. Thompson sampling uses uncertainty about those rates as part of the randomization rule: it draws a plausible rate for each treatment from its posterior distribution and assigns the next participant to the treatment with the larger draw. Thus, the probability of assigning a participant to arm A is the posterior probability that A’s rate exceeds B’s. This tutorial works through that calculation from observed successes and failures to the final allocation probability.
The calculation is conditional on the trial information available when the next assignment is made. It does not mean that the treatment with the larger observed success proportion must receive the next participant. The posterior distributions account for both the observed data and the prior information, and the randomization probability reflects uncertainty in the comparison.
Model, Updating, and the Allocation Rule
Let \(Y_{ij}\) be the binary response for participant \(j\) assigned to treatment \(i\), where \(i\) is A or B; a success is coded 1 and a failure 0. Assume that, conditional on a treatment-specific response probability \(\theta_i\), responses assigned to treatment \(i\) are Bernoulli with success probability \(\theta_i\). Write \(s_i\) for the number of observed successes and \(f_i\) for the number of observed failures on arm \(i\).
For this calculation, suppose the prior distributions are independent, with \(\theta_i\sim\operatorname{Beta}(a_i,b_i)\), where \(a_i,b_i>0\). The Beta density is proportional to \(\theta_i^{a_i-1}(1-\theta_i)^{b_i-1}\) on \(0<\theta_i<1\). The Beta-Bernoulli updating rule gives the posterior distribution by adding successes to the first parameter and failures to the second.
This posterior updating applies when the arm-specific response likelihood is Bernoulli and the prior distributions for the two rates are independent. In an adaptive trial, assignments may depend on earlier outcomes. Conditional on the observed assignment history, the usual updating is still valid when the assignment mechanism is known and does not introduce an additional dependence on the unknown response rates beyond the recorded outcomes. The posterior independence used by the two-draw rule follows from independent priors and separate arm-specific likelihoods. If a model instead couples the treatment rates, independent draws from the two marginal posteriors would not represent that joint posterior.
For independent continuous posterior draws, the probability of an exact tie is zero. Consequently, \(P(\Theta_A>\Theta_B\mid\text{data})\) and \(P(\Theta_B>\Theta_A\mid\text{data})\) sum to one. Continuity of the marginal distributions alone is not enough: the joint distribution must also give probability zero to the diagonal \(\Theta_A=\Theta_B\). The independent continuous Beta posteriors specified here satisfy that condition.
Let \(f_A(x)\) be the posterior density of \(\Theta_A\), and let \(F_B(x)\) be the posterior cumulative distribution function of \(\Theta_B\). Conditioning on the value drawn for A gives a convenient exact expression:
At a fixed value \(x\) drawn for A, the chance that the independent draw for B is smaller than \(x\) is \(F_B(x)\). Averaging that conditional chance over the density \(f_A(x)\) proves the integral expression. Reversing the roles gives \(P(\Theta_B>\Theta_A\mid\text{data})=\int_0^1 f_B(y)F_A(y)\,dy\). Under the no-tie condition, these two integrals sum to one.
Worked Calculations
Worked Example: One Observation on Each Arm
Suppose both arms have a \(\operatorname{Beta}(1,1)\) prior. In hypothetical data, A has one success and no failures, while B has no successes and one failure. The response model is Bernoulli, and the independent priors and separate arm likelihoods give independent posteriors.
Updating gives \(\theta_A\mid\text{data}\sim\operatorname{Beta}(2,1)\) and \(\theta_B\mid\text{data}\sim\operatorname{Beta}(1,2)\). Their densities are \(f_A(x)=2x\) and \(f_B(y)=2(1-y)\), so \(F_B(x)=1-(1-x)^2\). Therefore,
The next participant is therefore assigned to A with probability about 0.8333 and to B with probability about 0.1667. These are probabilities of winning the posterior draw comparison, not statements that A’s true response rate is known.
Worked Example: More Balanced Evidence on Both Arms
Now suppose both arms again start with independent \(\operatorname{Beta}(1,1)\) priors. In hypothetical data, A has two successes and one failure, while B has one success and two failures. Each arm’s response records are treated as Bernoulli observations, so the posterior parameters are obtained by adding the respective counts.
The posteriors are \(\theta_A\mid\text{data}\sim\operatorname{Beta}(3,2)\) and \(\theta_B\mid\text{data}\sim\operatorname{Beta}(2,3)\). The densities are \(f_A(x)=12x^2(1-x)\) and \(f_B(y)=12y(1-y)^2\). Integrating B’s density gives \(F_B(x)=6x^2-8x^3+3x^4\). Substitution into the allocation formula yields
Thus, under this posterior state, A receives the next participant with probability about 0.7571 and B with probability \(1-0.7571=0.2429\). The calculation uses the full posterior distributions; it is not obtained by comparing only the two observed success proportions.
Worked Example: A More Concentrated Posterior for A
For a third hypothetical data set, let both priors be independent \(\operatorname{Beta}(1,1)\). Suppose A has two successes and no failures, and B has one success and one failure. These are valid binary-response counts, and the same Bernoulli likelihood and independent-prior assumptions apply.
The resulting posteriors are \(\theta_A\mid\text{data}\sim\operatorname{Beta}(3,1)\) and \(\theta_B\mid\text{data}\sim\operatorname{Beta}(2,2)\). Their relevant functions are \(f_A(x)=3x^2\) and \(F_B(x)=3x^2-2x^3\). Hence,
The conditional allocation probability for A is 0.8000, and the probability for B is 0.2000. The posterior for A is concentrated toward larger response rates in this example, but the allocation probability is still calculated by comparing random draws, not by assigning A with certainty.
Checking the Calculation and Its Meaning
A reliable calculation has three linked parts: the posterior parameters, the probability comparison, and the randomization interpretation. A mistake in the first part changes the distributions being compared; a mistake in the last can turn a valid probability into an incorrect clinical claim.
For each arm, state the prior parameters and the observed success and failure counts. Confirm that the counts sum to the number of observed responses on that arm.
Calculate \(a_i+s_i\) and \(b_i+f_i\). A failure increases the second Beta parameter; it does not increase the first.
Use \(P(\Theta_A>\Theta_B\mid\text{data})=\int_0^1 f_A(x)F_B(x)\,dx\), or an equivalent exact calculation. Check that the complementary probability is between zero and one and sums to one when ties have probability zero.
Randomize to A with the calculated probability and to B with its complement. The result is conditional on the current data and specified model, prior, and allocation rule.
For a hand calculation, derive the posterior density and CDF explicitly, as in the examples, and check the integral’s bounds and arithmetic. For numerical integration, the integrand \(f_A(x)F_B(x)\) is nonnegative and its integral must lie between zero and one. A probability close to one should correspond to posterior mass that usually places A’s draw above B’s draw; a reported value outside \([0,1]\), or two complementary probabilities that do not sum to one under the no-tie assumptions, signals an error.
The main alternative to evaluating the integral is to estimate the same probability by simulation: repeatedly draw independently from the two posteriors and count the fraction of draws for which A exceeds B. This approximates the integral, while direct integration or an exact finite calculation avoids simulation error when available. A deterministic comparison of posterior means is different: it summarizes each posterior by one number rather than using the probability of a posterior draw being larger. It therefore does not give the Thompson-sampling probability.
The result depends on the outcome coding, the Bernoulli model, the prior parameters, and the assumption about dependence between treatment rates. If responses are not adequately represented as binary outcomes, if the prior joint distribution does not factor by arm, or if the allocation rule uses a different comparison, this calculation does not automatically apply. Also, a posterior allocation probability alone does not assess the broader operating characteristics of a trial, such as its long-run balance or inferential performance.
Check Your Understanding
For each question, calculate or explain the requested quantity. Solutions are included below the questions.
- With an independent \(\operatorname{Beta}(1,1)\) prior on each arm, A has one success and one failure, while B has one success and no failures. What are the two posterior distributions?
- If the posteriors are \(\Theta_A\sim\operatorname{Beta}(2,1)\) and \(\Theta_B\sim\operatorname{Beta}(1,2)\), calculate the probability of assigning the next participant to B.
- Why is it not enough to know that each posterior marginal distribution is continuous to conclude that ties have probability zero?
- Suppose A’s posterior mean exceeds B’s posterior mean. Does that fact alone determine the Thompson-sampling allocation probability? Explain.
Solutions. 1. Adding successes and failures gives \(\theta_A\mid\text{data}\sim\operatorname{Beta}(2,2)\) and \(\theta_B\mid\text{data}\sim\operatorname{Beta}(2,1)\). 2. From the first worked example, \(P(\Theta_A>\Theta_B\mid\text{data})=5/6\). Under independent continuous draws, \(P(\Theta_B>\Theta_A\mid\text{data})=1-5/6=1/6\approx0.1667\). 3. Marginal continuity does not rule out dependence that puts positive probability on \(\Theta_A=\Theta_B\); the joint distribution must assign zero probability to the diagonal. Independent continuous Beta draws do so. 4. No. The rule uses the probability that a posterior draw for A exceeds an independent posterior draw for B. Posterior means alone do not specify that probability.