Thompson Sampling With Unequal Arm Priors

Why Unequal Priors Change Allocation

In Thompson sampling, a treatment is selected by drawing a plausible response rate from each treatment’s current posterior distribution and assigning the next participant to the treatment with the largest draw. When the arms have unequal priors, their posterior distributions can differ before any trial outcomes are observed. The allocation probabilities can therefore differ even at the start of the trial.

This is more than a change in the estimated response rates. Thompson sampling uses the full posterior distributions, including their uncertainty. A prior with a higher mean can make an arm more likely to receive the next assignment, while a more concentrated prior can make its response-rate draws less variable. The allocation probability depends on how the two distributions compare, not just on which posterior mean is larger.

Assume a two-arm trial with independent binary outcomes, where a response is the favorable outcome and a higher response rate is better. For arm \(k\), let \(\theta_k\) be its response probability. The model is \(X_{ki}\mid\theta_k\sim\operatorname{Bernoulli}(\theta_k)\), with a Beta prior for each arm. Outcomes are assumed to be observed and recorded according to this model, and the arm-specific priors are assumed independent. These assumptions make the posterior distributions independent Beta distributions.

The Beta prior mean is \(a_k/(a_k+b_k)\), and \(a_k+b_k\) is often used as a measure of prior strength or effective sample size. It is a convenient way to describe concentration, not necessarily a literal number of observed patients. After data are observed, the posterior mean is \((a_k+s_k)/(a_k+b_k+s_k+f_k)\). These means summarize each posterior, but the allocation probability compares the distributions themselves.

Calculating the Probability of a Posterior Win

Write the two posterior distributions as \(\theta_1\mid\text{data}\sim\operatorname{Beta}(A_1,B_1)\) and \(\theta_2\mid\text{data}\sim\operatorname{Beta}(A_2,B_2)\), where \(A_k=a_k+s_k\) and \(B_k=b_k+f_k\). If \(F_2\) is the cumulative distribution function of the second posterior and \(f_1\) is the density of the first, independence gives

To see why, condition on a possible draw \(\theta_1=x\). Given that draw, arm 1 wins whenever the independent draw from arm 2 is below \(x\), an event with probability \(F_2(x)\). Averaging that conditional probability over draws from arm 1 gives the integral. This identity is also useful for numerical integration; simulation can approximate the same probability by repeatedly drawing from both posteriors and recording which draw is larger.

For a simple closed-form calculation, if \(Y\sim\operatorname{Beta}(1,d)\), then \(F_Y(x)=1-(1-x)^d\). If \(X\sim\operatorname{Beta}(a,1)\), its density is \(ax^{a-1}\), so

where \(B(u,v)=\int_0^1 x^{u-1}(1-x)^{v-1}\,dx\) is the Beta function. This follows by substituting the density of \(X\) into the expectation. The same integral definition applies when the posterior shapes do not give such a short expression.

Updating Unequal Priors With Trial Outcomes

To apply the allocation rule after outcomes accrue, first update each arm’s prior using its own response and nonresponse counts. Then calculate the probability that a draw from one posterior exceeds a draw from the other. The order matters: comparing prior means to choose an arm would omit both the observed evidence and the posterior uncertainty used by Thompson sampling.

A useful way to understand the role of unequal priors is to hold the observed data fixed and change the prior specification. If this changes the posterior distributions, it can change the probability of a posterior win and thus the randomization probability. The next example makes that sensitivity explicit.

Interpreting, Checking, and Comparing the Allocation

The probability \(P(\theta_1>\theta_2\mid\text{data})\) is a posterior probability that arm 1 has the higher response rate, conditional on the model, priors, and observed data. In two-arm Thompson sampling, it is also the probability of assigning the next participant to arm 1, because the algorithm selects the arm with the larger independent posterior draw. It is not the probability that the next participant will respond if assigned to arm 1.

Before using a calculated allocation probability, check that the model and computation match the trial:

• Confirm that the outcome is coded so the favorable direction is correct. If a lower rate is preferable, the comparison must reflect that direction.
• Check the response and nonresponse counts against the trial record. A miscoded or missing outcome can change the posterior parameters.
• Check that each prior is proper and that its mean and strength match the intended clinical information. A highly concentrated prior can exert substantial influence before much data accrue.
• Check that the arm posteriors are independent as assumed here. If the model links treatment effects, multiplying independent arm distributions is not justified.
• Validate numerical integration or simulation, especially when the calculated probability is near zero or one. A Monte Carlo estimate should be accompanied by enough draws to make its simulation error small relative to the precision reported.

The main alternatives answer different questions. Equal-prior Thompson sampling avoids prior differences between arms but does not remove the need to choose a prior distribution. A fixed allocation rule does not change assignment probabilities in response to accumulating outcomes. A rule that always selects the arm with the larger posterior mean uses a point summary and does not randomize according to posterior uncertainty. These alternatives may be appropriate for different trial aims, but they are not the unequal-prior Thompson sampling calculation shown here.

Unequal priors also have limitations. They can be clinically justified, but the resulting allocation may be sensitive to assumptions that are hard to elicit. A high probability of assigning to one arm does not by itself show that the trial will answer its scientific or safety questions well. Operational safeguards, the handling of missing outcomes, and the design’s inferential goals require separate consideration. If outcomes are not binary or treatment posteriors are dependent, the Beta-Bernoulli calculation above does not apply without a suitable model and a corresponding posterior comparison.