What Is Conditional on the Trial History?
In a sequential trial, the assignment rule uses the information available when a participant is ready to be randomized. That information can include prior assignments and observed outcomes, as well as the current participant’s eligibility and the set of arms currently available. The assignment probability is therefore not a fixed property of a treatment arm: it is conditional on the trial history at that decision point.
Consider a binary-response setting with \(K\) available treatment arms. Let \(\theta_k\) denote the response probability for arm \(k\), and let \(H_{i-1}\) denote the trial history available immediately before participant \(i\) is assigned. Write \(A_i\) for the arm assigned to that participant. In Thompson sampling, a draw is taken from each arm’s posterior distribution given \(H_{i-1}\), and the arm with the largest drawn response probability is selected.
This is a probability over the algorithm’s randomization before its posterior draws are made. Once the draws have been made, the selected arm is determined by those draws (apart from any tie rule); the realized assignment is one outcome of the randomization, not the assignment probability itself. Conditioning on the posterior draws as well as the history would make the assignment deterministic in the no-tie case.
Assumptions and the Posterior-Best-Arm Identity
For the calculations below, assume responses are binary and, conditional on an arm’s response probability \(\theta_k\), outcomes assigned to that arm follow a Bernoulli model. Each arm has a specified prior, and the posterior distribution is formed from the data observed by the decision point. The posterior-sampling rule draws from the joint posterior distribution of the arm response probabilities and assigns to an arm with the largest draw. No outcomes from participant \(i\) or later participants are included in \(H_{i-1}\).
When the arm-specific posteriors are independent and continuous, denote the posterior density and cumulative distribution function for arm \(k\) by \(f_k\) and \(F_k\). Conditional on a draw \(\tilde{\theta}_k=t\), arm \(k\) is selected if every other draw is less than \(t\). Independence makes the conditional probability of that event the product of the other arms’ cumulative probabilities. Averaging over the draw for arm \(k\) gives:
This integral is a calculation of a posterior probability, not a probability that the treatment will succeed for the current participant. For continuous posteriors, ties have probability zero, so exactly one arm is best with probability one. Consequently, the conditional assignment probabilities are nonnegative and sum to one. If posteriors are dependent, the identity as a joint probability still holds, but the product-integral formula generally does not. If ties can have positive posterior probability, the allocation rule must specify how tied arms are handled, and the probabilities must reflect that rule.
Worked Example: Two Arms With Different Beta Posteriors
Suppose, hypothetically, that immediately before a participant is assigned, the posterior distributions are independent with \(\theta_1\mid H_{i-1}\sim\mathrm{Beta}(2,1)\) and \(\theta_2\mid H_{i-1}\sim\mathrm{Beta}(1,2)\). Both posteriors are continuous, so ties have probability zero. The first density is \(f_1(x)=2x\), and the second cumulative distribution function is \(F_2(x)=1-(1-x)^2=2x-x^2\), for \(0\le x\le 1\).
The probability of assigning arm 1 is the probability its posterior draw exceeds the draw for arm 2:
There are only two arms and ties have probability zero, so \(P(A_i=2\mid H_{i-1})=1-\frac{5}{6}=\frac{1}{6}\approx0.1667\). The probabilities add to one. In this hypothetical decision, Thompson sampling assigns arm 1 about 83.33% of the time over repeated randomizations from this same posterior state, and arm 2 about 16.67% of the time.
Three Arms and a Useful Probability Check
With more than two arms, the probability for an arm is the chance its draw exceeds all the other draws, not just one selected competitor. Under independent continuous posteriors, the product in the integral accounts for every competing arm. The probabilities still sum to one because exactly one draw is largest, except on a tie event of probability zero.
Worked Example: Three Arms With One Nonuniform Posterior
Suppose, hypothetically, that \(\theta_1\mid H_{i-1}\sim\mathrm{Beta}(2,1)\), while \(\theta_2\) and \(\theta_3\) each have independent \(\mathrm{Beta}(1,1)\) posteriors. The latter are uniform on \([0,1]\). Conditional on the draw for arm 1 being \(x\), each competing draw is below \(x\) with probability \(x\), so both are below \(x\) with probability \(x^2\).
The density of the draw for arm 1 is \(2x\), giving:
Arms 2 and 3 have identical posterior distributions and play interchangeable roles in the assignment rule. Their assignment probabilities are therefore equal. Since the three probabilities sum to one, each of the remaining two is \((1-\frac{1}{2})/2=\frac{1}{4}\). Thus the vector of conditional assignment probabilities is \((0.5000,\,0.2500,\,0.2500)\), which sums to one.
A Realized Assignment Is Not Its Probability
The conditional probabilities describe the distribution of the next assignment before the algorithm’s posterior draws are made. A single participant receives one arm. That arm need not be the one with the largest conditional assignment probability: randomization deliberately allows other arms to be assigned as well. A realized assignment is not evidence that its conditional probability was one, nor does it change the probability that was calculated from the history before assignment.
Worked Example: A Possible Assignment Under a Simple Posterior State
Suppose, hypothetically, that the prior response probability for each of two arms is \(\mathrm{Beta}(1,1)\). By the decision point, arm 1 has one observed success and no failures, while arm 2 has no observed responses. Under the Beta-Bernoulli model, the posterior distributions are \(\theta_1\mid H_{i-1}\sim\mathrm{Beta}(2,1)\) and \(\theta_2\mid H_{i-1}\sim\mathrm{Beta}(1,1)\). The probability that arm 1’s draw is largest is its expected draw, because the other draw is uniform:
Therefore \(P(A_i=2\mid H_{i-1})=\frac{1}{3}\). Now suppose the algorithm happens to draw \(0.40\) for arm 1 and \(0.70\) for arm 2. It assigns the participant to arm 2 because \(0.70>0.40\). That realized assignment is consistent with a conditional assignment probability of \(1/3\) for arm 2: the probability describes how often that assignment would occur over repeated posterior draws at the same history, not what must happen on this particular draw.
How to Check and Interpret the Probabilities
In an implementation, the posterior state used for the calculation must be the state immediately before the participant’s randomization. The assignment probabilities are conditional on that state; after a response is observed, a later decision may use a different history and therefore different probabilities. The particular procedure for updating that state is separate from the probability calculation here.
Useful checks include confirming that every available arm is represented, that the probabilities are between zero and one, and that they sum to one up to numerical precision. If probabilities are estimated by Monte Carlo posterior draws rather than calculated exactly, their reported values are estimates and can differ slightly from the exact probabilities due to simulation error. Repeating the calculation with more draws can assess this numerical variability.
The assumptions behind a convenient calculation also need checking. The product-integral formula requires independent arm-specific posterior draws. The simple claim that ties have probability zero requires continuous posterior distributions. If either condition fails, calculate the joint probability under the actual posterior and apply the assignment rule’s specified tie handling. A sum that is far from one, or an arm probability that changes when the same history is supplied, can indicate a coding error, inconsistent posterior state, or mishandled tie rule.
| Approach | Assignment probability given history | What distinguishes it |
|---|---|---|
| Thompson sampling | Posterior probability that an arm is best | Uses posterior uncertainty as well as posterior location |
| Plug-in greedy allocation | Typically assigns to the arm with the largest estimated response probability | Does not sample from the posterior; it may assign deterministically to one arm |
| Fixed randomization | Specified probabilities that do not depend on the observed response history | Does not adapt its probabilities through posterior best-arm probabilities |
These approaches answer different questions. In particular, a posterior probability that an arm is best is not its posterior mean response probability. The earlier material on posterior best-arm probabilities and plug-in greedy allocation provides the broader distinction; here the key point is that the best-arm probability is also the conditional randomization probability under Thompson sampling. It is not, by itself, a guarantee of clinical benefit or a statement that the arm is best in the population with certainty.
Common Mistakes and Limitations
- Calling the realized arm the assignment probability. An observed participant assigned to arm 2 does not imply \(P(A_i=2\mid H_{i-1})=1\). The conditional probability is calculated before that randomization.
- Using a posterior mean as the Thompson-sampling probability. The assignment probability is a probability of being best, which depends on the full posterior distributions, not just their means.
- Forgetting the conditioning history. Probabilities must be tied to the information available before the decision, not to later outcomes.
- Multiplying marginal probabilities when posteriors are dependent. The product-integral expression assumes independent arm draws; dependence requires the joint posterior calculation.
- Ignoring Monte Carlo error. Simulated estimates of probabilities can fluctuate, especially with few posterior draws. They need not equal exact probabilities digit for digit.
Check Your Understanding
Answer each question using the history immediately before the participant is assigned.
- If three independent continuous posterior distributions are identical, what is each arm’s conditional assignment probability under Thompson sampling?
- Why can an arm with conditional assignment probability \(0.25\) be assigned to a particular participant?
- For independent continuous arm posteriors, what event does the integral of \(f_k(t)\prod_{j\ne k}F_j(t)\) represent?
- What assumption is needed for the product of the other arms’ cumulative distribution functions in that integral?
- In the two-arm example with arm 1 distributed as \(\mathrm{Beta}(2,1)\) and arm 2 uniformly distributed, what are the conditional assignment probabilities?
Solutions. 1. Each arm has probability \(1/3\), by symmetry; the probabilities sum to one and no ties occur. 2. The probability is over repeated randomizations from the same pre-assignment state. A particular posterior draw can make that arm’s sampled value the largest, so an assignment with probability \(0.25\) can occur. 3. It is the probability that arm \(k\)’s posterior draw exceeds every other arm’s draw, hence the probability of assigning arm \(k\). 4. The arm-specific posterior draws must be independent for the product of the other arms’ cumulative probabilities to give the conditional probability that all their draws are below \(t\). 5. Arm 1’s probability is \(\int_0^1 2x\cdot x\,dx=2/3\); arm 2’s is \(1-2/3=1/3\).