From Posterior-Best Probabilities to Assignment Probabilities
Thompson sampling converts uncertainty about which treatment is best into probabilities of assigning the next participant to each treatment. Those probabilities can become very small for an arm when the accumulated evidence favors other arms. In a clinical trial, the design may require every arm to retain a minimum chance of assignment, for reasons such as maintaining recruitment to each treatment or ensuring a minimum level of exploration. Imposing that requirement changes the assignment probabilities: they are no longer simply the posterior-best probabilities.
Let \(K\) denote the number of treatment arms, and let \(\mathcal{H}_t\) denote the trial history available immediately before participant \(t\) is assigned. For arm \(k\), write \(q_k=P(k\text{ is best}\mid\mathcal{H}_t)\). The probabilities \(q_1,\ldots,q_K\) are nonnegative and sum to one. They may be obtained by the posterior calculations or numerical integration described in the earlier material on posterior best probabilities. Here, the question is how to modify them to satisfy minimum assignment probabilities.
These probabilities have different meanings. The value \(q_k\) describes posterior uncertainty about which arm is best, given the trial history. The value \(p_k\) is the probability that the next participant is actually assigned to arm \(k\), conditional on that same history. Applying floors does not update the posterior; it changes the allocation rule.
The Floor-and-Renormalize Rule
A natural rule is to set any arm below its floor to that floor, then distribute the remaining probability among the other arms in proportion to their original posterior-best probabilities. Thus, an arm held at its floor receives no share of the remaining probability mass. Arms not held at their floors retain their relative proportions from \(q\).
When the floors are all the same value \(\ell\), feasibility requires \(K\ell\leq1\). If the floors sum to one, every arm must receive exactly its floor. If their sum is less than one, some probability mass remains to be allocated above the floors. The multiplier \(c\) is not generally one: it is chosen to make all assignment probabilities sum to one after the floors are imposed.
The rule can be calculated by identifying the arms that are held at their floors. Let \(F\) be that set and \(U\) the remaining arms. For a proposed set \(F\), the free arms must share the probability left after assigning the floors to \(F\). Therefore,
The proposed sets are consistent only if \(cq_k\leq\ell_k\) for arms in \(F\) and \(cq_k\geq\ell_k\) for arms in \(U\). One practical calculation starts with all arms free, computes \(c\), and moves any arm whose \(cq_k\) is below its floor into \(F\). Recompute \(c\) for the remaining free arms, repeating until no free arm falls below its floor. The final probabilities sum to one by construction, and each free arm meets its floor.
Why Renormalization Preserves Relative Probabilities
Once the set \(F\) is fixed, the total probability assigned to the free arms is \(1-\sum_{k\in F}\ell_k\). To preserve their relative posterior-best probabilities, their assignment probabilities must have the same ratios as their \(q_k\) values. For any \(i,j\in U\), this requirement is \(p_i/p_j=q_i/q_j\). Setting \(p_k=cq_k\) for all \(k\in U\) gives those ratios, and choosing \(c\) to use exactly the remaining probability mass yields the formula above.
Worked Example: One Arm Falls Below a Common Floor
Suppose three arms have posterior-best probabilities \(q=(0.70,0.20,0.10)\), and the design sets a common minimum of \(\ell=0.15\). The floors are feasible because \(3(0.15)=0.45\leq1\). The first two arms can initially be treated as free while the third is checked against its floor.
If arm 3 is held at its floor, the probability left for arms 1 and 2 is \(1-0.15=0.85\). Their posterior-best probabilities sum to \(0.70+0.20=0.90\), so their common multiplier is \(c=0.85/0.90=0.9444\). Thus \(cq_1=0.9444(0.70)=0.6611\), \(cq_2=0.9444(0.20)=0.1889\), and \(cq_3=0.9444(0.10)=0.09444\).
Because \(0.09444<0.15\), holding arm 3 at its floor is consistent. The adjusted probabilities are \(p=(0.6611,0.1889,0.1500)\). They sum to \(1.0000\), and arms 1 and 2 preserve their original ratio: \(0.6611/0.1889\approx0.70/0.20=3.5\).
Worked Example: An Arm Initially Above Its Floor
Suppose four arms have posterior-best probabilities \(q=(0.52,0.28,0.14,0.06)\) and a common floor of \(0.10\). The floors are feasible because \(4(0.10)=0.40\leq1\). Check arm 4 at its floor; the other three are provisionally free.
With \(p_4=0.10\), the remaining mass is \(0.90\). The free arms' posterior-best probabilities sum to \(0.52+0.28+0.14=0.94\), so \(c=0.90/0.94=0.9574\). The resulting free-arm probabilities are \(p_1=0.52(0.9574)=0.4979\), \(p_2=0.28(0.9574)=0.2681\), and \(p_3=0.14(0.9574)=0.1340\). Each exceeds \(0.10\), while \(cq_4=0.06(0.9574)=0.05745<0.10\), confirming that only arm 4 should be held at its floor.
The allocation is therefore \(p=(0.4979,0.2681,0.1340,0.1000)\), which sums to \(1.0000\) up to rounding. Arms 1 through 3 retain their posterior-best ratios; arm 4 receives the required minimum rather than its smaller proportional share.
Worked Example: Floors Can Bind for More Than One Arm
Suppose three arms have \(q=(0.80,0.15,0.05)\) and unequal floors \((0.10,0.20,0.25)\). These floors are feasible because their sum is \(0.55\). First hold arm 3 at \(0.25\). The remaining \(0.75\) is provisionally shared by arms 1 and 2, whose \(q\) values sum to \(0.95\). This gives \(c=0.75/0.95=0.7895\), so \(cq_1=0.6316\) and \(cq_2=0.1184\).
Arm 2 falls below its floor because \(0.1184<0.20\), so it too must be held at its floor. The remaining mass for arm 1 is \(1-0.20-0.25=0.55\). The final allocation is \(p=(0.55,0.20,0.25)\). Arm 1 exceeds its floor, and all three probabilities sum to one. The fact that \(p_1=0.55\) does not preserve its original value \(q_1=0.80\): it is the sole arm left to receive probability beyond the two binding floors.
Practical Checks and Failure Modes
An implementation should be checked against the allocation rule, not just against whether its entries look plausible. For a computed vector \(p\), verify that \(p_k\geq\ell_k\) for every arm, that \(\sum_k p_k=1\) up to numerical precision, and that the set of arms at their floors agrees with the comparisons between \(cq_k\) and \(\ell_k\). Among arms above their floors, \(p_i/p_j\) should equal \(q_i/q_j\). Small discrepancies from rounded displays are expected; the calculations should use unrounded values.
A common error is to clamp first and then normalize: set \(r_k=\max(q_k,\ell_k)\), then use \(p_k=r_k/\sum_j r_j\). This does not generally enforce the stated floors. For example, with \(q=(0.70,0.20,0.10)\) and common floor \(0.15\), clamping gives \(r=(0.70,0.20,0.15)\), whose sum is \(1.05\). Normalizing yields \(p_3=0.15/1.05=0.1429\), which is below the required floor. The correction is to reserve the floor for the binding arm and renormalize the free arms over the remaining mass, as in the first worked example.
Another option is to mix the posterior-best probabilities with a uniform distribution. With a common floor \(\ell\) and \(K\ell\leq1\), the rule \(p_k=\ell+(1-K\ell)q_k\) sums to one and guarantees \(p_k\geq\ell\). It does not, however, preserve the ratios \(q_i/q_j\) among arms above their floors. It is a different allocation rule, not an alternative calculation of the same floor-and-renormalize rule.
| Approach | Guarantees the specified floors? | What happens to relative probabilities? |
|---|---|---|
| Floor and renormalize | Yes, if the floors are feasible and the rule is applied correctly | Preserves ratios among arms not held at their floors |
| Clamp, then normalize | Not necessarily | Normalization can push a clamped arm below its floor |
| Uniform-probability mixture | Yes, for feasible common floors | Generally changes ratios among all arms |
| Use \(q\) without adjustment | Only if every \(q_k\geq\ell_k\) | Uses posterior-best probabilities unchanged |
The floors are design choices, not evidence that an arm is clinically effective or that posterior uncertainty has changed. They also cannot be feasible if \(\sum_k\ell_k>1\), because no probability vector can satisfy all those lower bounds and still sum to one. Finally, when an arm is held at its floor, its assignment probability need not reflect its posterior-best probability proportionally. The floor rule therefore changes the design's allocation behavior even though the underlying posterior remains the same.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- For \(q=(0.60,0.25,0.15)\) and a common floor of \(0.20\), which arm is held at its floor, and what are the adjusted probabilities?
- Why is it necessary to check that the floors sum to at most one?
- With \(q=(0.50,0.30,0.20)\) and a common floor of \(0.10\), does any arm bind? Give the allocation.
- What check detects the mistake of clamping the \(q_k\) values and then normalizing them?
Solutions. (1) Arm 3 binds. The free-arm mass is \(0.80\), and \(q_1+q_2=0.85\), so \(c=0.80/0.85=0.9412\). The allocation is \(p=(0.5647,0.2353,0.2000)\); both free probabilities exceed \(0.20\), and the total is one. (2) Since probabilities must sum to one, floors whose sum exceeds one cannot all be met. (3) No arm binds: \(q\) already sums to one and each entry is at least \(0.10\). Thus \(p=(0.50,0.30,0.20)\). (4) Check both \(p_k\geq\ell_k\) for every arm and \(\sum_kp_k=1\); normalization after clamping can violate the first condition.