Why “Best” Needs a Clinical Definition
Thompson sampling requires a rule for deciding which treatment is best in each draw from the posterior distribution. For a single binary response outcome, the rule may be to choose the treatment with the largest response probability. But clinical decisions often involve several outcomes: a treatment may improve response while also increasing serious toxicity, or may offer smaller symptom improvement with fewer adverse effects. In such settings, “best” is not determined by response rate alone. It depends on how the relevant outcomes are valued for the clinical decision.
The utility rule must be defined before using trial outcomes to determine allocations. It makes the decision target explicit: which outcomes count, how they are scaled, and what trade-offs the decision-maker is willing to make. Posterior uncertainty about each treatment’s utility can then be carried into allocation, rather than replacing that uncertainty with a single estimated utility.
The target utility is an expectation over the outcome distribution, not necessarily the utility of one particular participant’s realized outcome. It may be a mean utility per participant over a specified follow-up period. If the intended decision concerns a defined patient population, the utility should also specify how patient characteristics are handled, for example by averaging over a prespecified target distribution.
A utility function can assign values directly to outcome combinations or combine component outcomes using weights. Its scale and direction must be clear: here, higher utility is always better. For example, if \(R\) indicates response and \(T\) indicates serious toxicity, a function such as \(u(R,T)=10R-30T\) assigns 10 utility points to response and subtracts 30 points for serious toxicity. Its expected treatment utility is \(10P(R=1\mid a)-30P(T=1\mid a)\). The numerical weights are meaningful only if they represent an agreed decision rule; they are not learned automatically from the response data.
From a Utility Rule to an Allocation Probability
At each allocation decision, the posterior distribution represents uncertainty about the treatment-specific parameters. For each posterior draw, calculate each arm’s utility under the same prespecified rule, then identify the arm with the largest utility. The fraction of draws in which an arm is best estimates its posterior best-treatment probability. Thus, an arm can receive a high allocation probability without having the largest posterior mean utility, if its posterior draws often place it just above the alternatives.
This differs from assigning everyone to the arm with the largest estimated response rate, or even always assigning to the arm with the largest posterior mean utility. Those rules select an arm using a point summary. Thompson sampling instead randomizes according to posterior probabilities of being best under the chosen clinical objective.
Worked Example: Response and Serious Toxicity
Suppose two treatments, A and B, are compared using response and serious toxicity. The prespecified utility is \(u(R,T)=10R-30T\), where \(R=1\) denotes response and \(T=1\) denotes serious toxicity. For posterior draws of the response and toxicity probabilities, expected utility is \(U_a=10p_{R,a}-30p_{T,a}\), in utility points per participant. The following four paired rows are hypothetical posterior draws, not observed trial results.
| Draw | A: response probability | A: toxicity probability | A: utility | B: response probability | B: toxicity probability | B: utility | Best |
|---|---|---|---|---|---|---|---|
| 1 | 0.70 | 0.10 | 4.00 | 0.66 | 0.05 | 5.10 | B |
| 2 | 0.75 | 0.08 | 5.10 | 0.68 | 0.07 | 4.70 | A |
| 3 | 0.72 | 0.12 | 3.60 | 0.65 | 0.04 | 5.30 | B |
| 4 | 0.69 | 0.06 | 5.10 | 0.70 | 0.09 | 4.30 | A |
For example, in draw 1, A’s utility is \(10(0.70)-30(0.10)=7.00-3.00=4.00\); B’s is \(10(0.66)-30(0.05)=6.60-1.50=5.10\), so B is best. A is best in 2 of the 4 draws and B in 2 of 4. The draw-based estimates of their best-treatment probabilities are therefore \(2/4=0.50\) each. With a larger posterior simulation, the corresponding fractions would provide a more precise numerical approximation.
The average response probability across these draws is higher for A: \(0.715\), compared with \(0.6725\) for B. Choosing only by response would therefore favor A. The utility rule accounts for both response and toxicity, and its posterior best probabilities reflect their prespecified trade-off.
Why Posterior Best Probabilities Give the Allocation Rule
The connection between posterior utility draws and allocation probabilities follows directly from the probability of the event that an arm is best. Let \(I_a\) equal 1 when arm \(a\) has the largest utility in a posterior draw, and 0 otherwise. With a unique best arm, exactly one of the indicators \(I_1,\ldots,I_K\) equals 1 in each draw. Taking the posterior expectation gives the probability that each arm is best:
Because the indicators sum to 1 in every draw, their posterior expectations also sum to 1:
Thus the probabilities form a valid randomization distribution. In computation, if \(M\) posterior draws produce no ties and arm \(a\) is best in \(m_a\) draws, the Monte Carlo estimate is \(\hat q_{a,t}=m_a/M\). This is a simulation estimate of the posterior probability, not a separate clinical utility or a guarantee that the arm will have the best realized outcome.
The argument assumes that utility is defined for every arm on a common, comparable scale and that the posterior draws represent the joint posterior distribution of the relevant treatment quantities. If treatment utilities are dependent, the joint draws must preserve that dependence: independently pairing draws from marginal distributions can distort which arm is best. The probability statement also presumes either a unique best arm with posterior probability one or a tie-handling rule; the unique-best formula above is simplest when posterior ties have probability zero.
Worked Example: Three Outcomes and Three Treatments
Suppose utility rewards response and symptom improvement and penalizes serious toxicity. Define \(U_a=10p_{R,a}+1.5\mu_{Q,a}-25p_{T,a}\), where \(p_{R,a}\) is the response probability, \(\mu_{Q,a}\) is mean symptom improvement in points, and \(p_{T,a}\) is the serious-toxicity probability. The response, symptom, and toxicity components are combined into utility points per participant. The formula, including its weights, is assumed prespecified. A posterior simulation produces the following utility values across six hypothetical draws:
| Draw | A utility | B utility | C utility | Best |
|---|---|---|---|---|
| 1 | 8.0 | 7.0 | 6.5 | A |
| 2 | 5.0 | 9.0 | 7.0 | B |
| 3 | 6.0 | 8.0 | 10.0 | C |
| 4 | 9.0 | 8.0 | 7.0 | A |
| 5 | 7.0 | 6.0 | 8.0 | C |
| 6 | 4.0 | 7.0 | 6.0 | B |
For draw 1, suppose A has \(p_{R,A}=0.70\), \(\mu_{Q,A}=2.0\) points, and \(p_{T,A}=0.08\). Its utility is \(10(0.70)+1.5(2.0)-25(0.08)=7.0+3.0-2.0=8.0\) points, consistent with the table. The six draws give each arm two best outcomes, so the simulated allocation probabilities are \(2/6=0.333\) for A, \(2/6=0.333\) for B, and \(2/6=0.333\) for C. These equal estimates do not imply identical response rates or identical utility distributions; they mean that each arm was best in the same number of the displayed posterior draws.
Prespecification, Diagnostics, and Alternatives
A utility rule should be specified before examining the outcomes that will guide allocation. At minimum, document the outcome definitions and time horizon, the direction and scale of each component, the weights or utility values assigned to outcome combinations, and the population over which expected utility is defined. Clinical input can help determine the trade-offs, but the rule should be fixed and interpretable rather than adjusted during the trial to favor an arm whose observed results look promising.
A practical diagnostic is to inspect the utility calculation for clinically plausible outcome combinations. Confirm that worse toxicity cannot accidentally increase utility, that outcome coding follows the stated definition, and that the units and weights produce the intended trade-offs. Check posterior utility draws for impossible values or implausible patterns, and verify that each allocation probability is calculated from the same utility rule and the appropriate joint posterior draws. A mismatch between the recorded outcome coding and the prespecified rule can reverse treatment rankings.
Utility-based Thompson sampling is not the only possible decision rule. Selecting the arm with the highest posterior mean utility targets a different summary and typically does not randomize in proportion to uncertainty about which arm is best. Allocating by response probability ignores other outcomes unless the utility is defined to depend on response alone. A randomized allocation that does not adapt to posterior evidence is another alternative, but it does not use posterior best probabilities to change assignment. Each rule answers a different decision question; none makes a poorly chosen utility function clinically appropriate.
The main limitation is that posterior probabilities can be computed precisely for a decision rule that is itself debatable. A different but defensible utility function may change the ranking and allocation probabilities. Prespecification makes the decision rule transparent; it does not remove the need to justify its values. Likewise, posterior best probabilities depend on the outcome model and its assumptions. Poor model fit, inadequate representation of dependence between outcomes, or unaccounted-for missing outcomes can make the resulting probabilities misleading.
Worked Example: Utility Weights Change the Decision
Consider two treatments with posterior draws of response benefit \(R_a\) and toxicity burden \(T_a\), both expressed in hypothetical utility points. Use \(U_a=R_a-\lambda T_a\), where larger utility is better and \(\lambda\) is the prespecified penalty per toxicity point. The four paired draws below are hypothetical.
| Draw | A: \(R,T\) | B: \(R,T\) | Best at \(\lambda=0.5\) | Best at \(\lambda=3\) |
|---|---|---|---|---|
| 1 | (10, 2) | (8, 0) | A | B |
| 2 | (9, 0) | (7, 1) | A | A |
| 3 | (10, 2) | (9, 1) | A | B |
| 4 | (8, 1) | (7, 2) | A | A |
At \(\lambda=0.5\), draw 1 gives A utility \(10-0.5(2)=9\) and B utility \(8-0.5(0)=8\), so A is best; A is best in all four draws and the simulated probabilities are 1.00 for A and 0 for B. At \(\lambda=3\), draw 1 gives A utility \(10-3(2)=4\) and B utility \(8-3(0)=8\), so B is best. B is best in draws 1 and 3, while A is best in draws 2 and 4; the estimates are 0.50 for each arm. This comparison shows why the penalty is part of the decision definition. It is a sensitivity analysis, not a justification for selecting a weight after seeing which one favors a preferred treatment.
Common Mistakes
- Equating best response with best treatment. The wrong reasoning is that the arm with the largest response probability must be best. The correction is to calculate the prespecified utility from all relevant outcomes and compare utilities.
- Using the largest posterior mean as the Thompson probability. A posterior mean is a summary, not the probability of being best. Estimate the latter by the fraction of joint posterior draws in which each arm has the largest utility.
- Changing weights after seeing results. This changes the decision rule in response to trial outcomes. Prespecify the rule; use sensitivity analyses to show how alternative assumptions would affect conclusions, not to quietly redefine the primary allocation objective.
- Ignoring posterior dependence. Combining unrelated marginal draws can misrepresent the probability that one arm exceeds another. Use joint posterior draws that reflect the fitted model.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Why might the arm with the highest posterior response probability not have the highest posterior probability of maximizing clinical utility?
- In 1,000 posterior draws, arm A is best in 430 draws and arm B is best in 570 draws, with no ties. What are the Monte Carlo allocation probabilities?
- Suppose each arm’s utility is multiplied by the same positive constant. Does this change which arm is best in any draw, and why?
- Why must the treatment utilities be compared using joint posterior draws when their posterior quantities are dependent?
Solutions. (1) Utility may also include toxicity, symptom improvement, or other prespecified outcomes, so ranking by response alone can differ from ranking by utility. (2) The estimates are \(430/1000=0.430\) for A and \(570/1000=0.570\) for B; they sum to 1. (3) No. If \(c>0\), then \(U_a>U_b\) exactly when \(cU_a>cU_b\), so the within-draw ranking and best-arm probabilities are unchanged. (4) The probability that one utility exceeds another depends on their joint distribution. Independently pairing dependent marginal draws can alter the frequency of which arm is best and therefore distort the allocation probabilities.