Two Different Questions for Adaptive Allocation
An adaptive allocation rule needs to specify what it is trying to favor. One possibility is to favor the treatment that has the greatest posterior probability of having the best response parameter. Another is to favor treatments with a high posterior probability of exceeding a prespecified clinical threshold. These questions can lead to different assignment probabilities, even when they use the same posterior distribution.
The distinction matters because “best among the available arms” is a relative claim, whereas “clinically worthwhile” is an absolute claim defined against a threshold. An arm can be most likely to be best while still having little chance of reaching a clinically meaningful target. Conversely, several arms may have a good chance of exceeding the target, even though only one can be best.
The parameter and threshold must be on the same scale. For example, if \(\theta_j\) is a treatment effect measured in points on a symptom scale, then \(\delta\) must also be a treatment-effect threshold in points. If \(\theta_j\) is a response rate, the threshold is a rate. The threshold concerns an arm-level parameter, not whether a particular future patient will respond.
From Posterior Probabilities to Allocation Weights
In Thompson sampling, the best-arm rule can be implemented by drawing one value of each arm’s parameter from its current posterior distribution and assigning to the arm with the largest draw. The probability that arm \(j\) wins this posterior draw is \(b_j\); these probabilities sum to one when ties have probability zero. Thus, \(b_j\) directly supplies the assignment probability under best-arm Thompson sampling.
Threshold-exceedance probabilities do not generally sum to one. A threshold-based allocation policy must therefore specify how the \(q_j\) values translate into assignment. One possible rule is to normalize them:
This is a threshold-weighted allocation rule, not a probability distribution over mutually exclusive threshold events. It is a policy choice: another trial could use the probabilities to screen arms against a cutoff, or to inform a separate clinical-utility rule. If every \(q_j=0\), normalization is undefined and a prespecified fallback rule is needed. Neither the threshold nor the allocation rule should be selected after inspecting the current posterior without accounting for that choice.
For two independent normal posteriors, the best-arm probability has a particularly direct calculation. If \(\theta_A\sim N(m_A,s_A^2)\) and \(\theta_B\sim N(m_B,s_B^2)\), then \(\theta_A-\theta_B\sim N(m_A-m_B,s_A^2+s_B^2)\). Consequently, \(b_A=P(\theta_A>\theta_B\mid D)\) is the standard normal probability corresponding to \((m_A-m_B)/\sqrt{s_A^2+s_B^2}\). In contrast, \(q_A\) is obtained by standardizing \(m_A-\delta\) by \(s_A\). The formulas answer different comparisons.
Worked Example: The Best Arm and the Threshold Rule Give Different Weights
Suppose a hypothetical trial estimates each arm’s mean improvement over control, in symptom-scale points. The clinically meaningful threshold is \(\delta=0.5\) points. Assume the current posterior distributions are independent, with \(\theta_A\sim N(1.1,1^2)\) and \(\theta_B\sim N(0.3,1^2)\). These distributions are the model for this calculation; the normal posterior and independence are assumptions, not conclusions from the data.
First calculate threshold-exceedance probabilities. Using \(\Phi\) for the standard normal cumulative distribution function,
and
If the chosen policy normalizes these probabilities, their sum is \(0.7257+0.4207=1.1464\), so
Now calculate the best-arm probability. The difference \(\theta_A-\theta_B\) has mean \(1.1-0.3=0.8\) points and variance \(1^2+1^2=2\) square points. Therefore,
Both rules favor A, but they assign different probabilities: about 0.7140 under best-arm Thompson sampling and 0.6330 under this normalized threshold rule. The threshold probabilities themselves are not assignment probabilities until the chosen policy converts them.
A Threshold Probability Is Not a Predictive Response Probability
The threshold probability is a statement about uncertainty in an arm-level parameter. If \(\theta_j\) is the response rate, \(P(\theta_j>\delta\mid D)\) is the posterior probability that the rate exceeds \(\delta\). A predictive probability instead asks whether a future participant assigned to arm \(j\) will respond. That predictive quantity averages over uncertainty in \(\theta_j\), but it is not the same event as \(\theta_j>\delta\). The distinction between best-arm probability and predictive response probability is important here: neither should be relabeled as the other.
The posterior model must also match the response type and trial design. For binary responses, one might use a Bernoulli likelihood and a posterior for the arm response rate. For a continuous response, the parameter and posterior model should correspond to the clinically relevant scale. If the posterior distributions are correlated, best-arm probabilities must use the joint posterior; independent marginal calculations can be wrong. Threshold-exceedance probabilities can still be obtained from each marginal posterior, but the full allocation policy may depend on joint information.
Worked Example: Threshold Weights From Binary Responses
Consider a hypothetical binary-response trial with a uniform \(\operatorname{Beta}(1,1)\) prior for each arm’s response rate \(p_j\). After 7 responses and 3 nonresponses on A, and 4 responses and 4 nonresponses on B, the independent posteriors are \(p_A\mid D\sim\operatorname{Beta}(8,4)\) and \(p_B\mid D\sim\operatorname{Beta}(5,5)\). Suppose the clinical target is a response rate above \(0.5\).
For integer beta parameters, the beta-binomial identity gives \(P(p_A>0.5)=P\{Y\leq 7\}\), where \(Y\sim\operatorname{Binomial}(11,0.5)\). Thus
The \(\operatorname{Beta}(5,5)\) posterior is symmetric about \(0.5\), and has no point mass at the threshold, so \(q_B=P(p_B>0.5)=0.5000\). Under normalized threshold weighting,
These calculations establish how plausible it is that each arm’s response rate exceeds 0.5. They do not calculate the probability that A’s response rate exceeds B’s. That relative probability requires comparing the two posterior distributions, not checking each one against the clinical target.
When the Rules Favor Different Arms
A treatment with a high posterior mean is not necessarily the one most likely to exceed a threshold, nor the one most likely to be best. Posterior uncertainty affects each comparison differently. A diffuse posterior can leave substantial probability both above and below a threshold, while a more precise but slightly lower posterior may have a larger probability of clearing that threshold. Best-arm probability also depends on the other arms: the same posterior for A can yield a different \(b_A\) when its competitors change.
Worked Example: A More Uncertain Arm Can Be More Likely to Be Best
Suppose a hypothetical two-arm study measures improvement in points and uses \(\delta=0.5\) points. Let the independent posteriors be \(\theta_A\sim N(0.8,1.5^2)\) and \(\theta_B\sim N(0.6,0.2^2)\). These are stipulated posterior summaries for illustration.
For the threshold comparison,
The normalized threshold weights are \(\rho_A=0.5793/(0.5793+0.6915)\approx 0.4559\) and \(\rho_B=0.6915/(0.5793+0.6915)\approx 0.5441\). Thus the threshold-weighted rule favors B.
For the best-arm comparison, \(\theta_A-\theta_B\) has mean \(0.8-0.6=0.2\) points and variance \(1.5^2+0.2^2=2.29\) square points. Hence
The best-arm rule slightly favors A, while the threshold rule favors B. A’s greater uncertainty gives it a substantial chance of being the better arm, but B’s more concentrated posterior puts more probability above the target. Neither conclusion is a calculation error; each answers a different question.
Choosing, Checking, and Interpreting the Allocation Rule
The choice between the rules should follow the trial’s objective, specified before using the observed data to adapt allocation. Best-arm sampling targets relative ranking among the available treatments. Threshold-based weighting targets evidence that arms reach a clinically defined level, but normalized weights are only one possible way to translate that evidence into assignments. Clinical utility, safety constraints, and the need to retain adequate information about all arms may call for a different policy.
Several checks help detect mistakes:
- Confirm that \(\theta_j\) and \(\delta\) represent the same clinical quantity and use the same units.
- Check that every reported \(q_j\) lies between 0 and 1. Do not expect threshold probabilities to sum to one.
- For best-arm probabilities in a two-arm model with no posterior ties, check that \(b_A+b_B=1\).
- When normalizing threshold weights, verify that the denominator is positive and that the resulting weights sum to one, allowing for rounding.
- Check whether the posterior dependence structure matches the calculation. Correlated arm posteriors cannot in general be treated as independent when computing the probability of being best.
Neither rule alone guarantees a reliable final treatment comparison. Adaptive assignment changes the data collected over time, and very small allocation probabilities can leave an arm with little information. Threshold weighting can also give similar assignment weight to arms whose threshold probabilities are both small if all the other probabilities are even smaller. A design therefore needs a prespecified policy for low probabilities, safety monitoring, and the final scientific objective; those decisions are not settled by the posterior probabilities themselves.
Common mistake: “The probability that A exceeds the clinical threshold is 0.73, so A has a 73% probability of being the best arm.” This is incorrect: threshold exceedance compares A with a fixed target, while best-arm probability compares A with every competing arm. Calculate \(P(\theta_A>\delta\mid D)\) for the first question and \(P(\theta_A=\max_k\theta_k\mid D)\) for the second.
Check Your Understanding
Answer each question, then compare with the solutions.
- In one sentence, distinguish \(P(\theta_j=\max_k\theta_k\mid D)\) from \(P(\theta_j>\delta\mid D)\).
- Two arms have threshold probabilities \(0.80\) and \(0.40\). What are their normalized threshold weights, and do the original probabilities represent mutually exclusive events?
- For independent posteriors \(\theta_A\sim N(2,1)\) and \(\theta_B\sim N(1,1)\), with threshold \(\delta=1.5\), calculate \(q_A\) and \(b_A\).
- Why can a threshold-weighted rule favor B while a best-arm rule favors A?
Solutions. (1) The first is the posterior probability that arm \(j\) has the largest parameter among the arms; the second is the posterior probability that its parameter exceeds a fixed clinical target. (2) The sum is \(1.20\), so the normalized weights are \(0.80/1.20=0.6667\) and \(0.40/1.20=0.3333\). The original events are not mutually exclusive: both arms can exceed the threshold. (3) \(q_A=\Phi((2-1.5)/1)=\Phi(0.5)\approx0.6915\). Since \(\theta_A-\theta_B\sim N(1,2)\), \(b_A=\Phi(1/\sqrt{2})=\Phi(0.7071)\approx0.7602\). (4) The rules compare different things: an arm can have a higher chance of exceeding the fixed target, while a competitor has a lower chance of exceeding it but is less likely to surpass the other arm in a posterior comparison.