Symmetry of Allocation Under Identical Arms

Why identical arms should be treated symmetrically

Thompson sampling assigns patients using posterior uncertainty about which treatment is best. If the treatment arms have the same prior status and the observed outcomes provide the same evidence for each, no arm has a statistical advantage. The posterior probabilities that the arms are best should therefore be equal. This conclusion follows from symmetry, rather than from a numerical approximation to each probability.

The result depends on a precise meaning of “the same.” The prior must be invariant to relabeling the arms, and the likelihood contributed by the observed data must also be invariant to those relabelings. Equal prior means alone are not enough: priors can have the same marginal means but different joint distributions. Likewise, identical prior distributions do not guarantee equal posterior probabilities after different outcomes have been observed.

Exchangeability is stronger than saying each arm has the same marginal prior distribution. For example, two response rates could each have a uniform marginal prior while being dependent in a way that favors one arm being larger than the other. Such a joint prior is not exchangeable unless swapping the arms leaves their joint distribution unchanged. Independent, identically distributed priors are exchangeable, but independence is not required.

When the posterior preserves symmetry

Let \(D_i\) denote the outcomes observed on arm \(i\), and suppose the response model is the same for all arms. If the arms have identical observed data in the sense that their likelihood contributions have the same form, the likelihood does not favor any label. Under a Bernoulli response model, for example, this condition holds when every arm has the same number of successes and failures. If arm \(i\) has \(s_i\) successes and \(f_i\) failures, its likelihood contribution is proportional to \(\theta_i^{s_i}(1-\theta_i)^{f_i}\); identical values of \(s_i\) and \(f_i\) make these contributions identical across arms.

Assume that the assignment mechanism is known and its probabilities, conditional on the observed trial history, do not depend on the unknown response rates. Then assignment probabilities do not supply arm-specific response-rate information in the posterior. Also assume the response model is correctly specified and does not use arm-specific parameters or covariates that treat labels differently. Under these conditions, an exchangeable prior and identical arm-level likelihoods produce an exchangeable posterior.

The first conclusion is about the full joint posterior, including any dependence among response rates. The second conclusion additionally requires that ties for the largest response rate have posterior probability zero. Continuous posterior distributions, such as ordinary Beta posteriors, satisfy this condition when their joint distribution assigns zero probability to equality between two rates.

Why equal posterior best-arm probabilities follow

For each arm \(i\), define the event \(A_i=\{\theta_i>\theta_j\text{ for every }j\ne i\}\), that arm \(i\) is the unique best arm. Exchangeability means any relabeling of the arms leaves posterior probabilities unchanged. In particular, a permutation that swaps arms \(i\) and \(j\) maps \(A_i\) to \(A_j\), so \(P(A_i\mid D)=P(A_j\mid D)\). If a unique best arm exists with probability one, the events \(A_1,\ldots,A_K\) are disjoint and their union has posterior probability one. Their equal probabilities must therefore each be \(1/K\).

This argument establishes equality without integrating a complicated posterior density or running a simulation. It is also a useful check on computation: if a simulation estimates substantially different best-arm probabilities under exact symmetry, the discrepancy needs explanation.

Ties require care. If the posterior can assign positive probability to two or more arms tying for the maximum, the strict-maximum events \(A_i\) need not cover all posterior probability. Their probabilities can still be equal under exchangeability, but they need not sum to one or equal \(1/K\). A symmetric decision rule can assign a tied maximum uniformly among the tied arms; by symmetry, the resulting probability that each arm is selected is \(1/K\). A deterministic rule that always awards ties to the lowest-numbered arm is not symmetric and can create unequal selection probabilities even when the posterior is symmetric.

A short justification using the posterior density

The invariance can also be checked directly. Write \(p(\boldsymbol{\theta})\) for the prior density and \(L(D\mid\boldsymbol{\theta})\) for the likelihood, where \(\boldsymbol{\theta}=(\theta_1,\ldots,\theta_K)\). Bayes’ rule gives a posterior density proportional to their product. For any permutation \(\pi\), an exchangeable prior satisfies \(p(\boldsymbol{\theta})=p(\boldsymbol{\theta}_{\pi})\), and a permutation-invariant likelihood satisfies \(L(D\mid\boldsymbol{\theta})=L(D\mid\boldsymbol{\theta}_{\pi})\). Their product is unchanged, and its normalizing constant is unchanged as well. Hence the posterior density is invariant to the same permutation.

This argument also clarifies why the two assumptions are both needed. If either the prior or the likelihood changes under relabeling, the posterior product need not be invariant. Equal prior marginals cannot repair an asymmetric likelihood, and identical observed outcomes cannot erase an asymmetric prior.

Ways symmetry can fail—and how to diagnose it

A useful diagnostic is to check the entire inferential setup, not just whether the arms have similar posterior means. Symmetry can be broken by unequal prior distributions or a non-exchangeable joint prior; different success and failure counts; arm-specific response models; unequal handling of missing outcomes; or a decision rule that treats labels differently. Different sample sizes with the same observed response proportions generally do not give identical likelihoods: more observations typically contribute more information and can change the posterior best-arm probabilities.

The following comparison helps separate exact symmetry from situations that merely look similar:

For simulation, draw from the joint posterior and count which arm has the largest draw. Under exact symmetry and continuous posteriors, the simulation estimates should be close to \(1/K\), but they need not be exactly equal in a finite run. Increasing the number of draws reduces Monte Carlo error; it does not change the exact result. A persistent, material discrepancy suggests checking that every arm uses the same prior and update, that all outcomes are included and coded consistently, and that ties are handled symmetrically.

A related mistake is to infer symmetry from equal posterior means. Two distributions can have the same means but different variances, shapes, or dependence with other arms. Best-arm probabilities depend on the joint distribution and on the probability of exceeding every competing arm, not just on a comparison of means.

Common mistake and practical limits

Wrong reasoning: “The arms started with the same prior, so their posterior probabilities of being best must remain equal.” This ignores the information in the observed outcomes. In the two-arm example with 2 successes for one arm and 0 for the other, the posterior best-arm probability is 0.950 versus 0.050, not 0.500 each. Correction: establish symmetry for both the prior and the observed-data likelihood before applying the equal-probability result.

The symmetry result is exact but conditional on its assumptions. It does not establish that the response model is scientifically appropriate, that outcomes are comparable across arms, or that a response rate fully represents clinical benefit. It also concerns posterior probability of being best—not a general guarantee about future outcomes or the value of a treatment decision. When the arms are not exchangeable, compute or estimate their probabilities from the actual posterior rather than forcing them to be equal.