Deriving the Normal-Normal Posterior for Allocation

Why Derive the Posterior Update?

With normal outcomes, a normal prior gives a normal posterior for a treatment’s unknown response mean when the observation variance is known. The update is useful in Thompson sampling because it turns each treatment’s accumulated outcomes into a distribution from which its mean can be drawn. The draw, rather than just the current posterior mean, allows uncertainty to affect allocation.

The result depends on a specific model: outcomes within an arm are independent and normally distributed around that arm’s mean, the observation variance is known, and the prior for the mean is normal. These assumptions distinguish the calculation from an update that estimates the outcome variance as well. Here, the posterior variance describes uncertainty about the unknown mean; it is not the variance of an individual response.

Deriving the Normal-Normal Posterior

The likelihood, as a function of \(\mu\), is proportional to \(\exp[-\sum_{i=1}^{n}(x_i-\mu)^2/(2\sigma^2)]\). The prior density is proportional to \(\exp[-(\mu-\mu_0)^2/(2\tau_0^2)]\). Multiplying them gives the posterior density up to a factor that does not depend on \(\mu\). To identify its distribution, collect the terms involving \(\mu\) in the exponent.

Since \(\sum_{i=1}^{n}(x_i-\mu)^2=\sum_{i=1}^{n}(x_i-\bar{x})^2+n(\bar{x}-\mu)^2\), the first term on the right does not depend on \(\mu\). The remaining quadratic terms in the log posterior are

Let \(V_n\) be the reciprocal of the coefficient of \(\mu^2\) inside the brackets. Completing the square identifies the posterior mean as \(V_n\) times the coefficient multiplying \(\mu\) in the linear term. Therefore,

This derivation also gives a useful interpretation: posterior precision is prior precision plus data precision. The posterior mean is a precision-weighted average of the prior mean and the sample mean:

The weight on \(\bar{x}\) grows with the number of observations and shrinks as the known observation variance increases. The prior contribution does not disappear by assertion; it becomes relatively smaller as the data precision \(n/\sigma^2\) increases. When \(n=0\), the formulas return the prior mean and variance.

Using Posterior Draws for Allocation

For each treatment arm \(j\), let its mean response be \(\mu_j\), its known within-arm variance be \(\sigma_j^2\), and its prior be \(\mu_j\sim N(\mu_{0j},\tau_{0j}^2)\). Given \(n_j\) responses with sample mean \(\bar{x}_j\), apply the preceding formulas using that arm’s quantities. If the arm priors and response models are independent across arms, the resulting posterior distributions are independent conditional on the observed trial data.

When larger responses are preferred, a Thompson allocation step draws one value from each treatment’s posterior and assigns the next participant to the arm with the largest draw. The draw is a temporary representation of uncertainty about that arm’s mean, not a new observed response. With two arms, the probability that an assignment goes to arm A is the posterior probability \(P(\mu_A>\mu_B\mid\text{data})\), provided ties have probability zero. In practice, repeated independent posterior draws can estimate this probability by the fraction of draws for which arm A has the largest value.

Sequential Updating and a Useful Check

The same formula can be applied after each response or to all responses accumulated so far. For a fixed known \(\sigma^2\), updating sequentially produces the same posterior as using the full sample: each new independent observation adds \(1/\sigma^2\) to precision and \(x_i/\sigma^2\) to the precision-weighted mean numerator. This provides a practical check on an implementation.

Assumptions, Diagnostics, and Alternatives

The update is exact under the stated model, but that does not make the model automatically suitable for a clinical outcome. Check that the response scale is meaningfully represented by a normal distribution, that observations are conditionally independent within each arm, and that the known \(\sigma^2\) is defensible. A normal likelihood can be a poor description for highly skewed, bounded, binary, or count outcomes. The known-variance condition is substantive: replacing \(\sigma^2\) by a sample variance does not give this same normal posterior in general.

For implementation, check that posterior variances are positive, that \(V_n<\tau_0^2\) whenever \(n>0\), and that the posterior mean lies between \(\mu_0\) and \(\bar{x}\). The last check follows because both receive positive precision weights. For a simulated allocation, verify that each draw uses the current posterior parameters and that the correct direction of preference is used. If smaller outcomes are better, select the smallest draw, or define a utility with larger values representing better outcomes.

The posterior mean alone gives a plug-in greedy rule: choose the arm with the largest \(m_j\). That rule ignores uncertainty in the means and is not posterior sampling. Thompson sampling instead draws from the posterior, allowing a less certain arm to be selected on some assignments. A diffuse or flat-prior analysis is another possible reference, but it does not encode the same prior information; a normal prior with very large variance can approximate a flat prior over a relevant range, while still requiring care about whether the approximation is appropriate.

This conjugate calculation is simple and transparent, but its convenience is also a limitation. It relies on a known variance, a normal sampling model, and (for separate arm-by-arm updates) independent arm priors. It does not account for shared parameters or dependence between treatment means. If those assumptions are not credible, the resulting posterior draws may misrepresent uncertainty and therefore affect allocation in the wrong direction.