Beta-Bernoulli Updating for Binary Outcomes

Why Binary Outcomes Lead to Beta Posteriors

In a trial with a binary response, each assigned participant contributes either a success or a failure for the treatment received. Thompson sampling represents uncertainty about a treatment’s response probability with a probability distribution. For binary outcomes, the Beta distribution is a convenient choice: after observing responses, the updated distribution remains Beta, with its parameters increased by the observed counts.

The key is to keep the outcome model distinct from the assignment mechanism. The response model describes the chance of success under each treatment. The assignment mechanism describes how the next treatment is selected from the trial history. Under the conditions below, the adaptive assignment rule affects which responses are observed, but its probabilities do not add terms involving the response-rate parameters to the posterior update.

The parameters \(a_j\) and \(b_j\) are positive shape parameters, not necessarily literal counts of prior patients. The prior mean is \(a_j/(a_j+b_j)\). Larger \(a_j+b_j\), with the mean held fixed, generally corresponds to a more concentrated prior. A uniform prior is \(\operatorname{Beta}(1,1)\).

Model and Assumptions

For each arm \(j\), assume a stable response probability \(\theta_j\) during the period being modelled. Conditional on the assigned arm and its response probability, each observed response is Bernoulli: it is 1 with probability \(\theta_j\) and 0 with probability \(1-\theta_j\). The simple model treats responses as conditionally independent given the arm probabilities. Before seeing trial responses, assign a Beta prior to each \(\theta_j\); the usual arm-by-arm update assumes these priors are independent across arms.

In an adaptive trial, assignment at a given step may depend on earlier assignments and responses. For this update, the assignment rule must be known or conditioned upon, and its randomization probabilities must not themselves contain unknown response parameters in a way that supplies additional information about them. The response model also assumes that the recorded binary outcome is defined consistently and observed as modelled. If response rates change over time, outcomes are clustered, or missingness depends on unobserved response, a single independent Bernoulli model may be inadequate.

Deriving the Update

Consider the responses observed on arm \(j\). For each success, the likelihood contributes a factor \(\theta_j\); for each failure, it contributes \(1-\theta_j\). Consequently, if there are \(S_j\) successes and \(F_j\) failures, the likelihood as a function of \(\theta_j\) is proportional to \(\theta_j^{S_j}(1-\theta_j)^{F_j}\). Multiplying by the Beta prior gives the posterior density up to a normalizing constant.

This has the form of a Beta density. Its normalizing constant is the reciprocal of the Beta function \(B(a_j+S_j,b_j+F_j)\), where \(B(u,v)=\int_0^1 t^{u-1}(1-t)^{v-1}\,dt\). Thus the normalized posterior is \(\operatorname{Beta}(a_j+S_j,b_j+F_j)\). The argument also shows why only the counts matter for this model: the order of the successes and failures does not change the likelihood once the total counts are fixed.

With independent priors across arms and separate Bernoulli response models, the joint posterior factors into one updated Beta distribution per arm. An adaptive assignment schedule does not change the count update: conditional on the observed assignment history, the probability of the realized assignments is determined by the randomization rule and observed history, not by \(\theta_j\). It therefore cancels from the posterior density for the response probabilities. This does not mean the assignment history is irrelevant to the trial; it determines which arm’s outcomes were observed and hence which counts enter each update.

Updating Arm by Arm in an Adaptive Trial

Counts must be attached to the treatment actually received. A response observed after assignment to arm A updates \(\theta_A\), not the response probability of another arm. Repeated updates can be done one response at a time or all at once: adding a success increments the first shape parameter by one, and adding a failure increments the second. Either route gives the same final posterior if the same responses are included.

The same count update applies when observations arrive sequentially. If the current posterior for an arm is \(\operatorname{Beta}(a^\star,b^\star)\), then a new success gives \(\operatorname{Beta}(a^\star+1,b^\star)\), and a new failure gives \(\operatorname{Beta}(a^\star,b^\star+1)\). These are conditional posterior updates given the data so far. They do not require treating the adaptive allocation as if it were fixed in advance.

Why the Update Matters, and What to Check

The posterior distribution retains uncertainty about each arm’s response probability rather than replacing it with a single estimate. Its mean is \( (a_j+S_j)/(a_j+b_j+S_j+F_j)\), while its full shape describes remaining uncertainty. That distribution is the relevant uncertainty summary when a later procedure uses posterior draws. This tutorial establishes the update, not a rule for comparing arms or choosing the next allocation.

Before applying the formula, verify the following:

• Outcome coding: identify which binary value is success. Reversing the coding swaps the roles of \(S_j\) and \(F_j\).
• Arm assignment: count each response only for the arm the participant actually received.
• Data completeness: an unobserved response is not automatically a failure. A missing outcome needs an explicit model or handling rule.
• Response stability: check whether calendar time, changing eligibility, or treatment modifications make one fixed \(\theta_j\) implausible.
• Dependence: repeated observations on the same participant or clustering by site can violate conditional independence.
• Prior specification: confirm both Beta shape parameters are positive and that the prior is on the intended success probability.

A useful diagnostic is to compare observed response patterns over time or across relevant participant groups with the assumed constant-rate model. Marked changes may indicate that a single arm-level probability is not adequate. The Beta-Bernoulli update itself will still produce a mathematical posterior, but a precise calculation under a poor model does not make the model appropriate.

Common Mistake and Alternatives

A frequent error is to treat the assignment probability as another Bernoulli response. For example, a participant assigned to A contributes an outcome to A’s response likelihood; the fact that A was selected with some adaptive probability is not an additional success or failure. Under the stated conditions, assignment probabilities depend on the observed history and known randomization rule, so they do not change the Beta shape parameters.

Another common error is to add successes to both parameters or to add the total sample size to the first parameter. The correct correspondence is exact: successes add to \(a_j\), failures add to \(b_j\). A related error is to use the total number enrolled on an arm as \(S_j\); the update requires separate outcome counts.

The sample proportion \(S_j/(S_j+F_j)\), when there is at least one observed response, is a frequentist point estimate of the arm’s response probability. The Beta posterior is a Bayesian distribution that also incorporates the prior and expresses uncertainty. A binomial likelihood with another prior family could be used instead, but the posterior would not generally have this simple Beta form. The Beta prior is attractive here for conjugacy and transparent updating; it does not remove the need to assess the response model or justify the prior.