Deriving Beta-Binomial Updates From Likelihood Kernels

Why Derive the Update From the Kernel?

The earlier material on Beta-Binomial Posterior Updates gives the familiar rule for updating a response-rate prior. Here the focus is on why that rule follows from the model, and on the distinction between a likelihood kernel and a fully normalized probability. That distinction matters in clinical-trial calculations: constants can be dropped when identifying a posterior shape, but not when calculating the probability of the observed data.

Suppose a trial observes a binary endpoint, such as whether each participant has a response by a specified assessment time. Let \(\theta\) be the common response probability, \(n\) the number of participants with observed outcomes, and \(Y\) the number who respond. The conjugate update follows by multiplying the prior density for \(\theta\) by the probability of the observed data conditional on \(\theta\).

Model and assumptions: For a fixed \(n\), the outcomes are conditionally independent given one common response probability \(\theta\), and each outcome is Bernoulli with probability \(\theta\). Thus \(Y\mid\theta\sim\operatorname{Binomial}(n,\theta)\). The prior is \(\theta\sim\operatorname{Beta}(a,b)\), where \(a>0\) and \(b>0\). Outcomes are completely observed, and the model does not include covariates or participant-specific response probabilities.

The assumptions are substantive. Conditional independence and a common \(\theta\) mean the model does not account for clustering, changing response probabilities across participants, or extra variation beyond the binomial model. The fixed-\(n\) formulation also treats the number of assessed participants as given. If these assumptions are not credible, the algebra below can still be performed, but it will update the wrong model.

Prior Density, Likelihood, and Kernel

The beta function is the integral that normalizes powers of a probability and its complement:

\mathrm{B}(a,b)=\int_0^1 \theta^{a-1}(1-\theta)^{b-1}\,d\theta =\frac{\Gamma(a)\Gamma(b)}{\Gamma(a+b)}.

Accordingly, the Beta\((a,b)\) prior density is

\pi(\theta)=\frac{\theta^{a-1}(1-\theta)^{b-1}}{\mathrm{B}(a,b)},\qquad 0<\theta<1.

For the observed count \(Y=y\), the binomial probability mass function is

P(Y=y\mid\theta)=\binom{n}{y}\theta^y(1-\theta)^{n-y},\qquad y=0,1,\ldots,n.

The likelihood is this probability viewed as a function of \(\theta\), with the observed \(n\) and \(y\) held fixed. The binomial coefficient does not involve \(\theta\). Therefore the likelihood kernel—the part that determines its shape as a function of \(\theta\)—is proportional to \(\theta^y(1-\theta)^{n-y}\).

Definition: A kernel in \(\theta\) is an expression proportional to a function of \(\theta\), where the omitted factor is constant with respect to \(\theta\). A likelihood kernel is sufficient for identifying the posterior shape, but omitted factors must be restored when evaluating a probability or marginal likelihood.

Deriving the Posterior and Its Normalizer

Bayes’ rule says that the posterior density is proportional to the prior density multiplied by the likelihood. Substituting the two expressions and collecting powers gives

\begin{aligned} \pi(\theta\mid Y=y) &\propto \pi(\theta)P(Y=y\mid\theta)\\ &\propto \theta^{a-1}(1-\theta)^{b-1} \theta^y(1-\theta)^{n-y}\\ &= \theta^{a+y-1}(1-\theta)^{b+n-y-1}. \end{aligned}

This is the kernel of a Beta density. Its exponents must be one less than the two beta shape parameters, so the updated parameters are \(a+y\) and \(b+n-y\). The remaining question is the normalizing constant: the density must integrate to one over \(0<\theta<1\).

Result: Under the stated model and prior,

\theta\mid Y=y\sim\operatorname{Beta}(a+y,b+n-y), \qquad \pi(\theta\mid Y=y)= \frac{\theta^{a+y-1}(1-\theta)^{b+n-y-1}} {\mathrm{B}(a+y,b+n-y)}.

The normalization can be verified directly. By the definition of the beta function, the integral of the unnormalized posterior kernel is \(\mathrm{B}(a+y,b+n-y)\). Dividing the kernel by that integral therefore produces a density with total probability one.

\int_0^1 \frac{\theta^{a+y-1}(1-\theta)^{b+n-y-1}} {\mathrm{B}(a+y,b+n-y)}\,d\theta=1.

The same calculation identifies the marginal probability of the count. Integrating the likelihood over the prior gives

\begin{aligned} P(Y=y) &=\int_0^1 P(Y=y\mid\theta)\pi(\theta)\,d\theta\\ &=\binom{n}{y} \frac{\mathrm{B}(a+y,b+n-y)}{\mathrm{B}(a,b)}. \end{aligned}

This is the beta-binomial probability mass for the observed count. It includes the binomial coefficient because \(Y=y\) denotes the count, which can arise from multiple participant-level outcome sequences. The posterior update itself is unchanged if a factor independent of \(\theta\) is omitted from the likelihood; the marginal probability is not.

Worked Example: A Count With a Beta(2,3) Prior

Hypothetical trial data: Among \(n=10\) participants with observed binary outcomes, \(y=7\) respond. Suppose the response probability has a Beta\((2,3)\) prior. The model treats the ten outcomes as conditionally independent Bernoulli observations with one common \(\theta\).

The prior density is \(\pi(\theta)=\theta(1-\theta)^2/\mathrm{B}(2,3)\), and \(\mathrm{B}(2,3)=1/12\). The likelihood for the count is

P(Y=7\mid\theta)=\binom{10}{7}\theta^7(1-\theta)^3 =120\theta^7(1-\theta)^3.

Multiplication gives a posterior kernel proportional to \(\theta^{1+7}(1-\theta)^{2+3}=\theta^8(1-\theta)^5\). Thus the posterior is Beta\((9,6)\), with density \(18018\theta^8(1-\theta)^5\), since \(\mathrm{B}(9,6)=1/18018\).

The marginal probability of observing a count of seven is

\begin{aligned} P(Y=7) &=120\frac{\mathrm{B}(9,6)}{\mathrm{B}(2,3)} =120\frac{1/18018}{1/12}\\ &=\frac{240}{3003} =\frac{80}{1001}\approx 0.07992. \end{aligned}

The calculation uses the fully specified count likelihood, including \(\binom{10}{7}=120\). The posterior shape, by contrast, could be identified from the kernel alone.

Ordered Outcomes and Aggregate Counts

The likelihood kernel depends on how the data are represented only through the number of successes and failures. If the data are recorded as a particular ordered sequence, the likelihood is \(\theta^y(1-\theta)^{n-y}\). If the data are recorded as the aggregate count \(Y=y\), the likelihood includes \(\binom{n}{y}\). That coefficient counts the distinct sequences that yield the same count. Since it does not depend on \(\theta\), it cancels from the normalized posterior density, but it remains part of the probability of the count.

Worked Example: One Sequence Versus a Count

Hypothetical trial data: Six participants have the ordered outcomes response, response, no response, response, no response, response. Thus \(n=6\) and \(y=4\). Use a Beta\((3,2)\) prior. This model assumes the six outcomes are conditionally independent with the same response probability.

For this particular sequence, the likelihood is \(\theta^4(1-\theta)^2\). Multiplying by the prior kernel \(\theta^2(1-\theta)\) gives \(\theta^6(1-\theta)^3\), hence a Beta\((7,4)\) posterior. For the count \(Y=4\), the likelihood is instead \(15\theta^4(1-\theta)^2\), since \(\binom{6}{4}=15\); the posterior is still Beta\((7,4)\).

Here \(\mathrm{B}(7,4)=1/840\) and \(\mathrm{B}(3,2)=1/12\). The probability of this specified sequence, averaged over the prior, is

\frac{\mathrm{B}(7,4)}{\mathrm{B}(3,2)} =\frac{1/840}{1/12} =\frac{1}{70}.

The probability of the count is 15 times as large:

P(Y=4)=15\cdot\frac{1}{70}=\frac{3}{14}\approx 0.2143.

Thus the posterior is identical under the two representations, while the marginal probability differs because the count represents 15 possible sequences.

Using the Normalizer as a Check

The prior and posterior normalizers provide useful algebra checks. In any calculation, the prior shape parameters must be positive, and the observed count must satisfy \(0\le y\le n\). The posterior parameters \(a+y\) and \(b+n-y\) must therefore be positive. The posterior kernel must also integrate to the beta function with those parameters—not the beta function with the prior parameters.

Worked Example: Uniform Prior and a Predictive Count Probability

Hypothetical trial data: Suppose three responses are observed among eight participants, with independent binary outcomes conditional on a common response probability. Use a Beta\((1,1)\) prior, whose density is uniform on \((0,1)\).

The likelihood kernel is \(\theta^3(1-\theta)^5\), so the posterior is Beta\((4,6)\). Since \(\mathrm{B}(4,6)=1/504\), its normalized density is \(504\theta^3(1-\theta)^5\).

The prior-predictive probability of the count is

P(Y=3)=\binom{8}{3}\frac{\mathrm{B}(4,6)}{\mathrm{B}(1,1)} =56\cdot\frac{1/504}{1} =\frac{1}{9}\approx 0.1111.

This probability is for the count of three responses, not for one specified arrangement of three responses among eight participants. As a check, under a Beta\((1,1)\) prior the marginal probabilities of the nine possible counts \(0,\ldots,8\) are all \(1/9\); they sum to one.

Common Errors, Diagnostics, and Alternatives

A frequent mistake is to treat the likelihood kernel as though it were the full probability of the observed count. For posterior shape this is harmless, because the omitted binomial coefficient is constant in \(\theta\). For the marginal probability of the count it is wrong: leaving out \(\binom{n}{y}\) gives the prior-averaged probability of one specified ordering, not of all orderings that produce \(y\).

Another error is to add \(y\) to the second prior parameter or \(n-y\) to the first. The exponent check prevents this: the prior contributes powers \(a-1\) and \(b-1\), while the likelihood contributes \(y\) and \(n-y\). The resulting exponents are \(a+y-1\) and \(b+n-y-1\), respectively.

The update is exact only for the stated likelihood and prior. Warning signs include outcomes clustered by site or clinician, marked participant heterogeneity, unmodeled changes in the response definition or assessment time, and missing outcomes handled as though they were observed failures. Such features can undermine the common-probability binomial assumption. Prior predictive checks can help assess whether a chosen prior and sampling model generate plausible outcomes, but they do not establish that the likelihood assumptions are true.

For this conjugate model, multiplying kernels and normalizing with the beta function gives a closed-form answer. Numerical integration on a grid is an alternative when the model is still one-dimensional but the prior or likelihood does not yield a recognized density; it requires checking grid resolution and numerical normalization. More general computational methods may be needed for more complicated models. The closed-form result is convenient, but it does not repair a poor model, and the posterior is conditional on the assumed prior and likelihood.

Key takeaway: Multiplying the Beta prior kernel by the binomial likelihood kernel adds successes to the first shape parameter and failures to the second. The beta function supplies the posterior normalizer; the binomial coefficient is unnecessary for the posterior shape but essential for the marginal probability of an aggregate count.

Check Your Understanding

Answer each question, then compare with the solutions below.

With a Beta\((2,4)\) prior and \(n=9\), \(y=5\), what is the posterior distribution?
For \(n=5\), \(y=2\), what factor distinguishes the likelihood of the aggregate count from the likelihood of one specified sequence?
With a Beta\((1,1)\) prior and \(n=3\), calculate the marginal probability of \(Y=0\).
Why can the binomial coefficient be omitted when deriving the posterior kernel but not when calculating \(P(Y=y)\)?

Solutions. (1) The likelihood contributes \(\theta^5(1-\theta)^4\), so the posterior is Beta\((2+5,4+9-5)=\operatorname{Beta}(7,8)\). (2) The factor is \(\binom{5}{2}=10\), the number of sequences with two responses. (3) The formula gives \(P(Y=0)=\binom{3}{0}\mathrm{B}(1,4)/\mathrm{B}(1,1)=1\cdot(1/4)/1=0.25\). (4) The coefficient is constant in \(\theta\), so it cancels when the posterior density is normalized; it counts the possible sequences represented by the aggregate count, so it is required for that count’s probability.