Deriving The Beta-Binomial Marginal Distribution

From a Basket-Specific Probability to a Count Distribution

In a binary basket trial, the response probability for a basket may be uncertain before its participants are observed. A hierarchical model represents that uncertainty by treating the basket’s response probability as a random quantity. The count of responses then has a distribution that averages over possible values of that probability. This marginal distribution is called the beta-binomial distribution.

The distinction between a conditional and a marginal model matters. Conditional on a particular response probability, the count is binomial. Marginally, after averaging over a distribution of possible response probabilities, the count is generally more variable than a binomial count with the same average response rate. The derivation makes both the source and the size of that difference explicit.

Definition: Let \(X\) be the number of responses among \(n\) participants in one basket. Suppose the basket has response probability \(p\), with \(p\sim\operatorname{Beta}(a,b)\), where \(a>0\) and \(b>0\), and suppose \(X\mid p\sim\operatorname{Binomial}(n,p)\). The marginal distribution of \(X\), obtained by integrating out \(p\), is beta-binomial.

The beta density and conditional binomial probability mass function are

f(p\mid a,b)=\frac{p^{a-1}(1-p)^{b-1}}{B(a,b)},\quad 0<p<1, \qquad P(X=x\mid p)=\binom{n}{x}p^x(1-p)^{n-x}, \quad x=0,\ldots,n,

where \(B(a,b)\) is the beta function,

B(a,b)=\int_0^1 p^{a-1}(1-p)^{b-1}\,dp =\frac{\Gamma(a)\Gamma(b)}{\Gamma(a+b)}.

Assumptions and Marginalization

The model assumes a fixed number \(n\) of participants and a single latent response probability \(p\) for the basket. Conditional on \(p\), participant outcomes are independent Bernoulli outcomes with the same probability \(p\), so their total \(X\) is binomial. The beta distribution describes uncertainty or variation in \(p\) before observing the basket’s outcomes. The parameters \(a\) and \(b\) are fixed and positive for this marginal calculation.

Thus, conditional independence does not imply marginal independence. Once \(p\) is integrated out, outcomes in the same basket share the uncertain probability. This construction is appropriate when participants in a basket can reasonably be regarded as having a common response probability, while uncertainty about that probability is represented by the beta distribution. It does not, by itself, account for covariates, changing response probabilities over time, or dependence that remains even after conditioning on \(p\).

For any possible count \(x\), marginalization averages the conditional probability over all possible \(p\) values, weighting each value by its beta density:

P(X=x)=\int_0^1 P(X=x\mid p)f(p\mid a,b)\,dp.

Substituting the two model components gives

\begin{aligned} P(X=x) &=\int_0^1 \binom{n}{x}p^x(1-p)^{n-x} \frac{p^{a-1}(1-p)^{b-1}}{B(a,b)}\,dp\\ &=\binom{n}{x}\frac{1}{B(a,b)} \int_0^1 p^{a+x-1}(1-p)^{b+n-x-1}\,dp\\ &=\binom{n}{x}\frac{B(a+x,b+n-x)}{B(a,b)}, \qquad x=0,\ldots,n. \end{aligned}

The last equality follows from the definition of the beta function: the integrand has the beta-function form with arguments \(a+x\) and \(b+n-x\). This expression is nonnegative for every allowed \(x\). It also sums to one: summing the integral expression over \(x=0,\ldots,n\) places the finite sum of conditional binomial probabilities inside the integral, and that sum is 1 for every \(p\). The remaining integral is the integral of the beta density, also 1.

Marginal probability mass function: Under the stated assumptions,

P(X=x\mid n,a,b) =\binom{n}{x}\frac{B(a+x,b+n-x)}{B(a,b)}, \qquad x=0,1,\ldots,n.

The notation emphasizes that this is a distribution for the count after \(p\) has been integrated out. It is not the binomial probability mass function evaluated at a single estimated response probability.

Worked Example: Integrating a Binomial Probability

Suppose a hypothetical basket has \(n=4\) participants and \(p\sim\operatorname{Beta}(2,3)\). Find the marginal probability of exactly \(x=2\) responses.

The beta-binomial formula gives

P(X=2) =\binom{4}{2}\frac{B(2+2,3+4-2)}{B(2,3)} =6\frac{B(4,5)}{B(2,3)}.

Using \(B(r,s)=\Gamma(r)\Gamma(s)/\Gamma(r+s)\), the two beta-function values are

B(4,5)=\frac{3!\,4!}{8!}=\frac{1}{280}, \qquad B(2,3)=\frac{1!\,2!}{4!}=\frac{1}{12}.

Therefore,

P(X=2)=6\left(\frac{1/280}{1/12}\right) =6\left(\frac{12}{280}\right) =\frac{9}{35}\approx 0.2571.

Under this hypothetical model, the prior-marginal probability of exactly two responses among four participants is about 25.7%. It is an average over the beta-distributed values of \(p\), not the probability conditional on one particular response rate.

Mean and Variance

The moments can be derived without summing the beta-binomial probability mass function. Use the conditional moments of a binomial count and then average over \(p\). For \(X\mid p\sim\operatorname{Binomial}(n,p)\),

E(X\mid p)=np, \qquad \operatorname{Var}(X\mid p)=np(1-p).

For \(p\sim\operatorname{Beta}(a,b)\), let \(\kappa=a+b\). The beta distribution has

E(p)=\frac{a}{\kappa}, \qquad \operatorname{Var}(p)=\frac{ab}{\kappa^2(\kappa+1)}, \qquad E\{p(1-p)\}=\frac{ab}{\kappa(\kappa+1)}.

The law of total expectation gives \(E(X)=E\{E(X\mid p)\}\). The law of total variance separates variation in the count conditional on \(p\) from variation due to uncertainty in \(p\):

\begin{aligned} E(X)&=E(np)=n\frac{a}{\kappa},\\ \operatorname{Var}(X) &=E\{\operatorname{Var}(X\mid p)\} +\operatorname{Var}\{E(X\mid p)\}\\ &=n\frac{ab}{\kappa(\kappa+1)} +n^2\frac{ab}{\kappa^2(\kappa+1)}\\ &=\frac{nab(\kappa+n)}{\kappa^2(\kappa+1)}. \end{aligned}

Writing \(\mu=a/(a+b)\) for the mean response probability gives an equivalent and useful form:

E(X)=n\mu, \qquad \operatorname{Var}(X) =n\mu(1-\mu)\frac{\kappa+n}{\kappa+1} =n\mu(1-\mu)\left(1+\frac{n-1}{\kappa+1}\right).

The binomial variance at the same mean response probability is \(n\mu(1-\mu)\). The beta-binomial variance is at least as large because \(\kappa>0\) and \(n\geq1\). For \(n>1\), it is strictly larger. The additional variation reflects uncertainty in the shared basket probability: baskets with a higher latent \(p\) tend to produce more responses, and those with a lower \(p\) tend to produce fewer.

Worked Example: Quantifying Overdispersion

Consider a hypothetical basket with \(n=10\) participants and \(p\sim\operatorname{Beta}(2,3)\). Calculate the expected response count and its variance, then compare the variance with that of a binomial model using the same mean response probability.

Here \(\kappa=2+3=5\) and \(\mu=2/5=0.4\). The expected count is

E(X)=n\mu=10(0.4)=4 \text{ responses}.

The beta-binomial variance is

\operatorname{Var}(X) =10(0.4)(0.6)\frac{5+10}{5+1} =2.4\left(\frac{15}{6}\right) =6.00 \text{ responses}^2.

A binomial model with fixed \(p=0.4\) has variance \(10(0.4)(0.6)=2.40\) responses\(^2\). Thus the beta-binomial variance is 2.5 times as large. This comparison does not say the mean response count changes: both models have mean 4. It says the beta-binomial assigns greater spread to counts because the response probability varies across its beta distribution.

What the Marginal Distribution Adds

A useful way to see the effect of integrating out \(p\) is to consider a small basket and a diffuse distribution for its response probability. For \(p\sim\operatorname{Beta}(a,b)\), the concentration \(\kappa=a+b\) controls how tightly the beta distribution is concentrated around its mean. Holding \(\mu=a/(a+b)\) fixed, smaller \(\kappa\) means greater variation in \(p\). The variance formula shows that this also increases the marginal variance of \(X\).

Worked Example: A Diffuse Probability Distribution and Extreme Counts

Suppose a hypothetical basket has \(n=3\) participants and \(p\sim\operatorname{Beta}(1,1)\), the uniform distribution on the interval from 0 to 1. Find the probability of each possible count and compare the probability of an extreme count with a fixed-probability binomial model having the same mean.

For \(x=0,1,2,3\), the beta-binomial mass is

P(X=x)=\binom{3}{x}\frac{B(1+x,1+3-x)}{B(1,1)}.

Since \(B(1,1)=1\), direct substitution gives

\begin{aligned} P(X=0)&=B(1,4)=\frac14,\\ P(X=1)&=3B(2,3)=3\left(\frac1{12}\right)=\frac14,\\ P(X=2)&=3B(3,2)=3\left(\frac1{12}\right)=\frac14,\\ P(X=3)&=B(4,1)=\frac14. \end{aligned}

The mean response probability is \(1/2\), so a fixed-\(p=1/2\) binomial model also has mean count \(3/2\). Under that binomial model, the probability of an extreme count, either 0 or 3, is

P(X=0\text{ or }X=3) =\left(\frac12\right)^3+\left(\frac12\right)^3 =\frac14.

Under the beta-binomial model, the same probability is \(1/4+1/4=1/2\). The models have the same mean, but the beta-binomial assigns twice as much probability to the two extreme counts in this example. The difference comes from averaging over a wide range of possible basket response probabilities.

Interpretation, Diagnostics, and Alternatives

The beta-binomial model is useful when the trial design and scientific context support a shared but uncertain response probability within a basket. Its marginal variance provides a direct indication of how much this uncertainty inflates count variability relative to a fixed-\(p\) binomial model. The beta-binomial probability mass function also makes it possible to calculate probabilities for particular response counts while accounting for the distribution of \(p\).

Model checking should focus on whether its assumptions are plausible, not just whether the formula can be evaluated. Consider whether participants within a basket can reasonably share one response probability, whether conditional independence is credible, and whether a beta distribution represents plausible variation in that probability. In observed data, counts that show more or less dispersion than the model allows may signal a mismatch, although a small number of baskets gives limited information for diagnosing dispersion. Covariate imbalance, changing eligibility or assessment procedures, and time trends can also create patterns that a single shared \(p\) does not represent.

The main alternative is a binomial model with a fixed response probability. It is appropriate when that probability is treated as known for the relevant calculation, but it does not account for variation in \(p\). A beta-binomial model instead integrates over uncertainty in \(p\), producing a marginal count distribution. It is not a universal correction for overdispersion: if the beta distribution or conditional binomial assumptions are unsuitable, the resulting marginal probabilities may still be misleading. In particular, a beta-binomial model does not explain the source of heterogeneity; it encodes a particular distribution for it.

Model	Response probability	Count variance	Interpretation
Binomial	Fixed at \(p\)	\(np(1-p)\)	Participants share a fixed probability for the calculation.
Beta-binomial	Drawn from \(\operatorname{Beta}(a,b)\)	\(n\mu(1-\mu)(\kappa+n)/(\kappa+1)\)	The count averages over uncertainty or variation in the shared probability.

A common mistake is to substitute the prior mean \(\mu=a/(a+b)\) into a binomial model and assume this gives the beta-binomial distribution. That substitution matches the mean count, but it discards variation in \(p\). For example, in the \(\operatorname{Beta}(2,3)\), \(n=10\) calculation, the fixed-\(p=0.4\) binomial variance is 2.40, whereas the beta-binomial variance is 6.00. The binomial substitution therefore fails to reproduce the marginal spread and count probabilities.

Key takeaway: Integrating a binomial likelihood over a beta distribution gives \(P(X=x)=\binom{n}{x}B(a+x,b+n-x)/B(a,b)\). The resulting count has mean \(n\mu\) and variance \(n\mu(1-\mu)(\kappa+n)/(\kappa+1)\), reflecting both binomial sampling and uncertainty in the shared response probability.

Check Your Understanding

Use the model and formulas developed above. Solutions follow the questions.

For \(p\sim\operatorname{Beta}(1,2)\) and \(n=2\), write the beta-binomial expression for \(P(X=1)\) and calculate its value.
For \(p\sim\operatorname{Beta}(3,3)\) and \(n=6\), calculate the mean response count.
For the model in question 2, calculate the beta-binomial variance and the binomial variance at the same mean response probability.
In one sentence, explain why the beta-binomial count can have more variance than a binomial count with the same mean.

Solutions. For question 1,

P(X=1)=\binom21\frac{B(2,3)}{B(1,2)} =2\frac{1/12}{1/2} =\frac13\approx0.3333.

For question 2, \(\mu=3/(3+3)=1/2\), so \(E(X)=6(1/2)=3\) responses. For question 3, \(\kappa=6\), so

\operatorname{Var}_{\mathrm{Beta\text{-}Binomial}}(X) =6\left(\frac12\right)\left(\frac12\right)\frac{6+6}{6+1} =\frac{18}{7}\approx2.571 \text{ responses}^2.

The fixed-probability binomial variance is \(6(1/2)(1/2)=1.50\) responses\(^2\). For question 4, integrating over the beta distribution allows the shared response probability to vary, adding variation in the count beyond binomial sampling at a fixed probability.