Beta Mixing Derives the Beta-Binomial Distribution

Why Mix a Binomial Model?

A binomial model assumes that the success probability is fixed for all trials in a group. That assumption can be unsuitable when groups differ in their underlying propensity for success. For example, patients treated at the same clinic may share conditions affecting their outcomes, or schools may differ in their underlying pass rates. A natural model lets each group have its own success probability, while describing how those probabilities vary across groups.

The beta-binomial distribution follows by combining two familiar ingredients: conditional on a group's probability, its success count is binomial; across groups, that probability follows a beta distribution. The derivation is an example of a mixture distribution: the marginal probability of a count averages the conditional binomial probability over the distribution of the latent probability.

Definition: Let \(P\) have a beta distribution with parameters \(\alpha>0\) and \(\beta>0\). Given \(P=p\), let \(Y\) have a binomial distribution with \(m\) trials and success probability \(p\). The marginal distribution of \(Y\), obtained by integrating out \(P\), is the beta-binomial distribution with parameters \(m,\alpha,\beta\). Here \(m\) is a nonnegative integer, and \(Y\) takes values \(0,1,\ldots,m\).

The beta distribution has density on \(0<p<1\)

f_P(p)=\frac{p^{\alpha-1}(1-p)^{\beta-1}}{B(\alpha,\beta)}, \qquad B(\alpha,\beta)=\int_0^1 u^{\alpha-1}(1-u)^{\beta-1}\,du.

The positive parameters \(\alpha\) and \(\beta\) control the distribution of group probabilities. The conditional binomial mass at \(y\), for a given \(p\), is \(\binom{m}{y}p^y(1-p)^{m-y}\). Because the same \(P\) applies to every trial in a group, the marginal count is found by averaging this mass over the beta density.

Deriving the Beta-Binomial Mass

For any \(y\in\{0,\ldots,m\}\), the law of total probability for a continuous mixing variable gives

\begin{aligned} P(Y=y) &=\int_0^1 P(Y=y\mid P=p)f_P(p)\,dp\\ &=\int_0^1 \binom{m}{y}p^y(1-p)^{m-y} \frac{p^{\alpha-1}(1-p)^{\beta-1}}{B(\alpha,\beta)}\,dp\\ &=\binom{m}{y} \frac{\int_0^1p^{\alpha+y-1}(1-p)^{\beta+m-y-1}\,dp} {B(\alpha,\beta)}\\ &=\binom{m}{y} \frac{B(\alpha+y,\beta+m-y)}{B(\alpha,\beta)}. \end{aligned}

The last equality uses the definition of the beta function: the powers of \(p\) and \(1-p\) in the integral match its two arguments. This is the beta-binomial probability mass function.

Theorem: If \(m\) is a nonnegative integer and \(\alpha,\beta>0\), the beta-binomial mass function is

P(Y=y)=\binom{m}{y} \frac{B(\alpha+y,\beta+m-y)}{B(\alpha,\beta)}, \qquad y=0,1,\ldots,m.

Proof: Conditional on \(P=p\), the binomial mass is a probability mass function. For each \(y\), multiply it by the beta density of \(P\) and integrate over \(p\in(0,1)\), as above. This gives the displayed expression. To verify that the resulting masses sum to one, the sum is finite, so it can be moved inside the integral:

\begin{aligned} \sum_{y=0}^m P(Y=y) &=\int_0^1 \left\{\sum_{y=0}^m \binom{m}{y}p^y(1-p)^{m-y}\right\}f_P(p)\,dp\\ &=\int_0^1 \{p+(1-p)\}^m f_P(p)\,dp\\ &=\int_0^1 f_P(p)\,dp=1. \end{aligned}

The binomial theorem gives the second equality, and the beta density integrates to one by the definition of \(B(\alpha,\beta)\). Thus the expression is a valid probability mass function. \(\square\)

For computation, the beta function can also be written as \(B(a,b)=\Gamma(a)\Gamma(b)/\Gamma(a+b)\) for \(a,b>0\), where \(\Gamma\) is the gamma function. When parameters are positive integers, this becomes \(B(a,b)=(a-1)!(b-1)!/(a+b-1)!\). Another useful notation is the rising factorial \((a)^{\overline{k}}=a(a+1)\cdots(a+k-1)\), with \((a)^{\overline{0}}=1\).

Equivalent form: For \(y=0,\ldots,m\),

P(Y=y)=\binom{m}{y} \frac{(\alpha)^{\overline{y}}(\beta)^{\overline{m-y}}} {(\alpha+\beta)^{\overline{m}}}.

Derivation: Substitute \(B(a,b)=\Gamma(a)\Gamma(b)/\Gamma(a+b)\) into the beta-function ratio. The gamma recurrence \(\Gamma(a+k)/\Gamma(a)=(a)^{\overline{k}}\), for integer \(k\geq 0\), yields the stated expression after cancellation. The beta-function form is often clearest for deriving the mixture; the rising-factorial form can be convenient for evaluating its probabilities.

Worked Examples

Worked Example: A Count from a Beta(2, 3) Mixture

A group has \(m=4\) trials. Its success probability \(P\) has a beta distribution with \(\alpha=2\) and \(\beta=3\). Find the marginal probability of exactly two successes.

Use the mixture formula with \(y=2\):

P(Y=2)=\binom{4}{2}\frac{B(2+2,3+4-2)}{B(2,3)} =6\frac{B(4,5)}{B(2,3)}.

All beta-function arguments here are positive integers. Thus

B(4,5)=\frac{3!\,4!}{8!}=\frac{144}{40320}=\frac{1}{280}, \qquad B(2,3)=\frac{1!\,2!}{4!}=\frac{2}{24}=\frac{1}{12}.

Therefore,

P(Y=2)=6\frac{1/280}{1/12} =6\cdot\frac{12}{280} =\frac{72}{280} =\frac{9}{35}.

The answer is a marginal probability: it accounts for the fact that the group's success probability is not fixed at one value, but varies according to the beta distribution.

Worked Example: A Uniform Distribution of Group Probabilities

Suppose \(P\) is uniform on \((0,1)\), which is the beta distribution with \(\alpha=\beta=1\). For \(m=3\), find the probability of each possible count and check that the probabilities sum to one.

For each \(y=0,1,2,3\), the formula becomes

P(Y=y)=\binom{3}{y}B(1+y,1+3-y),

because \(B(1,1)=1\). Evaluate the four cases:

\begin{aligned} P(Y=0)&=1\cdot B(1,4)=\frac{0!\,3!}{4!}=\frac14,\\ P(Y=1)&=3\cdot B(2,3)=3\cdot\frac{1!\,2!}{4!}=\frac14,\\ P(Y=2)&=3\cdot B(3,2)=3\cdot\frac{2!\,1!}{4!}=\frac14,\\ P(Y=3)&=1\cdot B(4,1)=\frac{3!\,0!}{4!}=\frac14. \end{aligned}

The total is \(1/4+1/4+1/4+1/4=1\), as required. In this case, every possible count has equal marginal probability. This is not the conditional binomial distribution at a particular \(p\); it results from averaging over all group probabilities with equal density.

Worked Example: Same Central Probability, Different Group Variation

Compare two-trial groups under \(P\sim\operatorname{Beta}(2,2)\) and \(P\sim\operatorname{Beta}(8,8)\). Both beta densities are symmetric about \(1/2\), but the second puts more probability near \(1/2\). Find the probability that both trials succeed in each model.

Conditional on \(P=p\), the probability of two successes is \(p^2\). Integrating over the mixing distribution gives

P(Y=2)=\int_0^1p^2f_P(p)\,dp =\frac{B(\alpha+2,\beta)}{B(\alpha,\beta)}.

For \(\operatorname{Beta}(2,2)\), use \(B(a+1,b)/B(a,b)=a/(a+b)\), which follows directly from the gamma-function form of \(B\):

\frac{B(4,2)}{B(2,2)} =\frac{2}{4}\cdot\frac{3}{5} =\frac{3}{10}.

For \(\operatorname{Beta}(8,8)\), the same calculation gives

\frac{B(10,8)}{B(8,8)} =\frac{8}{16}\cdot\frac{9}{17} =\frac{9}{34}.

Thus the probabilities are \(3/10=0.30\) and \(9/34\approx0.265\), respectively. Although the mixing distributions are centered at the same value, the more dispersed group probabilities in the first model make extreme group outcomes, such as two successes, more likely. This comparison illustrates why the distribution of group probabilities matters; it is not a calculation of the beta-binomial variance.

What the Mixture Represents

The key modeling feature is that each group has one latent probability \(P\), shared across all its trials. If group \(j\) has probability \(P_j\), then conditional on \(P_j\), its count \(Y_j\) is binomial with \(m_j\) trials and success probability \(P_j\). Across groups, the \(P_j\) values can be modeled as draws from the same beta distribution. Integrating over \(P_j\) gives each group's beta-binomial count distribution.

Consequently, the mixture represents between-group probability variation, not a different fixed probability for every trial within a group. When a group's latent probability happens to be high, successes in that group are more likely together; when it is low, failures are more likely together. After \(P_j\) is integrated out, the trials in a group are generally not independent marginally, even though they are conditionally independent given \(P_j\). This shared latent probability is what permits the count distribution to differ from an ordinary binomial with one common fixed probability.

This interpretation is useful when grouped success counts show more variability across groups than a common-probability binomial model can represent. A beta-binomial model supplies a specific probability model for that heterogeneity. It is distinct from quasi-binomial modeling: the beta-binomial specifies a full marginal distribution for the counts, obtained from a stated mixing distribution.

Common Mistake: Mixing at the Wrong Level

A tempting but incorrect explanation is: “For each trial, independently draw a new probability from a beta distribution, then draw that trial's outcome; this gives a beta-binomial count.” That does not produce the beta-binomial mixture described here. If each trial gets an independent \(P_i\), then after integrating out each \(P_i\), the outcomes remain independent Bernoulli variables with the same marginal success probability. The group count is then binomial with that marginal probability.

For the beta-binomial, draw one \(P\) for the group, then conditionally generate all \(m\) trials using that same \(P\). The group-level probability is integrated out only after the conditional count probability is formed. The order and level of mixing are essential.

Key takeaway: A beta-binomial count is a binomial count with a shared, group-specific success probability drawn from a beta distribution. Integrating over that probability gives \(\binom{m}{y}B(\alpha+y,\beta+m-y)/B(\alpha,\beta)\); the mixing describes variation between groups.

Check Your Understanding

Work through these questions to check your understanding of the mixture construction.

Write the integral that gives \(P(Y=y)\) when \(Y\mid P=p\) is binomial with \(m\) trials and \(P\) has a beta distribution with parameters \(\alpha,\beta\).
For \(m=2\), \(\alpha=1\), and \(\beta=2\), calculate \(P(Y=1)\) using the beta-function form.
Why does the beta-binomial construction use the same latent probability for all trials in one group?
Explain why independent beta-distributed probabilities for each trial do not give the same mixture as a single group-level beta-distributed probability.
What feature of the model is represented by variation in the beta-distributed probabilities across groups?