Beta-Binomial Mean, Variance, and Intraclass Correlation

Why a Shared Probability Changes the Count

A binomial count assumes that every trial has the same success probability and that outcomes are independent. In some grouped data, trials within a group share an unobserved propensity for success: one group may be more likely to succeed across all its trials than another. A beta-binomial model represents that variation by drawing one probability for the group and then generating its trial outcomes conditionally on that probability.

The key consequence is dependence. Given the group’s probability, the trials are independent; after averaging over the unobserved probability, two trials in the same group tend to move together. This tutorial derives how that dependence affects the count’s variance and measures it with an intraclass correlation.

Definition: Let $P\sim\operatorname{Beta}(\alpha,\beta)$, where $\alpha>0$ and $\beta>0$. Conditional on $P$, let $Y\mid P\sim\operatorname{Binomial}(m,P)$, with $m$ a positive integer. The marginal count $Y$, taking values $0,\ldots,m$, has a beta-binomial distribution. Write $s=\alpha+\beta$ and $q=\alpha/s$ for the mean of the latent probability.

Equivalently, introduce trial indicators $X_1,\ldots,X_m$, where $X_j=1$ denotes success on trial $j$, and $Y=\sum_{j=1}^m X_j$. Conditional on $P$, these indicators are independent Bernoulli random variables with success probability $P$. Marginally, they share the same random $P$, which is the source of their positive dependence.

Moments of the Latent Probability

The beta density is $f(p)=p^{\alpha-1}(1-p)^{\beta-1}/B(\alpha,\beta)$ for $0<p<1$, where $B(a,b)=\int_0^1 p^{a-1}(1-p)^{b-1}\,dp$. The beta-function identities $B(a+1,b)=aB(a,b)/(a+b)$ and $B(a+2,b)=a(a+1)B(a,b)/[(a+b)(a+b+1)]$ give the first two moments directly.

E[P]=\frac{B(\alpha+1,\beta)}{B(\alpha,\beta)} =\frac{\alpha}{s}=q, \qquad E[P^2]=\frac{B(\alpha+2,\beta)}{B(\alpha,\beta)} =\frac{\alpha(\alpha+1)}{s(s+1)}.

Subtracting the square of the mean gives

\operatorname{Var}(P) =\frac{\alpha(\alpha+1)}{s(s+1)}-\frac{\alpha^2}{s^2} =\frac{\alpha\beta}{s^2(s+1)} =\frac{q(1-q)}{s+1}.

The last equality uses $1-q=\beta/s$. Thus $q$ describes the average success probability, while $\operatorname{Var}(P)$ describes how much that probability varies across groups.

Worked Example: Finding the Latent-Probability Moments

Suppose $P\sim\operatorname{Beta}(2,3)$. Find its mean and variance.

Here $s=2+3=5$ and $q=2/5=0.4$. Therefore,

E[P]=0.4, \qquad \operatorname{Var}(P)=\frac{(0.4)(0.6)}{5+1} =\frac{0.24}{6}=0.04.

The latent probability has a standard deviation of $\sqrt{0.04}=0.2$. It is not fixed at $0.4$ for every group; its variation is what will increase the marginal variance of a group’s count.

Mean and Variance of the Beta-Binomial Count

Theorem: For $P\sim\operatorname{Beta}(\alpha,\beta)$ and $Y\mid P\sim\operatorname{Binomial}(m,P)$, with $\alpha,\beta>0$ and positive integer $m$, $$ E[Y]=mq,\qquad \operatorname{Var}(Y)=mq(1-q)\left(1+\frac{m-1}{s+1}\right), $$ where $s=\alpha+\beta$ and $q=\alpha/s$.

Proof. Conditional on $P$, the binomial mean and variance are $E[Y\mid P]=mP$ and $\operatorname{Var}(Y\mid P)=mP(1-P)$. By iterated expectation,

E[Y]=E[E[Y\mid P]]=E[mP]=mE[P]=mq.

For the variance, apply the law of total variance, $\operatorname{Var}(Y)=E[\operatorname{Var}(Y\mid P)]+\operatorname{Var}(E[Y\mid P])$. The first term is

E[\operatorname{Var}(Y\mid P)] =mE[P(1-P)] =m\{E[P]-E[P^2]\} =\frac{m\alpha\beta}{s(s+1)}.

The second term is

\operatorname{Var}(E[Y\mid P]) =\operatorname{Var}(mP) =m^2\operatorname{Var}(P) =\frac{m^2\alpha\beta}{s^2(s+1)}.

Adding the terms and using $q(1-q)=\alpha\beta/s^2$ yields

\operatorname{Var}(Y) =\frac{m\alpha\beta}{s(s+1)} +\frac{m^2\alpha\beta}{s^2(s+1)} =\frac{m\alpha\beta(s+m)}{s^2(s+1)} =mq(1-q)\frac{s+m}{s+1}.

Since $(s+m)/(s+1)=1+(m-1)/(s+1)$, this is the stated variance. $\square$

Worked Example: Count Moments for a Group of Ten

Let $P\sim\operatorname{Beta}(2,3)$, and conditionally on $P$, let $Y\sim\operatorname{Binomial}(10,P)$. Calculate the beta-binomial mean and variance, then compare the variance with a binomial count having success probability $0.4$.

From the preceding example, $s=5$, $q=0.4$, and $m=10$. The mean is

$$ E[Y]=10(0.4)=4. $$

A binomial count with $m=10$ and probability $0.4$ has variance $10(0.4)(0.6)=2.4$. For the beta-binomial count, the multiplier is $1+(10-1)/(5+1)=1+9/6=2.5$, so

\operatorname{Var}(Y)=2.4(2.5)=6.

The count has the same mean as the corresponding binomial count but a variance $6/2.4=2.5$ times as large. This difference comes from variation in $P$, not from a change in the conditional binomial variance formula.

Intraclass Correlation and Variance Inflation

An intraclass correlation is the correlation between two distinct outcomes in the same group. In this setting, choose $j\ne k$. Each indicator has marginal mean $E[X_j]=E[P]=q$, and hence marginal variance $q(1-q)$. Conditional independence gives $E[X_jX_k\mid P]=P^2$, so $E[X_jX_k]=E[P^2]$.

Theorem: For any two distinct trial indicators in the beta-binomial mixture, $$ \operatorname{Corr}(X_j,X_k)=\rho=\frac{1}{s+1} =\frac{1}{\alpha+\beta+1}. $$ Consequently, for $m\ge 2$, $$ \operatorname{Var}(Y)=mq(1-q)\{1+(m-1)\rho\}. $$

Proof. The covariance is

\operatorname{Cov}(X_j,X_k) =E[X_jX_k]-E[X_j]E[X_k] =E[P^2]-q^2 =\operatorname{Var}(P).

Both indicators have variance $q(1-q)>0$, since $\alpha,\beta>0$ imply $0<q<1$. Dividing the covariance by their common variance gives

\operatorname{Corr}(X_j,X_k) =\frac{\operatorname{Var}(P)}{q(1-q)} =\frac{1}{s+1} =\rho.

For the count $Y=\sum_{j=1}^m X_j$, expand the variance of a sum. There are $m$ individual variances and $m(m-1)$ ordered covariance terms:

\operatorname{Var}(Y) =\sum_{j=1}^m\operatorname{Var}(X_j) +2\sum_{1\le j<k\le m}\operatorname{Cov}(X_j,X_k) =mq(1-q)+m(m-1)\rho q(1-q).

Factoring gives $mq(1-q)\{1+(m-1)\rho\}$, as claimed. For $m=1$, the variance formula still holds, but there is no pair of distinct trials whose correlation could be calculated. $\square$

Worked Example: Interpreting an Intraclass Correlation

Suppose $P\sim\operatorname{Beta}(1,4)$, and each group has $m=6$ trials. Find the intraclass correlation, the count mean and variance, and the variance inflation over a binomial count with the same mean.

Here $s=5$, $q=1/5=0.2$, and $\rho=1/(5+1)=1/6$. The expected count is $mq=6(0.2)=1.2$. The binomial variance at that mean and trial count is

$$ mq(1-q)=6(0.2)(0.8)=0.96. $$

There are $m-1=5$ other trials that can be correlated with any given trial, so the inflation factor is $1+5(1/6)=11/6$. Thus,

\operatorname{Var}(Y)=0.96\left(\frac{11}{6}\right)=1.76.

The variance is $1.76/0.96=11/6$ times the binomial variance. The correlation $1/6$ is modest for a pair of trials, but its contribution to the count variance accumulates across the five other trials.

Reading the Inflation Factor

The factor $1+(m-1)\rho$ compares the beta-binomial variance with the binomial variance $mq(1-q)$ at the same $m$ and marginal success probability $q$. Because $\alpha,\beta>0$, the mixture has $0<\rho<1$; for $m>1$, its variance is therefore strictly larger than the corresponding binomial variance. When $m=1$, the factor is one: a single Bernoulli outcome has no within-group pair of trials to reveal dependence.

The factor depends on both the pairwise correlation and the number of trials. Holding $\rho$ fixed, adding trials increases the count’s relative variance inflation. Holding $m$ fixed, a larger $s=\alpha+\beta$ reduces $\rho=1/(s+1)$: the beta distribution is more concentrated around its mean, so groups differ less in their latent probabilities. As $s$ grows with $q$ fixed, the inflation factor tends to one.

The correlation here is specifically within a group. It does not say that outcomes from different groups are correlated; under a model with independently drawn latent probabilities for different groups, those groups are independent. Nor is $\rho$ the correlation between the count $Y$ and a trial indicator: it is the correlation between two distinct indicators in the same group.

Common Mistake: Treating Marginal Trials as Independent

A tempting argument is: “The trials are conditionally independent, so their unconditional covariance is zero and the count has binomial variance.” The first clause is true, but it only describes the distribution after conditioning on $P$. Conditional independence does not in general imply marginal independence when the variables share a random latent quantity.

The correction is to calculate the marginal covariance. The conditional covariance is zero, but the indicators’ conditional means both equal $P$. Averaging over that shared probability gives $\operatorname{Cov}(X_j,X_k)=\operatorname{Var}(P)>0$. The count’s variance must therefore include the covariance contributions, or, equivalently, the extra term in the law of total variance.

Key takeaway: In the beta-binomial mixture, the mean count is $mq$, while its variance is the binomial variance multiplied by $1+(m-1)\rho$. The intraclass correlation $\rho=1/(\alpha+\beta+1)$ measures how the shared latent probability creates dependence among trials.

Check Your Understanding

Use the latent-probability mixture to answer the following questions.

For $P\sim\operatorname{Beta}(3,3)$ and $m=8$, calculate $q$, $\rho$, the mean count, and the variance.
Why can conditionally independent trial indicators have positive marginal covariance?
For a group of $m$ trials, what is the variance inflation factor relative to a binomial count with the same $m$ and marginal success probability?
What happens to the variance inflation factor when $m=1$, and why is there no within-group intraclass correlation between distinct trials in that case?
For fixed $q$, what happens to $\rho$ as $s=\alpha+\beta$ increases, and what does that imply about variation in group-specific success probabilities?