Bayesian Shrinkage for Allele-Frequency Estimates

Why Shrink an Allele-Frequency Estimate?

An observed allele frequency is a sample proportion. When a small sample contains no copies of an allele, or only copies of that allele, the observed proportion is exactly zero or one. Those endpoints can look more certain than the data warrant: another sample from the same population could contain the allele that was not observed. Bayesian estimation makes this uncertainty explicit by combining the observed count with a prior distribution for the population frequency.

The result depends on both the data and the prior. A prior can pull an estimate away from zero or one, a phenomenon called shrinkage. Shrinkage is not a correction that automatically improves every estimate: it reflects the prior information or regularization encoded in the model. Its strength and direction should therefore be reported and justified.

The Beta-Binomial Model

Let $p$ denote the population frequency of the alternate allele. Suppose $m$ allele copies are sampled, and let $K$ be the number of alternate copies observed. For this model, conditional on $p$, each sampled copy has alternate-allele probability $p$, and the copies are sampled independently. Thus $K$ has a binomial distribution. In diploid data, $m$ may be twice the number of individuals with complete genotype calls, provided this allele-copy sampling model is appropriate.

Definition: A beta prior with parameters $a>0$ and $b>0$ assigns density $$ \pi(p)=\frac{p^{a-1}(1-p)^{b-1}}{B(a,b)},\qquad 0<p<1, $$ where $B(a,b)=\int_0^1 p^{a-1}(1-p)^{b-1}\,dp$ is the beta function. Under the binomial sampling model, observing $K=k$ alternate copies among $m$ sampled copies gives the posterior distribution $$ p\mid K=k \sim \operatorname{Beta}(a+k,\ b+m-k). $$

The beta family is useful because its shape matches the binomial likelihood: multiplying the prior density by the likelihood adds the observed alternate count to the first parameter and the observed reference count to the second. The commonly used uniform prior is $\operatorname{Beta}(1,1)$; it gives equal prior density to frequencies between zero and one. Other choices express different prior beliefs. For example, $\operatorname{Beta}(2,8)$ has prior mean $0.2$, while $\operatorname{Beta}(1/2,1/2)$ puts more density near the endpoints than the uniform prior does.

The posterior mean and variance follow from the moments of a beta distribution. If $A=a+k$ and $C=b+m-k$, then $$ E(p\mid K=k)=\frac{A}{A+C}=\frac{a+k}{a+b+m}, \qquad \operatorname{Var}(p\mid K=k)=\frac{AC}{(A+C)^2(A+C+1)}. $$ The posterior mean is one point estimate; the posterior distribution also describes uncertainty about $p$.

Why the Posterior Mean Is a Shrunk Estimate

Write the observed alternate-allele proportion as $\hat{p}=k/m$, and the prior mean as $\mu_0=a/(a+b)$. Rewriting the posterior mean shows exactly how the two quantities are combined:

E(p\mid K=k) =\frac{k+a}{m+a+b} =\frac{m}{m+a+b}\hat{p} +\frac{a+b}{m+a+b}\mu_0.

The weights are nonnegative and add to one. The posterior mean therefore lies between the sample proportion and the prior mean. With more sampled copies, the weight on $\hat{p}$ increases; with a larger prior concentration $a+b$, the weight on the prior mean increases. If the prior mean and sample proportion agree, there is no movement between them. This weighted-average identity is the central reason beta priors produce shrinkage.

Specify the sampling model.
Record the number $m$ of sampled allele copies and the number $k$ of alternate copies. Use the binomial model only when its independent-copy sampling assumption is reasonable.

Choose and state the prior.
Give both beta parameters and, preferably, explain the prior mean and concentration in context. Do not treat the prior as if it came from the current sample.

Update and summarize.
The posterior is $\operatorname{Beta}(a+k,b+m-k)$. Report a posterior summary and, when relevant, a credible interval.

Worked Example: No Alternate Copies Observed

Suppose a hypothetical sample contains $m=20$ allele copies and $k=0$ alternate copies. Use a uniform $\operatorname{Beta}(1,1)$ prior. Assume the 20 copies satisfy the independent binomial sampling model.

The posterior parameters are $1+0=1$ and $1+20-0=21$, so $$ p\mid K=0\sim\operatorname{Beta}(1,21). $$ The posterior mean is $1/(1+21)=1/22\approx0.04545$. The observed proportion is $0/20=0$, but the posterior mean is above zero because the continuous prior and the data do not establish that the population frequency is exactly zero.

For a beta distribution with parameters $(1,21)$, the cumulative distribution function is $F(p)=1-(1-p)^{21}$. An equal-tail 95% credible interval has 2.5% posterior probability below its lower endpoint and 2.5% above its upper endpoint. Solving $F(p)=q$ gives $p=1-(1-q)^{1/21}$, so the endpoints are $$ 1-0.975^{1/21}\approx0.001205, \qquad 1-0.025^{1/21}\approx0.1611. $$ Thus the interval is approximately $[0.001205,\,0.1611]$. Conditional on this model and prior, 95% of the posterior probability for $p$ lies in this interval. It does not mean there is a 95% frequentist probability that a fixed $p$ lies in this particular interval.

Credible Intervals and Boundary Counts

For a continuous posterior distribution, an equal-tail 100$(1-\alpha)$% credible interval is the interval between the posterior quantiles $\alpha/2$ and $1-\alpha/2$. Each tail outside the interval has posterior probability $\alpha/2$. The quantiles are usually obtained from a beta-distribution calculation or statistical software. Unlike a confidence interval, a Bayesian credible interval is interpreted using posterior probabilities conditional on the model, prior, and observed data.

A continuous beta prior assigns probability zero to the exact events $p=0$ and $p=1$. The posterior remains continuous beta, so it also assigns probability zero to either exact endpoint. This remains true if all sampled copies carry the same allele. A posterior interval near an endpoint expresses support for frequencies close to that endpoint, not positive posterior probability that the frequency equals it. To assign positive probability to $p=0$ or $p=1$, the model must include a prior with a point mass at the relevant endpoint as well as any continuous component.

Worked Example: Every Sampled Copy Is Alternate

In a second hypothetical sample, all $m=20$ copies are alternate, so $k=20$. Use the same uniform $\operatorname{Beta}(1,1)$ prior and the same binomial sampling assumption.

The posterior is $$ p\mid K=20\sim\operatorname{Beta}(21,1), $$ with posterior mean $21/22\approx0.9545$, compared with the observed proportion $20/20=1$. For this posterior the cumulative distribution function is $F(p)=p^{21}$. Its equal-tail 95% credible interval is therefore $$ [\,0.025^{1/21},\ 0.975^{1/21}\,]\approx[\,0.8389,\ 0.9988\,]. $$

The posterior places substantial probability on high frequencies below one, while retaining uncertainty across a range of values. Because this is a continuous beta posterior, $P(p=1\mid K=20)=0$; the interval does not represent uncertainty about the event $p=1$. A model intended to assign positive probability to that event would need a point mass at one.

Worked Example: Comparing Prior Choices With Intermediate Counts

Suppose a third hypothetical sample has $m=20$ copies, of which $k=6$ are alternate. The observed proportion is $6/20=0.3000$. Assume binomial sampling, and compare a uniform $\operatorname{Beta}(1,1)$ prior with a $\operatorname{Beta}(2,8)$ prior.

Under the uniform prior, the posterior is $\operatorname{Beta}(1+6,1+14)=\operatorname{Beta}(7,15)$. Its mean is $7/22\approx0.3182$. The weighted-average form gives the same result: $$ \frac{20}{22}(0.3000)+\frac{2}{22}(0.5000) =0.2727+0.04545 \approx0.3182. $$ The sample proportion is pulled toward the prior mean of $0.5$.

Under the $\operatorname{Beta}(2,8)$ prior, the posterior is $\operatorname{Beta}(8,22)$, with mean $8/30\approx0.2667$. Equivalently, $$ \frac{20}{30}(0.3000)+\frac{10}{30}(0.2000) =0.2000+0.06667 \approx0.2667. $$ This estimate is pulled toward the prior mean of $0.2$, and this prior has greater concentration $a+b=10$ than the uniform prior's concentration of 2. Neither calculation makes one prior universally correct. The appropriate choice depends on defensible information about the frequency before observing these 20 copies.

Assumptions, Prior Choice, and Common Pitfalls

The update is mathematically straightforward, but its interpretation depends on the model. The likelihood assumes that, conditional on $p$, the sampled allele copies have the same alternate-allele probability and are independent. It also assumes the recorded counts are the counts being modeled. The earlier material on genotype quality and allele counting helps establish which calls and copies enter those counts; uncertainty in genotype calls is not represented by this binomial model.

The beta prior must be stated rather than hidden inside a calculation. Its mean $a/(a+b)$ sets the direction of shrinkage, and its concentration $a+b$ affects its strength. A prior selected only because it yields a convenient result can make an estimate misleading. In an analysis, explain whether the prior is a simple baseline choice or reflects external knowledge, and examine whether conclusions are sensitive to reasonable alternatives.

Mistake: treating zero observed copies as proof that $p=0$. A finite sample with $k=0$ is compatible with small positive frequencies. Under a continuous beta prior the posterior probability at exactly zero is still zero.
Mistake: saying a continuous posterior supports $p=1$ as an exact event. A credible interval near one supports high values below one; positive probability for $p=1$ requires a prior point mass there.
Mistake: interpreting the posterior mean as the observed frequency. The sample proportion is $k/m$; the posterior mean combines that proportion with the prior mean.
Mistake: reporting a credible interval without its conditioning model. State that the interval is conditional on the specified likelihood, prior, and observed data, and describe its posterior probability rather than using confidence-interval language.
Mistake: assuming more shrinkage is always better. Shrinkage depends on the prior and can move an estimate away from the truth if that prior is inappropriate.

Bayesian shrinkage is especially visible at the boundaries, where raw sample proportions cannot express the uncertainty left by limited observations. The posterior keeps the estimate and its uncertainty connected: counts update the prior, while the prior influences how strongly a finite sample is allowed to move the estimate. Clear reporting of both makes the role of the modeling choice visible.

Check Your Understanding

Use the beta-binomial model and distinguish posterior statements from statements about the observed sample.

With a $\operatorname{Beta}(1,1)$ prior, what is the posterior distribution after observing $k=3$ alternate copies among $m=12$ allele copies?
Explain why the posterior mean is a weighted average of the sample proportion and the prior mean, and identify what increases the weight on the prior mean.
Under a continuous beta prior, what is the posterior probability that $p=0$ after observing no alternate copies? Explain what the posterior can still say about small positive frequencies.
For $m=20$ and $k=6$, which direction does the posterior mean move under a prior with mean $0.2$? Why?
What additional prior structure is needed to assign positive posterior probability to the event $p=1$?