Admissibility of Beta-Binomial Posterior Means

Why a Beta Posterior Mean Is an Admissible Rule

For a binomial observation, the sample proportion is a natural estimate of the success probability, but it can equal zero or one even when those extreme estimates are not appropriate for the inferential problem. A Beta prior leads to a posterior mean that pulls the estimate away from the boundaries. The key question here is not merely how to calculate that mean: it is whether some other estimator can have no greater risk at every parameter value and strictly smaller risk somewhere.

We will answer that question for a fixed binomial sample size and squared-error loss. The proof uses a proper Beta prior in two ways. Its posterior mean minimizes average risk under that prior, and its density is positive throughout the interior of the parameter space. Risk continuity then makes a strict improvement at even an endpoint incompatible with the Bayes comparison.

Model, Loss, and Posterior Mean

Definition: Let $n\geq 1$ be fixed, let the parameter space be $\Theta=[0,1]$, and suppose $X\sim\operatorname{Bin}(n,\theta)$. An estimator is a function $\delta:\{0,1,\ldots,n\}\to\mathbb{R}$. Under squared-error loss $L(\theta,\delta(x))=(\delta(x)-\theta)^2$, its risk is $$ R(\theta,\delta)=E_\theta[(\delta(X)-\theta)^2] =\sum_{x=0}^{n}\binom{n}{x}\theta^x(1-\theta)^{n-x}(\delta(x)-\theta)^2. $$ An estimator is admissible if there is no estimator with risk no greater at every $\theta\in\Theta$ and strictly smaller at at least one $\theta\in\Theta$.

Take a proper Beta prior with parameters $a>0$ and $b>0$, with density $\pi(\theta)=\theta^{a-1}(1-\theta)^{b-1}/B(a,b)$ for $0<\theta<1$. The likelihood, viewed as a function of $\theta$, is proportional to $\theta^x(1-\theta)^{n-x}$. Multiplying by the prior density gives the posterior distribution $ \theta\mid X=x\sim\operatorname{Beta}(a+x,b+n-x)$.

Posterior-mean rule: Under squared-error loss, the posterior mean is the Bayes estimator. For the Beta-binomial model it is $$ \delta_{a,b}(x)=E[\theta\mid X=x]=\frac{a+x}{a+b+n},\qquad x=0,1,\ldots,n. $$

The mean formula follows from the Beta distribution’s mean: a $\operatorname{Beta}(u,v)$ random variable has mean $u/(u+v)$ for $u,v>0$. In particular, the posterior mean combines the observed count $x$ with the prior parameters $a,b$. We next calculate the rule and its associated quantities in concrete cases.

Worked Example: Updating a Beta Prior After Eight Trials

Suppose $X\sim\operatorname{Bin}(8,\theta)$, the prior is $\operatorname{Beta}(2,3)$, and the observed count is $x=3$. Find the posterior distribution and posterior-mean estimate.

The prior density is proportional to $\theta^{2-1}(1-\theta)^{3-1}$. The likelihood at $x=3$ is proportional to $\theta^3(1-\theta)^5$; the factor $\binom{8}{3}$ does not depend on $\theta$ and cancels in the posterior normalization. Thus the posterior density is proportional to $\theta^{4}(1-\theta)^{7}$, which is the kernel of a $\operatorname{Beta}(5,8)$ density.

The posterior mean is $\delta_{2,3}(3)=(2+3)/(2+3+8)=5/13$, approximately $0.385$. The sample proportion is $3/8=0.375$. Both calculations are estimates of $\theta$, but only the former is the posterior mean under the stated prior.

Worked Example: Predictive Probabilities and Integrated Risk

Let $n=2$ and use a $\operatorname{Beta}(2,2)$ prior. Calculate the prior predictive probabilities of $X=0,1,2$, the posterior means, and the integrated risk of the posterior-mean rule.

For a count $x$, the prior predictive probability is $m(x)=\binom{2}{x}B(x+2,2-x+2)/B(2,2)$. Since $B(2,2)=1/6$, the three probabilities are $m(0)=B(2,4)/B(2,2)=(1/20)/(1/6)=3/10$, $m(1)=2B(3,3)/B(2,2)=2(1/30)/(1/6)=2/5$, and $m(2)=B(4,2)/B(2,2)=(1/20)/(1/6)=3/10$. They are positive and sum to $3/10+4/10+3/10=1$.

The posterior distributions for $x=0,1,2$ are respectively $\operatorname{Beta}(2,4)$, $\operatorname{Beta}(3,3)$, and $\operatorname{Beta}(4,2)$. Their means are $1/3,1/2,2/3$, so the posterior-mean rule takes those three values.

For a $\operatorname{Beta}(u,v)$ distribution the variance is $uv/[(u+v)^2(u+v+1)]$. The posterior variances here are $8/(36\cdot7)=2/63$, $9/(36\cdot7)=1/28$, and $2/63$. The integrated risk of the posterior mean equals the prior predictive average of those posterior variances: $(3/10)(2/63)+(2/5)(1/28)+(3/10)(2/63)=2/105+1/70=1/30$.

Integrated-Risk Comparison

Write $R_\pi(\delta)=\int_0^1 R(\theta,\delta)\pi(\theta)\,d\theta$ for the risk averaged over the Beta prior. Let $m(x)=P_\pi(X=x)$ be the prior predictive probability, and let $\pi(\theta\mid x)$ denote the posterior density. Each $m(x)$ is positive: the binomial likelihood is positive for $0<\theta<1$, and the Beta prior has positive density there.

Theorem: For every estimator $\delta:\{0,\ldots,n\}\to\mathbb{R}$, $$ R_\pi(\delta) =R_\pi(\delta_{a,b})+ \sum_{x=0}^{n}m(x)\bigl(\delta(x)-\delta_{a,b}(x)\bigr)^2. $$ Consequently, $\delta_{a,b}$ uniquely minimizes integrated squared-error risk among these estimators.

Proof. Because the sum over $x$ is finite, integrating the risk and grouping terms by the observed count gives $R_\pi(\delta)=\sum_{x=0}^n m(x)E[(\delta(x)-\theta)^2\mid X=x]$. For each fixed $x$, write $\mu_x=E[\theta\mid X=x]=\delta_{a,b}(x)$. Expanding around $\mu_x$ yields $$ E[(\delta(x)-\theta)^2\mid X=x] =(\delta(x)-\mu_x)^2+E[(\mu_x-\theta)^2\mid X=x], $$ because the cross term is $2(\delta(x)-\mu_x)E[\mu_x-\theta\mid X=x]=0$. The second term is the posterior variance and is exactly the conditional squared-error risk of the posterior-mean rule. Multiplying by $m(x)$, summing over $x$, and identifying the resulting posterior-mean risk gives the displayed identity. Since every $m(x)>0$, the sum of squared differences is nonnegative and equals zero only if $\delta(x)=\delta_{a,b}(x)$ for every $x$. This proves both claims. $\square$

Worked Example: Checking Risk in the One-Trial Uniform-Prior Model

Let $n=1$ and take a $\operatorname{Beta}(1,1)$ prior, which is uniform on $[0,1]$. The posterior-mean estimator is $\delta(0)=1/3$ and $\delta(1)=2/3$. Derive its risk and integrated risk.

The observation equals zero with probability $1-\theta$ and one with probability $\theta$. Therefore $R(\theta,\delta)=(1-\theta)(1/3-\theta)^2+\theta(2/3-\theta)^2$. Expanding the two terms gives $(1-\theta)(\theta^2-2\theta/3+1/9)+\theta(\theta^2-4\theta/3+4/9)$. The cubic terms cancel, leaving $R(\theta,\delta)=(1-3\theta+3\theta^2)/9$.

Averaging under the uniform prior gives $\int_0^1(1-3\theta+3\theta^2)/9\,d\theta =(1-3/2+1)/9=1/18$. The prior predictive probabilities for $X=0,1$ are both $1/2$; the posterior variances are both $1/18$, which also gives integrated risk $(1/2)(1/18)+(1/2)(1/18)=1/18$.

This calculation also warns against assuming the posterior mean dominates the sample proportion $\delta_{\mathrm{prop}}(x)=x$. At $\theta=0$, the sample proportion has risk $0$, while the posterior mean has risk $1/9$. At $\theta=1/2$, the risks are $1/4$ and $1/36$, respectively. Neither rule dominates the other at every parameter value.

Admissibility by Strict Risk Comparison

Theorem: For $n\geq1$, $a>0$, and $b>0$, the posterior-mean estimator $\delta_{a,b}(x)=(a+x)/(a+b+n)$ is admissible for estimating $\theta\in[0,1]$ under squared-error loss.

Proof. Suppose, to the contrary, that an estimator $\delta$ has $R(\theta,\delta)\leq R(\theta,\delta_{a,b})$ for every $\theta\in[0,1]$, with strict inequality at some $\theta_0\in[0,1]$. Define the risk difference $D(\theta)=R(\theta,\delta_{a,b})-R(\theta,\delta)$. Then $D$ is nonnegative throughout $[0,1]$ and $D(\theta_0)>0$.

For either estimator, its risk is a finite sum of terms involving $\binom{n}{x}\theta^x(1-\theta)^{n-x}$ and the fixed finite number $(\delta(x)-\theta)^2$, or its corresponding value under $\delta_{a,b}$. Each term is continuous on $[0,1]$, including at its endpoints. Hence both risks, and therefore $D$, are continuous on $[0,1]$. Continuity and $D(\theta_0)>0$ imply that $D$ is bounded below by a positive constant on a relative neighborhood of $\theta_0$ in $[0,1]$.

That neighborhood contains an interval of positive length inside $(0,1)$, even if $\theta_0=0$ or $\theta_0=1$. The Beta prior density is strictly positive throughout $(0,1)$, so this interval has positive prior probability. Since $D\geq0$ everywhere, it follows that $\int_0^1D(\theta)\pi(\theta)\,d\theta>0$, or equivalently $R_\pi(\delta)<R_\pi(\delta_{a,b})$. But the integrated-risk identity proves $R_\pi(\delta)\geq R_\pi(\delta_{a,b})$ for every estimator $\delta$. This contradiction shows that no such dominating estimator exists. Thus $\delta_{a,b}$ is admissible. $\square$

Why the Endpoint Detail Matters

A proper Beta prior with $a,b>0$ assigns probability zero to the individual points $\theta=0$ and $\theta=1$. That fact does not weaken this proof. A strict risk improvement at an endpoint cannot remain isolated: continuity makes the improvement hold on a one-sided interval next to that endpoint, and the prior assigns positive probability to the interval. The proof relies on this combination of continuity and positive density in the interior, not on positive prior mass at each parameter value.

A common mistake is to argue that a Bayes estimator must automatically be admissible simply because it minimizes average risk. The missing step is that an average-risk comparison alone does not rule out a strict improvement at a prior-null parameter point. Here the correction is to establish continuity of the risk difference and use the prior’s positive density on every interior interval. Conversely, prior mass at the endpoints is not required.

Key takeaway: For the binomial model on $[0,1]$, a proper $\operatorname{Beta}(a,b)$ prior with $a,b>0$ yields the posterior mean $(a+x)/(a+b+n)$. Its unique integrated-risk minimum, together with continuous risk and positive prior density in the interior, proves admissibility.

Check Your Understanding

Work through these questions to check both the calculations and the admissibility argument.

For $n=5$, prior $\operatorname{Beta}(2,4)$, and observed $x=2$, identify the posterior distribution and calculate its mean.
Why is the prior predictive probability $m(x)$ positive for every $x=0,\ldots,n$ when $a,b>0$?
In the integrated-risk identity, what makes the cross term vanish when squared error is expanded around the posterior mean?
Why does a strict risk improvement at $\theta=0$ imply a positive integrated-risk improvement under a proper Beta prior, even though the prior assigns zero probability to $\theta=0$ itself?
For the one-trial uniform-prior example, compare the posterior-mean risk with the sample-proportion risk at $\theta=0$ and $\theta=1/2$. What does this show about claiming dominance?