Worked Risk Calculations For Discrete Models

Why Exact Risk Calculations Matter

An estimator can look reasonable and still perform differently across the parameter space. In a discrete model, the data distribution gives us a direct route to its exact squared-error risk: calculate the estimator’s bias and variance, or sum the loss over the possible observations. This makes it possible to compare rules at particular parameter values rather than relying on a broad impression such as “shrinkage reduces variability.”

The boundary values deserve special attention. A binomial proportion can be exactly zero or one, and a Poisson mean can be zero. At such values, the data may be degenerate, and an estimator that adds a positive adjustment can have positive risk even when the natural estimator has risk zero. The calculations below make these effects explicit.

Definition: If $X$ has distribution indexed by a parameter $\theta$, an estimator $\delta(X)$ has squared-error risk $R(\theta,\delta)=E_\theta[(\delta(X)-\theta)^2]$, provided this expectation exists. Risk is a function of the true parameter; it is not the realized error from one observed sample.

Binomial Proportion: Risk of the Sample Proportion

Let $X\sim\operatorname{Binomial}(n,p)$, where $n$ is a positive integer and $0\le p\le1$. The natural estimator of $p$ is the sample proportion $\hat p=X/n$. The binomial moments are $E_p[X]=np$ and $\operatorname{Var}_p(X)=np(1-p)$. Consequently, $\hat p$ is unbiased and its risk is its variance.

Result: For $X\sim\operatorname{Binomial}(n,p)$ and $\hat p=X/n$, $$ R(p,\hat p)=\frac{p(1-p)}{n},\qquad 0\le p\le1. $$

To see the calculation directly, unbiasedness gives $E_p[\hat p-p]=0$, so the risk is $\operatorname{Var}_p(\hat p)$. Scaling variance by $1/n^2$ yields

R(p,\hat p) =\operatorname{Var}_p\!\left(\frac{X}{n}\right) =\frac{1}{n^2}\operatorname{Var}_p(X) =\frac{np(1-p)}{n^2} =\frac{p(1-p)}{n}.

The risk is largest at $p=1/2$, where it equals $1/(4n)$, and is zero at both boundaries. At $p=0$, $X=0$ with probability one; at $p=1$, $X=n$ with probability one. In either case the sample proportion equals the true parameter exactly.

Worked Example: The Sample Proportion at an Interior Value

Suppose $n=20$ and $p=0.3$. The sample proportion has expectation $E_p[X/20]=p=0.3$, so its squared-error risk is its variance:

R(0.3,\hat p) =\frac{0.3(1-0.3)}{20} =\frac{0.21}{20} =0.0105.

This is an exact risk, averaging over all possible binomial samples of size 20. It is not the squared error for any one observed value of $X$. At the same sample size, the maximum risk occurs at $p=0.5$ and is $0.25/20=0.0125$.

Comparing the Sample Proportion with a Symmetric Adjustment

To see how a small adjustment changes risk, consider the estimator $\tilde p=(X+c)/(n+2c)$, where $c>0$ is fixed. It moves the estimate inward from the endpoints. Its expectation and bias follow from $E_p[X]=np$:

E_p[\tilde p]=\frac{np+c}{n+2c}, \qquad E_p[\tilde p]-p=\frac{c(1-2p)}{n+2c}.

Its variance is $\operatorname{Var}_p(\tilde p)=np(1-p)/(n+2c)^2$. The bias-variance identity says that mean squared error equals variance plus squared bias. Here this gives

R(p,\tilde p) =\frac{np(1-p)+c^2(1-2p)^2}{(n+2c)^2}.

The identity used here follows by writing $\delta-\theta=(\delta-E[\delta])+(E[\delta]-\theta)$, squaring, and taking expectations. The cross term vanishes because $E[\delta-E[\delta]]=0$. Thus the squared bias cannot be omitted just because an adjustment may reduce variance.

Worked Example: An Adjustment Helps at the Center but Not Everywhere

Take $n=10$, $c=1$, and compare $\hat p=X/10$ with $\tilde p=(X+1)/12$. At $p=0.5$, the natural estimator has risk

R(0.5,\hat p)=\frac{0.5(0.5)}{10}=0.025.

For the adjusted estimator, $1-2p=0$, so its bias is zero at this value. Its risk is

R(0.5,\tilde p) =\frac{10(0.5)(0.5)+1^2(0)^2}{12^2} =\frac{2.5}{144} \approx 0.01736.

The adjustment has lower risk here. Now take the interior value $p=0.1$. The natural estimator’s risk is $0.1(0.9)/10=0.009$, whereas

R(0.1,\tilde p) =\frac{10(0.1)(0.9)+1^2(1-0.2)^2}{144} =\frac{0.9+0.64}{144} \approx 0.01069.

At this interior value the adjustment has higher risk. At the boundary $p=0$, $X=0$ with probability one. The natural estimator has risk zero, but $\tilde p=1/12$ with probability one and therefore has risk $1/144$. An adjustment can help at some parameter values without improving risk everywhere.

Poisson Mean: Risk of the Count

Now let $X\sim\operatorname{Poisson}(\lambda)$, with $\lambda\ge0$. The natural estimator of the mean is $\hat\lambda=X$. Since $E_\lambda[X]=\lambda$ and $\operatorname{Var}_\lambda(X)=\lambda$, it is unbiased and has risk $\lambda$. These moments also hold at $\lambda=0$, where the distribution is concentrated at zero.

Result: For $X\sim\operatorname{Poisson}(\lambda)$ and $\hat\lambda=X$, $$ R(\lambda,\hat\lambda)=\lambda,\qquad \lambda\ge0. $$

Indeed, the estimator is unbiased, so its squared-error risk equals its variance:

R(\lambda,\hat\lambda) =E_\lambda[(X-\lambda)^2] =\operatorname{Var}_\lambda(X) =\lambda.

At the boundary $\lambda=0$, $X=0$ with probability one, and the natural estimator has zero risk. For positive $\lambda$, the risk increases linearly with the mean; unlike the binomial risk, it has no finite maximum over the parameter space.

Worked Example: Natural Poisson Risk

For a Poisson count with mean $\lambda=3$, the estimator $\hat\lambda=X$ is unbiased and has variance 3. Therefore

R(3,\hat\lambda) =E_3[(X-3)^2] =\operatorname{Var}_3(X) =3.

At $\lambda=0$, by contrast, the observation must be zero, so $R(0,\hat\lambda)=0$. This is an exact boundary calculation, not a limiting approximation.

Exact Comparison with Shrinkage Toward a Reference Value

For comparison, define a Poisson estimator that shrinks the observed count toward a fixed reference value $c\ge0$: $\delta_a(X)=(1-a)X+ac$, where $0\le a\le1$. The expectation is $(1-a)\lambda+ac$, giving bias $a(c-\lambda)$, and the variance is $(1-a)^2\lambda$. Adding variance and squared bias gives the exact risk

R(\lambda,\delta_a) =(1-a)^2\lambda+a^2(c-\lambda)^2.

The difference from the natural estimator’s risk is

R(\lambda,\delta_a)-R(\lambda,\hat\lambda) =-2a\lambda+a^2\bigl[\lambda+(c-\lambda)^2\bigr].

For $a>0$, the shrinkage rule has lower risk precisely when $2\lambda>a[\lambda+(c-\lambda)^2]$. This condition depends on the true mean: reducing variance does not by itself guarantee a risk reduction, because shrinkage also introduces bias.

Worked Example: Shrinkage Can Help in the Interior and Hurt at the Boundary

Choose $c=2$ and $a=1/4$, so $\delta_a(X)=0.75X+0.5$. At $\lambda=1$, its variance is $0.75^2(1)=0.5625$, and its squared bias is $[0.25(2-1)]^2=0.0625$. Thus

R(1,\delta_a)=0.5625+0.0625=0.625<1=R(1,\hat\lambda).

At $\lambda=2$, the bias is zero and the variance is $0.75^2(2)=1.125$, which is below the natural risk 2. At $\lambda=4$, the variance is $0.75^2(4)=2.25$, and the squared bias is $[0.25(2-4)]^2=0.25$. The total is $2.5$, which is below the natural risk 4.

At the boundary $\lambda=0$, however, $X=0$ with probability one. The shrinkage estimate is then $0.5$ with certainty, and

R(0,\delta_a)=(0.5-0)^2=0.25>0=R(0,\hat\lambda).

The interior calculations favor shrinkage at these selected values, but the boundary calculation favors the natural estimator. The reference value and amount of shrinkage matter, as does the parameter value.

How to Read a Risk Comparison

A risk comparison is pointwise unless a stronger claim is explicitly established. Saying that one rule has lower risk at $p=0.5$ does not show that it has lower risk for all $p$. Similarly, a calculation at one Poisson mean does not establish an ordering over all $\lambda$. Exact formulas make these qualifications visible: they retain both variance and bias and can be evaluated at interior and boundary values alike.

Write the model and parameter space.
Record the distribution and whether boundary values such as $p=0$ or $\lambda=0$ are included.

Find the estimator's mean and variance.
Use them to identify bias and variance, or sum squared error over the discrete support.

Calculate exact risk at each value of interest.
Compare the resulting numbers without treating a local advantage as a uniform one.

A common mistake is to compare estimators using variance alone. For example, the symmetric binomial adjustment reduces risk at $p=0.5$, where its bias vanishes, but its nonzero bias can make risk larger elsewhere. Another mistake is to infer the boundary risk from an informal description such as “the estimator is close to the natural one.” At a degenerate boundary, calculate the estimator’s actual constant value and square its error. Exact risk is determined by the full loss, not by variance alone or by behavior at a nearby interior point.

Key takeaway: For these discrete models, the natural estimators have risks $p(1-p)/n$ and $\lambda$. Adjustments change both variance and bias; their exact risks must be checked at the parameter values relevant to the comparison, including boundaries.

Check Your Understanding

Use the formulas and distinctions developed above to answer the following.

For $X\sim\operatorname{Binomial}(n,p)$, why is the risk of $X/n$ zero when $p=0$ or $p=1$?
For $n=10$, $c=1$, and $p=0.5$, calculate the risks of $X/n$ and $(X+c)/(n+2c)$. Which is smaller?
At $p=0.1$ in the same binomial comparison, identify the variance and squared-bias contributions to the adjusted estimator’s risk.
For $X\sim\operatorname{Poisson}(\lambda)$, explain why the risk of $X$ is zero at $\lambda=0$.
For the Poisson shrinkage rule, state the condition under which its risk is lower than the risk of $X$, for $a>0$.