Shrinkage Of A Poisson Rate Toward A Reference Value

Why Shrink a Poisson Count?

If $X$ is a Poisson count with rate $\lambda$, the count itself is an unbiased estimator of $\lambda$, with variance $\lambda$. Under squared-error loss, its risk is therefore $\lambda$. But an estimator need not be unbiased to have low risk: moving the count toward a useful reference value can reduce variability enough to compensate for the bias it introduces. Whether it does so depends on the true rate, the reference, and the amount of shrinkage.

This tutorial studies one specific rule: a fixed convex combination of the observed count and a specified reference rate. The reference and weight are treated as fixed, not estimated from the observed count. The aim is to calculate the rule’s risk and determine precisely when it is smaller than the risk of using $X$ alone.

Definition: Let $X\sim\operatorname{Poisson}(\lambda)$, where $\lambda\geq 0$. Fix a reference rate $\lambda_0\geq 0$ and a shrinkage weight $a\in[0,1]$. The fixed shrinkage estimator is $$ \delta_a(X)=(1-a)X+a\lambda_0. $$ It is a convex combination because its two weights are nonnegative and sum to one. Under squared-error loss $L(\lambda,\delta)=(\delta-\lambda)^2$, its risk is $R(\lambda,\delta_a)=E_\lambda[(\delta_a(X)-\lambda)^2]$.

When $a=0$, the rule is the raw count $X$; when $a=1$, it always reports the reference $\lambda_0$. Intermediate values retain part of the observed count and pull the estimate partway toward the reference. For a Poisson random variable, $E_\lambda[X]=\lambda$ and $\operatorname{Var}_\lambda(X)=\lambda$; these moment identities give a direct risk calculation.

Risk of the Fixed Shrinkage Rule

The estimator’s bias is the amount by which its expectation differs from the target. Since $E_\lambda[X]=\lambda$,

E_\lambda[\delta_a(X)]-\lambda =(1-a)\lambda+a\lambda_0-\lambda =a(\lambda_0-\lambda).

Its random part is $(1-a)X$, so its variance is $(1-a)^2\lambda$. The scalar bias-variance identity says that squared-error risk is variance plus squared bias. Combining these terms gives the exact risk.

Theorem: For $\lambda\geq 0$, fixed $\lambda_0\geq 0$, and $a\in[0,1]$, $$ R(\lambda,\delta_a)=(1-a)^2\lambda+a^2(\lambda_0-\lambda)^2. $$

Proof. Write the estimation error as $$ \delta_a(X)-\lambda=(1-a)(X-\lambda)+a(\lambda_0-\lambda). $$ The first term has expectation zero, and the second is constant. On squaring and taking expectation, the cross term vanishes:

\begin{aligned} E_\lambda[(\delta_a(X)-\lambda)^2] &=(1-a)^2 E_\lambda[(X-\lambda)^2] +2a(1-a)(\lambda_0-\lambda)E_\lambda[X-\lambda]\\ &\quad+a^2(\lambda_0-\lambda)^2\\ &=(1-a)^2\lambda+a^2(\lambda_0-\lambda)^2. \end{aligned}

Here $E_\lambda[X-\lambda]=0$ and $E_\lambda[(X-\lambda)^2]=\operatorname{Var}_\lambda(X)=\lambda$. Thus the expression is the risk, as claimed. $\square$

The first term of the risk is the variance remaining after scaling the count. It is at most $\lambda$, the risk of the raw count. The second term is squared bias, which is zero if the reference equals the true rate but can be substantial when they are far apart. The reduction in variance does not by itself establish an improvement; the two terms must be compared together.

Worked Example: Halfway Shrinkage Toward Rate 4

Take $\lambda_0=4$ and $a=\tfrac12$. Then $\delta_{1/2}(X)=\tfrac12X+2$, and the risk at an arbitrary true rate is

R(\lambda,\delta_{1/2}) =\frac14\lambda+\frac14(4-\lambda)^2.

At $\lambda=4$, the estimator is unbiased, so its risk is just its variance: $$ R(4,\delta_{1/2})=\frac14(4)+\frac14(0)^2=1. $$ The raw count has risk $R(4,X)=4$, so shrinkage reduces risk by $3$.

At $\lambda=1$, the same rule has risk $$ R(1,\delta_{1/2})=\frac14(1)+\frac14(3)^2=\frac14+\frac94=\frac52. $$ This exceeds the raw count’s risk of $1$. The reference is too far above the true rate for this fixed amount of shrinkage to help.

Exactly When Does Shrinkage Help?

The raw count has risk $R(\lambda,X)=\lambda$. Subtracting this benchmark from the shrinkage risk isolates the gain or loss. A negative difference means shrinkage helps; a positive difference means it hurts.

Theorem: If $0<a\leq 1$, the fixed shrinkage rule has strictly lower risk than the raw count exactly when $$ (\lambda_0-\lambda)^2<\frac{2-a}{a}\lambda. $$ For $a=0$, the two estimators are identical and their risks are equal for every $\lambda\geq0$.

Proof. Subtract the raw-count risk from the formula already derived:

\begin{aligned} R(\lambda,\delta_a)-R(\lambda,X) &=(1-a)^2\lambda+a^2(\lambda_0-\lambda)^2-\lambda\\ &=-a(2-a)\lambda+a^2(\lambda_0-\lambda)^2. \end{aligned}

When $a>0$, this difference is negative if and only if $$ a^2(\lambda_0-\lambda)^2<a(2-a)\lambda. $$ Dividing by the positive quantity $a$ and then by $a$ once more gives $$ (\lambda_0-\lambda)^2<\frac{2-a}{a}\lambda, $$ which proves the strict-improvement condition. If $a=0$, $\delta_0(X)=X$, so the risks agree. The condition also handles the boundary $\lambda=0$: its left side is $\lambda_0^2$, while its right side is zero, so strict improvement is impossible there, including when $\lambda_0=0$. $\square$

For a fixed positive weight, the condition describes an interval of rates around the reference. To make that interval explicit, let $c=(2-a)/a>0$. The strict-improvement inequality is equivalent to $$ \lambda^2-(2\lambda_0+c)\lambda+\lambda_0^2<0. $$ Its roots are $$ \lambda_{\pm}=\lambda_0+\frac{c}{2}\ \pm\ \frac12\sqrt{c^2+4c\lambda_0}. $$ Because the quadratic opens upward, improvement holds between the roots. For $\lambda_0>0$, both roots are positive. For $\lambda_0=0$, the roots are $0$ and $c$, and improvement holds for $0<\lambda<c$. Thus the reference and weight determine a range of rates where this particular rule helps; neither implies that it helps at all rates.

Worked Example: A Constant Rule with Reference Rate 2

Set $\lambda_0=2$ and $a=1$. The estimator ignores the count and always reports $2$. Its risk is $$ R(\lambda,\delta_1)=(2-\lambda)^2. $$ It improves on $X$ exactly when $$ (2-\lambda)^2<\lambda \quad\Longleftrightarrow\quad \lambda^2-5\lambda+4<0 \quad\Longleftrightarrow\quad (\lambda-1)(\lambda-4)<0. $$ Thus it helps precisely for $1<\lambda<4$. At $\lambda=2$, its risk is $0$, compared with risk $2$ for $X$. At $\lambda=5$, its risk is $9$, compared with risk $5$ for $X$, so it hurts. At $\lambda=1$ and $\lambda=4$, the risks are equal.

Worked Example: Shrinking Toward Zero

Let $\lambda_0=0$ and $a=\tfrac14$, so $\delta_{1/4}(X)=\tfrac34X$. The exact risk is $$ R(\lambda,\delta_{1/4}) =\frac{9}{16}\lambda+\frac{1}{16}\lambda^2. $$ The improvement condition is $$ \lambda^2<7\lambda, $$ so the rule improves on $X$ for $0<\lambda<7$, ties at $\lambda=0$ and $\lambda=7$, and has greater risk for $\lambda>7$. For example, at $\lambda=3$, $$ R(3,\delta_{1/4})=\frac{9}{16}(3)+\frac{1}{16}(9) =\frac{27+9}{16}=\frac94, $$ whereas $R(3,X)=3$. The risk reduction is $\tfrac34$. At $\lambda=8$, the shrinkage risk is $\tfrac{9}{16}(8)+\tfrac{64}{16}=8.5$, greater than the raw count’s risk of $8$.

Reading the Risk Comparison Carefully

The condition shows the tradeoff in a useful form. Shrinkage helps when the squared distance from the true rate to the reference is small enough relative to the rate itself. A stronger pull toward the reference can reduce more variance, but it also gives the estimator more bias when the reference is inaccurate. In particular, for $a>0$, the squared-bias term grows quadratically as $\lambda$ moves far from a fixed $\lambda_0$, whereas the raw-count risk grows linearly. A fixed reference therefore cannot guarantee improvement over the full unbounded parameter space.

For any given true rate, one could calculate the weight that minimizes risk if that rate were known. Writing $d=\lambda_0-\lambda$, the risk is the quadratic $$ R(\lambda,\delta_a)=\lambda-2a\lambda+a^2(\lambda+d^2). $$ When $\lambda>0$, its minimum over $a\in[0,1]$ occurs at $a=\lambda/(\lambda+d^2)$, which lies in $[0,1]$. This is an oracle calculation: it depends on the unknown $\lambda$, so it does not by itself specify an implementable choice of weight. The focus here is the risk of a fixed choice, not a procedure for selecting a weight from data.

A common mistake is to conclude that shrinkage must help because multiplying $X$ by $1-a$ reduces variance. That reasoning compares only the variance terms. The correct comparison includes the squared bias $a^2(\lambda_0-\lambda)^2$; the risk difference can be positive even when the variance is smaller. Another error is to use $\lambda_0$ in place of $\lambda$ for the Poisson variance. The distribution of the observation is governed by the true rate $\lambda$, not the fixed reference.

Key takeaway: For a fixed convex combination of a Poisson count and reference rate, risk equals the remaining variance plus squared bias. With positive shrinkage, it improves on the raw count exactly when the squared distance to the reference is less than $\frac{2-a}{a}\lambda$; outside that region it does not.

Check Your Understanding

Use the risk formula and comparison condition to answer the following.

For $a=\tfrac12$ and reference rate $\lambda_0=3$, write the exact risk function.
For $a=1$ and $\lambda_0=5$, determine the interval of positive rates where the constant estimator improves on $X$.
Explain why the rule with $a=\tfrac14$ and $\lambda_0=0$ ties rather than improves on $X$ at $\lambda=0$.
For a fixed $a>0$, what happens to the risk comparison when $\lambda$ is sufficiently far from the fixed reference rate? Explain using the orders of growth of the two terms.
Why does a weight that minimizes risk when $\lambda$ is known not automatically give a usable estimator when $\lambda$ is unknown?

Shrinkage Of A Poisson Rate Toward A Reference Value

What You'll Learn