Poisson Mean Estimation Under Squared Error

Introduction

An estimator’s risk depends both on the sampling model and on the loss used to judge its errors. For Poisson data, the sample mean is a natural estimate of the common event rate: it is unbiased, but its variability changes with that rate. Comparing its risk with the risk of a fixed estimate makes this dependence explicit.

Assume throughout that \(X_1,\ldots,X_n\) are independent observations from a Poisson distribution with common mean \(\theta\), where \(\theta\geq 0\) and \(n\geq 1\). The estimate is judged under squared-error loss. The calculations below include the boundary value \(\theta=0\), at which every observation is zero with probability one.

Definition: For an estimator \(\delta=\delta(X_1,\ldots,X_n)\) of \(\theta\), squared-error loss is \(L(\theta,\delta)=(\delta-\theta)^2\). Its risk at \(\theta\) is \(R(\theta,\delta)=E_\theta[(\delta-\theta)^2]\), where the expectation is taken under the sampling distribution indexed by \(\theta\). The sample mean is \(\bar X=n^{-1}\sum_{i=1}^n X_i\).

Moments of a Poisson Observation

For \(\theta>0\), a Poisson random variable \(X\) has mass function \(P(X=k)=e^{-\theta}\theta^k/k!\) for \(k=0,1,2,\ldots\). Its first two factorial moments give its mean and variance. For the first moment,

E[X]=\sum_{k=1}^{\infty} k e^{-\theta}\frac{\theta^k}{k!} =\theta e^{-\theta}\sum_{j=0}^{\infty}\frac{\theta^j}{j!} =\theta.

For the second factorial moment, terms with \(k<2\) are zero, and shifting the summation index gives

E[X(X-1)] =\sum_{k=2}^{\infty} k(k-1)e^{-\theta}\frac{\theta^k}{k!} =\theta^2 e^{-\theta}\sum_{j=0}^{\infty}\frac{\theta^j}{j!} =\theta^2.

Since \(X^2=X(X-1)+X\), it follows that \(E[X^2]=\theta^2+\theta\), and hence \(\operatorname{Var}(X)=E[X^2]-(E[X])^2=\theta\). When \(\theta=0\), \(X=0\) with probability one, so both moment formulas remain valid.

Worked Example: Moments and Risk for Four Observations

Suppose \(n=4\) independent observations each have a Poisson distribution with mean \(\theta=3\). Calculate the bias, variance, and squared-error risk of \(\bar X\).

Each observation has mean \(3\) and variance \(3\). Therefore,

E[\bar X]=\frac{1}{4}\sum_{i=1}^{4}E[X_i]=3, \qquad \operatorname{Var}(\bar X) =\frac{1}{16}\sum_{i=1}^{4}\operatorname{Var}(X_i) =\frac{12}{16} =\frac{3}{4}.

The bias is \(E[\bar X]-\theta=3-3=0\). Thus the risk is the variance, \(R(3,\bar X)=3/4=0.75\). This is a risk calculation over repeated samples; it does not depend on which particular four counts happen to be observed.

Bias, Variance, and Risk of the Sample Mean

Independence makes the variance calculation simple: the covariance between distinct observations is zero, so the variance of their sum is the sum of their variances. The following result gives the full risk function, not just its value at one parameter point.

Theorem: If \(X_1,\ldots,X_n\) are independent and identically distributed as \(\operatorname{Poisson}(\theta)\), with \(\theta\geq0\), then the sample mean \(\bar X\) has bias \(0\), variance \(\theta/n\), and squared-error risk \(R(\theta,\bar X)=\theta/n\).

Proof. By linearity of expectation and the Poisson mean formula,

E_\theta[\bar X] =\frac{1}{n}\sum_{i=1}^n E_\theta[X_i] =\frac{1}{n}\sum_{i=1}^n\theta =\theta.

So the bias \(E_\theta[\bar X]-\theta\) is zero. For the variance, independence gives \(\operatorname{Cov}(X_i,X_j)=0\) whenever \(i\ne j\). Therefore,

\operatorname{Var}_\theta(\bar X) =\frac{1}{n^2}\operatorname{Var}_\theta\left(\sum_{i=1}^n X_i\right) =\frac{1}{n^2}\sum_{i=1}^n\operatorname{Var}_\theta(X_i) =\frac{n\theta}{n^2} =\frac{\theta}{n}.

For any estimator with finite second moment, squared-error risk equals variance plus squared bias: expand \(\delta-\theta=(\delta-E[\delta])+(E[\delta]-\theta)\), square, and take expectations. The cross-term is zero because \(E[\delta-E[\delta]]=0\). Applying this identity to the unbiased \(\bar X\) gives \(R(\theta,\bar X)=\theta/n\). At \(\theta=0\), all observations and \(\bar X\) are zero with probability one, so the same risk formula gives zero. \(\square\)

Worked Example: Comparing the Sample Mean with the Zero Estimate

Consider \(n=5\), and compare \(\bar X\) with the fixed estimate \(0\). The sample mean has risk \(\theta/5\). The zero estimate has risk \((0-\theta)^2=\theta^2\). Equality occurs when

\theta^2=\frac{\theta}{5} \quad\Longleftrightarrow\quad \theta\left(\theta-\frac{1}{5}\right)=0,

so the risks are equal at \(\theta=0\) and \(\theta=0.2\). At \(\theta=0.1\), the zero estimate has risk \(0.1^2=0.01\), while the sample mean has risk \(0.1/5=0.02\); the fixed estimate is better there. At \(\theta=0.4\), the risks are \(0.4^2=0.16\) and \(0.4/5=0.08\), respectively, so the sample mean is better. In general, for this comparison the zero estimate has lower risk when \(0<\theta<1/5\), and the sample mean has lower risk when \(\theta>1/5\).

Risk of a Fixed Estimate

A fixed estimate \(c\) is an estimator that returns the same number for every possible data set. Take \(c\geq0\), so the fixed value itself is a possible nonnegative rate estimate. Because it has no sampling variability, its error is entirely its fixed discrepancy from the true parameter.

Result: The risk of the constant estimator \(\delta_c(X_1,\ldots,X_n)=c\) is \(R(\theta,\delta_c)=(c-\theta)^2\). It does not depend on the sample size \(n\).

Derivation. Under squared-error loss, \(\delta_c-\theta=c-\theta\) for every sample. This is a nonrandom quantity, so taking its expectation leaves it unchanged:

R(\theta,\delta_c) =E_\theta[(c-\theta)^2] =(c-\theta)^2.

The fixed estimate is biased unless \(c=\theta\): its bias is \(E_\theta[\delta_c]-\theta=c-\theta\), and its variance is zero. Its risk is exactly its squared bias.

Where the Two Risks Cross

To compare the fixed estimate \(c\) with the sample mean, subtract their risk functions:

R(\theta,\delta_c)-R(\theta,\bar X) =(c-\theta)^2-\frac{\theta}{n} =\theta^2-\left(2c+\frac{1}{n}\right)\theta+c^2.

The difference is a quadratic in \(\theta\). When \(c>0\), its roots are

\theta_- = c+\frac{1}{2n}-\sqrt{\frac{c}{n}+\frac{1}{4n^2}}, \qquad \theta_+ = c+\frac{1}{2n}+\sqrt{\frac{c}{n}+\frac{1}{4n^2}}.

These roots follow from the quadratic formula. Both are positive: the center \(c+1/(2n)\) is positive and its square exceeds the squared half-width by \(c^2\), so the lower root is positive too. The quadratic opens upward. Consequently, its value is negative between the roots and positive outside them. The fixed estimate has smaller risk for \(\theta_-<\theta<\theta_+\); the sample mean has smaller risk for \(0\leq\theta<\theta_-\) and \(\theta>\theta_+\). At either root the risks are equal.

Worked Example: A Fixed Estimate of One

Let \(n=1\) and compare the observation \(X_1\), which is the sample mean in this case, with the fixed estimate \(c=1\). Their risks are \(R(\theta,\bar X)=\theta\) and \(R(\theta,\delta_1)=(1-\theta)^2\). Equality requires

(1-\theta)^2=\theta \quad\Longleftrightarrow\quad \theta^2-3\theta+1=0,

whose roots are \((3-\sqrt{5})/2\approx0.382\) and \((3+\sqrt{5})/2\approx2.618\). At \(\theta=0.5\), which is between the roots, the fixed estimate’s risk is \((1-0.5)^2=0.25\), while the sample mean’s risk is \(0.5\). At \(\theta=3\), which is above the upper root, the fixed estimate’s risk is \((1-3)^2=4\), while the sample mean’s risk is \(3\). These values verify the risk comparison on both sides of the crossing interval.

Why the Comparison Matters

The risk functions show that there is no single fixed \(c\geq0\) whose risk is no greater than the sample mean’s risk at every \(\theta\geq0\), with strict improvement somewhere. If \(c>0\), then at \(\theta=0\) the sample mean has risk zero while the fixed estimate has risk \(c^2>0\). If \(c=0\), then for every \(\theta>1/n\) its risk \(\theta^2\) exceeds the sample mean’s risk \(\theta/n\). Thus each fixed estimate loses somewhere, even though it may win over a range of parameter values.

This is a comparison of risk functions across parameter values, not a claim that one rule is best for every possible situation. The sample mean’s risk increases linearly in \(\theta\) and decreases with sample size \(n\); a fixed estimate’s risk is the squared distance from \(\theta\), regardless of \(n\). At the boundary \(\theta=0\), the sample mean’s risk is exactly zero. That boundary fact is important when judging any proposed uniform risk advantage.

Common Mistake

Confusing zero bias with zero risk. The sample mean is unbiased, but that does not make its risk zero. Its variance is \(\theta/n\), so its risk is positive whenever \(\theta>0\).
Claiming the fixed estimate is better because it has no variance. Zero variance does not mean zero risk: the fixed estimate has squared bias \((c-\theta)^2\), which can be large.
Comparing the estimators at only one value of \(\theta\). A fixed estimate can have lower risk at some parameter values and higher risk at others. A local comparison does not establish uniform superiority.

Key takeaway: Under independent \(\operatorname{Poisson}(\theta)\) sampling, the sample mean has bias zero and risk \(\theta/n\). A fixed estimate \(c\) has risk \((c-\theta)^2\). Their relative performance depends on \(\theta\); neither risk formula alone makes one estimator uniformly better.

Check Your Understanding

Use the risk formulas and reasoning from this tutorial to answer the following.

For \(n=8\), what are the bias, variance, and risk of \(\bar X\) as functions of \(\theta\)?
For a fixed estimate \(c=2\), write its squared-error risk and explain why it does not depend on \(n\).
For \(n=4\), compare the risks of \(\bar X\) and the fixed estimate \(0\) at \(\theta=0.1\).
Why does unbiasedness not imply that the sample mean has zero risk when \(\theta>0\)?
Explain why no fixed \(c\geq0\) uniformly improves on the sample mean over all \(\theta\geq0\).