Poisson Mean Estimation on a Finite Parameter Set

Why Finite Priors Make the Calculation Concrete

When a Poisson mean is known only to belong to a finite set, a prior assigns probabilities to those candidate values. Each observed count updates those probabilities, and under squared-error loss the resulting Bayes rule is the posterior mean. The remaining task is to calculate how well that rule performs on average and, when appropriate, to choose the prior weights that make its Bayes risk as large as possible.

This is a useful finite calculation even though the observation \(X\) can take infinitely many values. The likelihood at each count has a simple form, and the posterior mean is a ratio of finite sums. The Bayes risk is then a convergent sum over the possible counts.

Model, Prior, and Posterior Mean

Suppose \(X\mid\theta\sim\operatorname{Poisson}(\theta)\), where the parameter belongs to the finite set \(\{\theta_1,\ldots,\theta_k\}\), with \(\theta_i\geq 0\). Write the prior probabilities as \(w_i\geq0\), with \(\sum_{i=1}^k w_i=1\). For \(\theta_i>0\), the probability of observing \(x\in\{0,1,2,\ldots\}\) is \(f_{\theta_i}(x)=e^{-\theta_i}\theta_i^x/x!\). At \(\theta_i=0\), the distribution is concentrated at \(x=0\).

Definition: For a prior \(\pi=(w_1,\ldots,w_k)\), its Bayes estimator under squared-error loss is an estimator minimizing the prior-averaged risk \(E_\pi[(\delta(X)-\theta)^2]\). Its Bayes risk, denoted \(B(\pi)\), is this minimum value.

Define the prior-predictive probability of count \(x\) and the corresponding posterior mean numerator by

m_\pi(x)=\sum_{i=1}^k w_i f_{\theta_i}(x), \qquad a_\pi(x)=\sum_{i=1}^k w_i\theta_i f_{\theta_i}(x).

Whenever \(m_\pi(x)>0\), Bayes’ rule gives posterior probability \(w_i f_{\theta_i}(x)/m_\pi(x)\) to \(\theta_i\). Thus the posterior mean is

\delta_\pi(x) = \frac{a_\pi(x)}{m_\pi(x)} = \frac{\sum_{i=1}^k w_i\theta_i f_{\theta_i}(x)} {\sum_{i=1}^k w_i f_{\theta_i}(x)}.

If \(m_\pi(x)=0\), that count has prior-predictive probability zero, so the estimator’s value there does not affect its Bayes risk. It may be defined arbitrarily at such counts.

Worked Example: A Prior on Means 1 and 3

Let \(\theta\in\{1,3\}\), with prior probabilities \(0.7\) and \(0.3\), respectively. Calculate the posterior mean at counts \(x=0\) and \(x=1\).

At \(x=0\), the likelihoods are \(e^{-1}\) and \(e^{-3}\). The posterior-mean numerator must include the parameter value multiplying each likelihood:

\delta_\pi(0) = \frac{0.7(1)e^{-1}+0.3(3)e^{-3}} {0.7e^{-1}+0.3e^{-3}} = \frac{0.7e^{-1}+0.9e^{-3}} {0.7e^{-1}+0.3e^{-3}} \approx 1.1096.

At \(x=1\), the likelihoods are \(e^{-1}\) and \(3e^{-3}\). The numerator again multiplies each likelihood by its parameter value:

\delta_\pi(1) = \frac{0.7(1)e^{-1}+0.3(3)(3e^{-3})} {0.7e^{-1}+0.3(3e^{-3})} = \frac{0.7e^{-1}+2.7e^{-3}} {0.7e^{-1}+0.9e^{-3}} \approx 1.2964.

The count \(x=1\) shifts posterior probability toward the larger mean, so the posterior mean rises from about \(1.1096\) to \(1.2964\). The factor \(\theta_i\) in the numerator is essential: without it, the numerator would equal the denominator and the purported posterior mean would incorrectly be 1 at every count.

Bayes Risk Under Squared-Error Loss

The posterior mean minimizes conditional expected squared error. To see this directly, let \(q_i\) be the posterior probability of \(\theta_i\) after observing \(x\), and let \(\mu=\sum_i q_i\theta_i\). For any action \(d\),

\sum_i q_i(d-\theta_i)^2 = (d-\mu)^2+\sum_i q_i(\theta_i-\mu)^2.

The second term does not depend on \(d\), and the first is minimized at \(d=\mu\). This proves that \(\delta_\pi(x)\) is a Bayes rule whenever the posterior is defined. Since the parameter set is finite, the required posterior moments are finite.

The conditional minimum is the posterior variance. Averaging it over the prior-predictive distribution gives a convenient risk formula. Define \(b_\pi(x)=\sum_i w_i\theta_i^2 f_{\theta_i}(x)\). At counts with \(m_\pi(x)>0\), the contribution to Bayes risk is \(b_\pi(x)-a_\pi(x)^2/m_\pi(x)\). Therefore

Result: For a finite prior on a finite set of Poisson means, the Bayes estimator under squared-error loss is the posterior mean, and its Bayes risk is

B(\pi) = \sum_{\substack{x\geq0\\m_\pi(x)>0}} \left[ b_\pi(x)-\frac{a_\pi(x)^2}{m_\pi(x)} \right].

A count with \(m_\pi(x)=0\) contributes zero.

Each summand is nonnegative: it is \(m_\pi(x)\) times a posterior variance. The series converges because its summands are bounded above by \(b_\pi(x)\), and

\sum_{x\geq0} b_\pi(x) = \sum_i w_i\theta_i^2 < \infty.

Worked Example: Computing Risk Contributions for Three Means

Take \(\theta\in\{0,1,2\}\), with prior weights \(0.2,0.5,0.3\). For \(x=0,1,2\), compute the predictive probabilities \(m_\pi(x)\), posterior-mean numerators \(a_\pi(x)\), and risk contributions \(b_\pi(x)-a_\pi(x)^2/m_\pi(x)\).

Using the Poisson probabilities at means 0, 1, and 2 gives

\begin{aligned} m_\pi(0)&=0.2+0.5e^{-1}+0.3e^{-2},& a_\pi(0)&=0.5e^{-1}+0.6e^{-2},& b_\pi(0)&=0.5e^{-1}+1.2e^{-2},\\ m_\pi(1)&=0.5e^{-1}+0.6e^{-2},& a_\pi(1)&=0.5e^{-1}+1.2e^{-2},& b_\pi(1)&=0.5e^{-1}+2.4e^{-2},\\ m_\pi(2)&=0.25e^{-1}+0.6e^{-2},& a_\pi(2)&=0.25e^{-1}+1.2e^{-2},& b_\pi(2)&=0.25e^{-1}+2.4e^{-2}. \end{aligned}

For example, at \(x=2\), the likelihood under mean 1 is \(e^{-1}/2\), and under mean 2 it is \(2e^{-2}\). Their contributions to \(m_\pi(2)\) are \(0.5(e^{-1}/2)=0.25e^{-1}\) and \(0.3(2e^{-2})=0.6e^{-2}\). Multiplication by the means gives the displayed \(a_\pi(2)\); multiplication by their squares gives \(b_\pi(2)\).

The posterior means at these counts are the ratios \(a_\pi(x)/m_\pi(x)\), approximately \(0.6245\), \(1.3063\), and \(1.4689\), respectively. Substituting the same quantities into the risk formula gives contributions of approximately \(0.1808\), \(0.0563\), and \(0.0431\). The full Bayes risk is the exact convergent sum

B(\pi) = \sum_{x=0}^{\infty} \left[ 0.5\frac{e^{-1}}{x!} + 1.2\frac{2^x e^{-2}}{x!} - \frac{\left(0.5\frac{e^{-1}}{x!} +0.6\frac{2^x e^{-2}}{x!}\right)^2} {0.2\,\mathbf{1}_{\{x=0\}} +0.5\frac{e^{-1}}{x!} +0.3\frac{2^x e^{-2}}{x!}} \right].

Here \(\mathbf{1}_{\{x=0\}}\) is 1 when \(x=0\) and 0 otherwise. The expression follows from the three prior components; the mean-zero component contributes only at \(x=0\). The first three risk contributions are part of the total, not the entire risk, because Poisson counts above 2 also have positive predictive probability.

Optimizing Prior Weights in a Two-Point Model

For a fixed finite parameter set, the prior weights range over the probability simplex. The largest Bayes risk over this set of priors is the relevant prior-optimization target; a prior attaining that largest value is called least favorable for the specified decision problem. Here the calculation can be done exactly for means 0 and 1.

Worked Example: The Least-Favorable Weight on Means 0 and 1

Let \(P(\theta=1)=p\) and \(P(\theta=0)=1-p\), for \(0\leq p\leq1\). Write \(c=e^{-1}\). If \(X>0\), the observation rules out \(\theta=0\), so the posterior mean is 1. At \(X=0\),

\delta_p(0)=\frac{pc}{1-p+pc}.

The only nonzero posterior uncertainty occurs at \(X=0\). Its predictive probability is \(1-p+pc\), and the posterior probability of \(\theta=1\) there is \(q=pc/(1-p+pc)\). A Bernoulli variable with success probability \(q\) has variance \(q(1-q)\), so

B(p) =(1-p+pc)q(1-q) = \frac{cp(1-p)}{1-p+pc}.

Differentiate on \(0<p<1\). With \(d=1-c\), the quotient rule gives

B'(p) = \frac{c\{1-2p+(1-c)p^2\}}{(1-(1-c)p)^2}.

The root in \((0,1)\) is \(p^*=1/(1+\sqrt c)\); the other root is outside this interval. Since \(B(0)=B(1)=0\), and the derivative changes from positive to negative at \(p^*\), this is the maximizing prior weight. Its Bayes risk is

B(p^*)=\frac{c}{(1+\sqrt c)^2} =\frac{e^{-1}}{(1+e^{-1/2})^2} \approx 0.1425.

As a check, the posterior probability of \(\theta=1\) at zero under this prior is \(q=\sqrt c/(1+\sqrt c)\). The risk at \(\theta=0\) is \(q^2\), since \(X=0\) surely. The risk at \(\theta=1\) is \(c(1-q)^2\), since only the event \(X=0\), of probability \(c\), produces an error. Both equal \(c/(1+\sqrt c)^2\), agreeing with the optimized Bayes risk.

Why the Prior Optimization Is Well-Behaved

For any fixed estimator, its prior-averaged risk is linear in the prior weights. The Bayes risk is the infimum of these linear functions over estimators, so it is concave in the prior. For a finite parameter set it is also continuous on the prior simplex: each finite partial sum in the risk formula is continuous, including at zero predictive probabilities, and the remaining tail is bounded uniformly by \(M^2\max_i P_{\theta_i}(X>N)\), where \(M=\max_i\theta_i\). This bound tends to zero as \(N\) tends to infinity. The simplex is compact, so a prior maximizing Bayes risk exists.

These facts justify searching across the full range of prior weights, including its boundary. For a small number of candidate means, one can substitute the weights into the posterior-mean and risk formulas and optimize directly. With more candidates, the risk remains a well-defined objective, but its evaluation involves a sum over counts that may be truncated for numerical work. The contribution of omitted counts can be bounded using the corresponding Poisson tail probabilities.

Common Pitfall

Omitting the parameter in the posterior-mean numerator. The posterior mean is not the ratio of the weighted likelihood sum to itself. Its numerator is \(\sum_i w_i\theta_i f_{\theta_i}(x)\); omitting \(\theta_i\) incorrectly returns 1 whenever the numerator is written as the same sum as the denominator.
Stopping the risk sum at the largest parameter value. The possible means form a finite set, but the Poisson observation is not bounded by that set. Counts larger than every candidate mean still have positive probability and must be included or covered by a tail bound.
Confusing posterior variance with Bayes risk. Posterior variance is conditional on the observed count. Bayes risk averages that variance over the prior-predictive distribution, including the probability of each count.

Key takeaway: For finite Poisson parameter sets and squared-error loss, compute the posterior mean by weighting each likelihood by both its prior probability and its parameter value. Then average posterior variance over the predictive distribution and optimize that Bayes risk over the prior weights.

Check Your Understanding

Try these questions using the formulas and reasoning developed above.

For means \(\theta_1,\theta_2\) with prior weights \(w_1,w_2\), write the posterior mean after observing \(x\). Which factor is absent if one mistakenly obtains the ratio of the denominator to itself?
Why does an observation \(X>0\) identify \(\theta=1\) with posterior probability 1 in the two-point model with means 0 and 1?
In the prior on \(\{0,1,2\}\), why does the mean-zero component contribute to the predictive probability at \(x=0\) but not at \(x=1\)?
Explain why a risk calculation using only \(x=0,1,2\) is not the full Bayes risk in a Poisson model.
For the two-point prior example, verify the equal risks at \(\theta=0\) and \(\theta=1\) using \(q=\sqrt{e^{-1}}/(1+\sqrt{e^{-1}})\).