Normal-Mean Minimaxity Through Diffuse Priors

Why Diffuse Priors Reveal a Minimax Value

A minimax calculation has two sides. One constructs a rule whose worst-case risk is no larger than a proposed value, and one proves that no rule can make its worst-case risk smaller. In the normal-mean problem, the sample mean supplies a particularly simple upper bound: its squared-error risk is constant in the unknown mean. Proper normal priors with increasingly large variance supply lower bounds that approach that same constant.

The argument is useful because it separates the role of a prior from the role of a minimax procedure. Each prior gives a lower bound through its Bayes risk; the prior itself need not produce a minimax estimator. Here the lower bounds become sharp in the limit, and comparison with the sample mean identifies the minimax value.

Model, Risk, and Minimax Value

The parameter space is the entire real line, and there is no restriction on the action: an estimate may be any real number. The sample mean is \(\bar X=n^{-1}\sum_{i=1}^n X_i\). Since \(E_\theta[\bar X]=\theta\) and \(\operatorname{Var}_\theta(\bar X)=\sigma^2/n\),

This proves an upper bound of \(\sigma^2/n\) on the minimax value. It does not by itself show that another rule cannot have a smaller worst-case risk. To obtain that lower bound, put a proper normal distribution on the unknown mean and calculate its Bayes risk.

Bayes Risk Under a Proper Normal Prior

For \(\tau^2>0\), assign the mean the proper prior distribution \(\theta\sim N(0,\tau^2)\). A prior is proper when it is a probability distribution; here its density integrates to one for every finite positive \(\tau^2\). The Bayes risk is the smallest risk averaged over this prior: $$ r_\tau=\inf_\delta \int_{\mathbb{R}}R(\theta,\delta)\,\pi_\tau(\theta)\,d\theta, $$ where \(\pi_\tau\) is the \(N(0,\tau^2)\) density. We now derive this quantity rather than assuming a formula for it.

For observed values \(x_1,\ldots,x_n\), the likelihood as a function of \(\theta\) is proportional to \(\exp\{-\sum_i(x_i-\theta)^2/(2\sigma^2)\}\). The identity \(\sum_i(x_i-\theta)^2=\sum_i(x_i-\bar x)^2+n(\theta-\bar x)^2\) shows that the part depending on \(\theta\), after multiplication by the prior density, is proportional to $$ \exp\left\{-\frac{1}{2}\left[ \left(\frac{n}{\sigma^2}+\frac{1}{\tau^2}\right)\theta^2 -2\frac{n\bar x}{\sigma^2}\theta \right]\right\}. $$ Completing the square gives the posterior distribution $$ \theta\mid x_1,\ldots,x_n\sim N(m_\tau(x),v_\tau), \qquad v_\tau=\left(\frac{n}{\sigma^2}+\frac{1}{\tau^2}\right)^{-1}, \qquad m_\tau(x)=v_\tau\frac{n\bar x}{\sigma^2}. $$

For any fixed observed data and any real action \(a\), the posterior expected squared loss decomposes as $$ E[(a-\theta)^2\mid x] =(a-m_\tau(x))^2+v_\tau. $$ Indeed, write \(a-\theta=(a-m_\tau)+(m_\tau-\theta)\), expand the square, and use \(E[m_\tau-\theta\mid x]=0\) and \(E[(\theta-m_\tau)^2\mid x]=v_\tau\). The expression is minimized at \(a=m_\tau(x)\), and its minimum is \(v_\tau\), independent of the observed data. Consequently the Bayes estimator is the posterior mean and the Bayes risk is $$ r_\tau=v_\tau =\left(\frac{n}{\sigma^2}+\frac{1}{\tau^2}\right)^{-1} =\frac{\sigma^2\tau^2}{\sigma^2+n\tau^2}. $$

Why the Bayes Risk Is a Minimax Lower Bound

Proof. The first inequality holds because an average of a nonnegative function with respect to a probability distribution cannot exceed its supremum: \(R(\theta,\delta)\leq\sup_\vartheta R(\vartheta,\delta)\) for every \(\theta\), and integration preserves this inequality. If the supremum is infinite, the inequality remains valid. The second inequality follows from the definition of Bayes risk as the infimum of integrated risk over all rules. These integrated risks are well-defined as nonnegative extended integrals; exchanging the order of expectation and integration, when needed to express one joint expectation, is justified by Tonelli’s theorem. The preceding posterior calculation gives the stated normal-prior Bayes risk. \(\square\)

This lower bound applies to each rule, not only to the Bayes estimator. In particular, taking the infimum over rules shows that the minimax value is at least \(r_\tau\) for every \(\tau^2>0\). As \(\tau^2\) increases, the prior places its mass over a wider range of means, and $$ \lim_{\tau^2\to\infty}r_\tau =\lim_{\tau^2\to\infty}\frac{\sigma^2\tau^2}{\sigma^2+n\tau^2} =\frac{\sigma^2}{n}. $$ The limit is a sharp lower bound: a number that is at least every \(r_\tau\) must be at least their limit. Combining this with the sample mean’s upper bound proves the minimax claim.

Proof. Since \(R(\theta,\bar X)=\sigma^2/n\) for all \(\theta\), the minimax value is at most \(\sigma^2/n\). For every \(\tau^2>0\), the Bayes-risk lower-bound argument gives \(\inf_\delta\sup_\theta R(\theta,\delta)\geq \sigma^2\tau^2/(\sigma^2+n\tau^2)\). Taking \(\tau^2\to\infty\) yields a lower bound of \(\sigma^2/n\). The two bounds agree, and the sample mean attains the resulting value. \(\square\)

Interpreting the Diffuse-Prior Limit

The limiting argument uses a sequence of proper priors, each with a finite variance. It does not require treating a flat distribution on the real line as a probability distribution. In fact, a constant density on \(\mathbb{R}\) cannot be normalized to integrate to one, so it is an improper prior. Such a prior can sometimes be useful for other calculations, but it cannot directly be inserted into the probability-average inequality above.

Nor does the limit say that the posterior mean equals the sample mean for every fixed finite prior variance. For \(N(0,\tau^2)\), the posterior mean is $$ m_\tau(x)=\frac{n\tau^2}{\sigma^2+n\tau^2}\bar x. $$ The multiplier is less than one for finite \(\tau^2\), and tends to one only as \(\tau^2\to\infty\). The minimax conclusion comes from the limiting Bayes risks matching the sample mean’s worst-case risk, not from claiming that one finite-prior Bayes rule is already minimax.

Common Pitfall

• Wrong reasoning: “A very diffuse prior is flat, so its Bayes risk is exactly the minimax value.” A flat density on the real line is not a probability distribution, and the Bayes-risk lower-bound argument requires a proper prior.
• Correction: Use proper \(N(0,\tau^2)\) priors with finite \(\tau^2\), establish the lower bound for each one, and then take the limit of those numerical bounds. The conclusion follows because each bound is valid before the limit is taken.
• Wrong reasoning: “The sample mean has constant risk, so it must be minimax.” Constant risk alone proves an upper bound only.
• Correction: Pair that upper bound with a lower bound valid for every rule. Here the Bayes risks of the diffuse-prior sequence converge to the same value.