Two-Point Normal Means with Squared-Error Loss

Introduction

When a normal observation can come from either of two possible means, the estimation problem is simple to state but still illustrates a central decision-theoretic idea: the prior changes how evidence in the observation is balanced against the relative plausibility of the two parameter values. Under squared-error loss, the Bayes estimator is the posterior mean. Its endpoint risks can then be compared, and in the common-variance normal model equal prior weights produce a rule with equal risks at both endpoints.

The calculations below make that conclusion precise. They also distinguish two claims that are easy to conflate: a posterior-mean rule is Bayes for its chosen prior, while equal endpoint risks require a particular choice of prior weights.

The Two-Point Normal Model

Let the parameter have two possible values, \(\theta_L<\theta_U\), and suppose the observation has a normal distribution with known variance \(\sigma^2>0\):

X\mid\theta=\theta_L\sim N(\theta_L,\sigma^2), \qquad X\mid\theta=\theta_U\sim N(\theta_U,\sigma^2).

Write \(d=\theta_U-\theta_L>0\) for the distance between the means and \(c=(\theta_L+\theta_U)/2\) for their midpoint. Use the prior \(P(\theta=\theta_U)=p\) and \(P(\theta=\theta_L)=1-p\), where \(0\leq p\leq1\). The loss for an estimate \(a\) is \((a-\theta)^2\). For a rule \(\delta\), its risk at \(\theta\) is \(R(\theta,\delta)=E_\theta[(\delta(X)-\theta)^2]\).

Definition: The posterior probability of the upper mean after observing \(X=x\) is \(q_p(x)=P(\theta=\theta_U\mid X=x)\). Under squared-error loss, the Bayes estimator is the posterior mean \(E[\theta\mid X=x]\), provided the posterior second moment is finite. Here the parameter has finite support, so that condition holds automatically.

For \(0<p<1\), the likelihood ratio of the upper mean to the lower mean is

\frac{f_U(x)}{f_L(x)} = \exp\left\{ \frac{(x-\theta_L)^2-(x-\theta_U)^2}{2\sigma^2} \right\} = \exp\left\{\frac{d(x-c)}{\sigma^2}\right\},

where \(f_L\) and \(f_U\) are the respective normal densities. Bayes' rule therefore gives posterior odds equal to prior odds multiplied by this likelihood ratio. Thus

q_p(x) = \frac{p\exp\{d(x-c)/\sigma^2\}} {(1-p)+p\exp\{d(x-c)/\sigma^2\}}, \qquad \delta_p(x)=\theta_L+dq_p(x).

At \(p=0\) or \(p=1\), the prior puts all its mass on one mean and the Bayes estimator is the corresponding constant estimate. The displayed expression for \(q_p(x)\) also gives these limits.

Why the Posterior Mean Is the Bayes Rule

The posterior-mean claim follows by minimizing conditional expected loss separately at each observation. For any fixed observed value \(x\), let \(m(x)=E[\theta\mid X=x]\). For any real action \(a\), expand the square around \(m(x)\):

E[(a-\theta)^2\mid X=x] = (a-m(x))^2+\operatorname{Var}(\theta\mid X=x).

The cross term vanishes because \(E[\theta-m(x)\mid X=x]=0\). The conditional variance does not depend on \(a\), and the other term is minimized at \(a=m(x)\). In this model \(m(x)=\theta_L+dq_p(x)\). Integrating the conditional loss over the marginal distribution of \(X\) shows that this rule minimizes prior-weighted risk. No interchange of a limit and an integral is needed: the decomposition is a pointwise identity, and all terms are integrable for this finite-parameter normal model.

Worked Example: Posterior Odds for Means Zero and Two

Suppose \(X\mid\theta\sim N(\theta,1)\), with \(\theta\in\{0,2\}\), and assign prior probability \(p=1/4\) to \(\theta=2\). Here \(d=2\), \(c=1\), and the likelihood ratio is \(\exp\{2(x-1)\}\). At \(x=2\), it equals \(\exp(2)\).

The prior odds in favor of \(\theta=2\) are \((1/4)/(3/4)=1/3\). The posterior odds are therefore \(\exp(2)/3\), so

q_{1/4}(2)=\frac{\exp(2)}{3+\exp(2)} =\frac{7.3890561}{10.3890561} \approx 0.7112.

The posterior mean, and hence the Bayes estimate, is \(0+2q_{1/4}(2)\approx1.4225\). This estimate lies between the two possible means, as a posterior average must.

Worked Example: A Smaller Prior Weight

Keep the same model and observation, but take \(p=1/10\). The prior odds are \(1/9\), so at \(x=2\) the posterior odds are \(\exp(2)/9\). Consequently,

q_{1/10}(2)=\frac{\exp(2)}{9+\exp(2)} =\frac{7.3890561}{16.3890561} \approx 0.4509, \qquad \delta_{1/10}(2)=2q_{1/10}(2)\approx0.9017.

The observation favors the upper mean because its likelihood ratio exceeds one, but the prior strongly favors zero. The posterior probability of the upper mean is therefore below one half, and the posterior mean is below one. The result also shows concretely how a smaller prior weight pulls the Bayes estimate downward.

Endpoint Risks and Equal Prior Weights

For any \(p\), the risk at each possible mean can be written directly using the posterior probability:

R(\theta_L,\delta_p) =d^2\int q_p(x)^2 f_L(x)\,dx, \qquad R(\theta_U,\delta_p) =d^2\int (1-q_p(x))^2 f_U(x)\,dx.

These formulas follow because \(\delta_p(x)-\theta_L=dq_p(x)\) and \(\delta_p(x)-\theta_U=-d(1-q_p(x))\). They provide an explicit equal-risk equation for a general prior: set the two integrals equal and solve for \(p\). In the common-variance normal model, the equal weights \(p=1/2\) solve it for every pair of endpoints.

Theorem: If \(X\mid\theta\sim N(\theta,\sigma^2)\), with \(\theta\in\{\theta_L,\theta_U\}\), \(\theta_L<\theta_U\), and \(\sigma^2>0\), then the posterior-mean rule under prior weights \(P(\theta=\theta_L)=P(\theta=\theta_U)=1/2\) has equal risks at the two parameter values.

Proof. For the equal-weight prior, the posterior odds are just the likelihood ratio. Since replacing \(x\) by \(2c-x\) changes \(x-c\) to \(-(x-c)\), the likelihood ratio is inverted. Hence

q_{1/2}(2c-x)=1-q_{1/2}(x), \qquad \delta_{1/2}(2c-x)=\theta_L+\theta_U-\delta_{1/2}(x).

The normal densities also transform into one another: \(f_U(2c-x)=f_L(x)\). In the integral for \(R(\theta_U,\delta_{1/2})\), make the substitution \(x=2c-y\). Its Jacobian has absolute value one, so

\begin{aligned} R(\theta_U,\delta_{1/2}) &=\int(\delta_{1/2}(x)-\theta_U)^2 f_U(x)\,dx\\ &=\int(\delta_{1/2}(2c-y)-\theta_U)^2 f_U(2c-y)\,dy\\ &=\int(\theta_L-\delta_{1/2}(y))^2 f_L(y)\,dy\\ &=R(\theta_L,\delta_{1/2}). \end{aligned}

The last equality holds because squaring removes the sign of the estimation error. All integrals are finite: the posterior mean is between the two finite parameter values, so its squared error is bounded by \(d^2\). This proves the equal-risk assertion. \(\square\)

Worked Example: Equal Risks for Means Minus One and One

Let \(X\mid\theta\sim N(\theta,1)\), with \(\theta\in\{-1,1\}\), and assign prior probability \(1/2\) to each mean. Here \(d=2\) and \(c=0\), so the posterior probability of \(\theta=1\) is

q_{1/2}(x)=\frac{\exp(2x)}{1+\exp(2x)}.

The estimator is \(2q_{1/2}(x)-1=\tanh(x)\). At \(\theta=-1\), its risk is \(E_{-1}[(\tanh(X)+1)^2]\); at \(\theta=1\), its risk is \(E_1[(\tanh(X)-1)^2]\). Under the transformation \(X\mapsto-X\), a \(N(1,1)\) observation becomes a \(N(-1,1)\) observation, and \(\tanh(-X)=-\tanh(X)\). Therefore the two squared-error expectations are equal. This works out the equal-risk property for a concrete estimator, without requiring a closed-form evaluation of the common risk.

Why Equal Risks Matter

The equal-risk property gives a useful minimax certificate. The maximum risk of the equal-prior rule is its common endpoint risk, say \(r\). For any other rule, its maximum risk is at least its average risk under the equal prior. By the definition of Bayes risk, that prior-average risk is at least \(r\), the Bayes risk achieved by the posterior-mean rule. Thus no rule can have maximum risk below \(r\), while the equal-prior rule has maximum risk exactly \(r\). It is therefore minimax.

This argument uses the equal prior specifically. An arbitrary prior still gives its own Bayes estimator, but its endpoint risks need not match. Equal prior weights are what make the reflection argument apply in this two-point, common-variance normal model.

Common Mistake

Wrong reasoning: “The Bayes estimator is the posterior mean, so its risks at the two means must be equal.” Being Bayes only says the rule minimizes risk averaged under the selected prior; it does not force the separate risks to match.
Correction: Write the two endpoint risks separately and identify why they are equal. Here, equal prior weights make the posterior rule transform into its reflected counterpart, while the normal sampling distributions reflect into one another. For other prior weights, that argument does not apply.
Numerical check: When calculating posterior odds, multiply prior odds by the likelihood ratio before converting odds to probability. For example, with \(p=1/4\) above, the posterior probability is about \(0.7112\), not \(0.7117\).

Key takeaway: For two normal means with a common known variance and squared-error loss, the Bayes rule is the posterior mean. Equal prior weights give equal risks at the two means, and this equal-risk Bayes rule is minimax.

Check Your Understanding

Use the posterior-odds formula and risk argument to answer the following.

For means \(\theta_L\) and \(\theta_U\), explain why the log likelihood ratio is linear in the observation \(x\).
In the model with means zero and two and variance one, what is the posterior probability of the upper mean at \(x=1\) under prior weight \(p=1/4\)?
Why does squared-error loss lead to the posterior mean rather than the posterior mode?
State the transformation that establishes equal endpoint risks for the equal-prior rule.
Explain why an equal-risk Bayes rule is minimax in this two-point problem.