Symmetry and Concavity in Two-Point Normal Means

The Symmetric Two-Point Problem

Suppose one observation $X$ has a normal distribution with known variance and a mean restricted to two symmetric possibilities. We use squared-error loss. The question here is not merely how to estimate the mean for a chosen prior; it is whether one prior is hardest for a decision rule to handle. In this problem, two properties answer that question: reflecting the model leaves it unchanged, and Bayes risk is concave as the prior weight varies.

Definition: Let $a>0$ and $\sigma^2>0$, and suppose $X\mid\theta\sim N(\theta,\sigma^2)$, where $\theta\in\{-a,a\}$. An estimator is a measurable rule $\delta:\mathbb R\to\mathbb R$, with loss $L(\theta,\delta(x))=(\delta(x)-\theta)^2$. Write $R(\theta,\delta)=E_\theta[(\delta(X)-\theta)^2]$. For $p\in[0,1]$, let $\pi_p$ assign probability $p$ to $a$ and $1-p$ to $-a$. Its Bayes risk is $$ r(p)=\inf_\delta\{pR(a,\delta)+(1-p)R(-a,\delta)\}. $$ A prior is least favorable within this family if its Bayes risk maximizes $r(p)$ over $p\in[0,1]$.

The densities under the two parameter values are $f_+(x)=\frac{1}{\sqrt{2\pi}\sigma}\exp[-(x-a)^2/(2\sigma^2)]$ and $f_-(x)=\frac{1}{\sqrt{2\pi}\sigma}\exp[-(x+a)^2/(2\sigma^2)]$. In particular, $f_+(x)=f_-(-x)$. That identity is the distributional reflection symmetry at the heart of the argument.

Bayes Rules and Posterior Risk

For $0<p<1$, the posterior probability of the positive mean after observing $X=x$ is

q_p(x)=P(\theta=a\mid X=x) =\frac{p f_+(x)}{p f_+(x)+(1-p)f_-(x)} =\frac{1}{1+\dfrac{1-p}{p}\exp(-2ax/\sigma^2)}.

The last equality follows because $f_+(x)/f_-(x)=\exp(2ax/\sigma^2)$. Under squared-error loss, the action minimizing posterior expected loss is the posterior mean. Indeed, for any real action $d$,

$$ q_p(x)(d-a)^2+(1-q_p(x))(d+a)^2 =(d-a(2q_p(x)-1))^2+4a^2q_p(x)(1-q_p(x)). $$

The squared term is minimized at $d=a(2q_p(x)-1)$, and the remaining term is the posterior variance. Thus a Bayes rule is $\delta_p(x)=a(2q_p(x)-1)$, and the Bayes risk is the prior-predictive expectation of $4a^2q_p(X)(1-q_p(X))$. At $p=0$ or $p=1$, the mean is known and the Bayes risk is zero.

Worked Example: Posterior Mean at a Given Observation

Take $a=1$, $\sigma^2=1$, and $p=1/4$, so the prior odds for $\theta=1$ against $\theta=-1$ are $1:3$. At observation $x=1/2$, the likelihood ratio is $\exp(2ax/\sigma^2)=\exp(1)=e$. Therefore

q_{1/4}(1/2)=\frac{e}{3+e}\approx 0.4754,\qquad \delta_{1/4}(1/2)=2q_{1/4}(1/2)-1 =\frac{e-3}{e+3}\approx -0.0492.

The negative estimate is consistent with the calculation: at this observation, the likelihood evidence for the positive mean is not quite strong enough to overcome the prior odds of $3:1$ in favor of $-1$. The posterior variance, and hence the posterior expected squared error of the Bayes action, is $4q(1-q)=12e/(3+e)^2\approx 0.9976$.

Reflection Makes Bayes Risk Symmetric

To compare prior weights $p$ and $1-p$, reflect both the observation and the action. Given a rule $\delta$, define its reflected rule by $\delta^*(x)=-\delta(-x)$. This transformation preserves measurability and is a bijection on the class of rules: applying it twice returns $\delta$.

Result: For every $p\in[0,1]$, the Bayes-risk function satisfies $r(p)=r(1-p)$.

Proof. If $X$ has density $f_+$, then $-X$ has density $f_-$, since $f_+(-x)=f_-(x)$. Consequently, $R(a,\delta^*)=R(-a,\delta)$. The same reflection gives $R(-a,\delta^*)=R(a,\delta)$. The prior-weighted risk of $\delta^*$ under $\pi_p$ is therefore $pR(-a,\delta)+(1-p)R(a,\delta)$, which is the prior-weighted risk of $\delta$ under $\pi_{1-p}$. Taking infima over rules on both sides, using the bijection $\delta\leftrightarrow\delta^*$, gives $r(p)=r(1-p)$. This also holds at the endpoints, where both reflected Bayes risks are zero. $\square$

Worked Example: Reflecting a Prior and a Rule

Let $a=2$, $\sigma^2=1$, and compare $p=0.8$ with $1-p=0.2$. At $x=0$, both normal likelihoods are equal, so the posterior probability of $\theta=2$ is $p$. The Bayes estimate is therefore $2(2p-1)$: it is $1.2$ under $p=0.8$, and $-1.2$ under $p=0.2$.

More generally, $q_{1-p}(-x)=1-q_p(x)$, as follows by reflecting the likelihoods and interchanging the prior weights. Hence $\delta_{1-p}(-x)=-\delta_p(x)$. The two rules are reflections of one another, and their Bayes risks are equal: $r(0.8)=r(0.2)$. The equality concerns the optimized average risks, not necessarily the risk of an arbitrary fixed rule.

Concavity Places a Maximum at Equal Weights

Bayes risk is concave in the prior weight. The reason is that, for each fixed rule, its prior-weighted risk is affine in $p$, and taking the infimum over rules preserves concavity. The argument is included here to make the step used in locating the maximum explicit.

Result: For $p_1,p_2\in[0,1]$ and $0\leq\lambda\leq1$, $$ r(\lambda p_1+(1-\lambda)p_2) \geq \lambda r(p_1)+(1-\lambda)r(p_2). $$

Proof. For any rule $\delta$, its risk at prior weight $\lambda p_1+(1-\lambda)p_2$ is \[ \lambda\{p_1R(a,\delta)+(1-p_1)R(-a,\delta)\} +(1-\lambda)\{p_2R(a,\delta)+(1-p_2)R(-a,\delta)\}. \] Each expression in braces is at least its infimum over rules, respectively $r(p_1)$ and $r(p_2)$. Thus the displayed risk is at least $\lambda r(p_1)+(1-\lambda)r(p_2)$ for every $\delta$. Taking its infimum proves the inequality. Rules with infinite risk do not affect the infimum; constant rules have finite risk here. $\square$

Theorem: The equal-weight prior $\pi_{1/2}$ maximizes Bayes risk over all priors $\pi_p$, $0\leq p\leq1$. Thus it is least favorable within this two-point prior family.

Proof. Apply concavity with $\lambda=1/2$ to the two weights $p$ and $1-p$. Their midpoint is $1/2$, so

r(1/2)\geq \frac{r(p)+r(1-p)}{2}=r(p),

where the equality uses reflection symmetry. This holds for every $p$, proving the claim. The argument establishes that equal weights are a maximizer; symmetry and concavity alone do not establish that no other weight also maximizes the risk. $\square$

Worked Example: Comparing Unequal Prior Weights

Suppose $a=1$ and $\sigma^2=1$, and consider $p=1/4$. Concavity and symmetry give

r(1/2)\geq \frac{r(1/4)+r(3/4)}{2} =\frac{r(1/4)+r(1/4)}{2}=r(1/4).

Thus no numerical integration of the normal-mixture risk is needed to establish this comparison. The same argument applies to every prior weight, including weights close to zero or one. At the endpoints themselves, $r(0)=r(1)=0$, because the parameter is known exactly under either endpoint prior.

Why the Equal-Weight Rule Is Minimax

The least-favorable-prior conclusion also gives a direct minimax certificate in this model. At equal weights, the posterior mean is $\delta_{1/2}(x)=a\tanh(ax/\sigma^2)$, which is odd: $\delta_{1/2}(-x)=-\delta_{1/2}(x)$. Reflection therefore makes its two endpoint risks equal. Their common value is their equally weighted average, which is exactly $r(1/2)$.

Result: For squared-error loss in this symmetric two-point normal-mean problem, $\delta_{1/2}$ is minimax and its maximum risk equals $r(1/2)$.

Proof. For any rule $\delta$, its maximum endpoint risk is at least its average endpoint risk, and that average is at least the Bayes risk: \[ \max\{R(a,\delta),R(-a,\delta)\} \geq \tfrac12R(a,\delta)+\tfrac12R(-a,\delta) \geq r(1/2). \] Taking the infimum over rules shows that the minimax risk is at least $r(1/2)$. For $\delta_{1/2}$, oddness of the rule and reflection of the normal distributions give $R(a,\delta_{1/2})=R(-a,\delta_{1/2})$. Its average risk is its Bayes risk $r(1/2)$, so each endpoint risk equals $r(1/2)$. Its maximum risk is therefore $r(1/2)$, giving the matching upper bound and proving minimaxity. $\square$

Worked Example: The Equal-Weight Rule at a Concrete Observation

Take $a=1$, $\sigma^2=1$, and $p=1/2$. At $x=1$, the likelihood ratio is $e^2$, so $q_{1/2}(1)=e^2/(1+e^2)\approx0.8808$. The posterior mean is \[ \delta_{1/2}(1)=2q_{1/2}(1)-1 =\frac{e^2-1}{e^2+1}=\tanh(1)\approx0.7616. \] The posterior variance is $4q(1-q)=4e^2/(1+e^2)^2\approx0.4200$.

For any observed $x$, the rule is $\tanh(x)$, so reflecting $x$ changes the estimate's sign. In particular, the risk under $\theta=1$, written explicitly, is $E_{X\sim N(1,1)}[(\tanh(X)-1)^2]$; the risk under $\theta=-1$ is equal by reflection. The theorem shows that this shared endpoint risk is the minimax risk, without requiring a closed-form evaluation of the integral.

Interpretation and a Common Pitfall

The key logical chain is concise: reflection makes the Bayes-risk curve symmetric around $1/2$; concavity means the midpoint is at least as high as the average of the values at any pair of reflected weights; together these facts show that the midpoint is a maximum. The minimax conclusion uses one further fact: the equal-prior Bayes rule has equal risks at the two parameter values. These are related conclusions, but the last one depends on the symmetry of the rule and the model as well as on the prior comparison.

Pitfall: claiming symmetry alone proves maximality. A symmetric function can have a minimum at its midpoint. The correction is to use concavity as well: symmetry gives $r(p)=r(1-p)$, and concavity then gives $r(1/2)\geq r(p)$.
Pitfall: claiming concavity alone identifies the equal-weight prior. A concave function need not be maximized at its midpoint. The reflection identity is essential to compare the two sides.
Pitfall: treating “least favorable” as automatically meaning “unique.” The argument proves that equal weights maximize Bayes risk, not that they are the only maximizer.
Pitfall: inferring minimaxity from least favorability alone. A least-favorable prior supplies a lower bound on minimax risk. To match it here, verify that the equal-prior Bayes rule has maximum endpoint risk equal to its Bayes risk.

Check Your Understanding

Use the symmetry, concavity, and equal-prior rule arguments developed above.

For $a=2$, $\sigma^2=1$, and $p=1/4$, what prior weight is paired with $p$ by reflection, and what equality holds between their Bayes risks?
Why is the posterior mean the Bayes action under squared-error loss for a two-point posterior?
State the concavity inequality that compares $r(1/2)$ with $r(p)$ using reflection symmetry.
What additional property of the equal-prior Bayes rule turns the least-favorable-prior lower bound into a minimax certificate?
Does the symmetry-and-concavity argument prove that the equal-weight prior is the unique maximizer? Explain.