Gaussian Location Risk For The Sample Mean

Why Compare a Data-Based Rule With a Fixed Estimate?

In a Gaussian location model, the sample mean uses the observations to estimate the unknown location. A fixed estimate does not: it always reports the same number, no matter what data are observed. Comparing their squared-error risks makes the role of the data precise. The sample mean has a constant risk across the parameter space, while a fixed estimate can do very well near its chosen value and poorly far from it.

The comparison is about repeated-sampling performance at each possible parameter value, not about which estimate happens to be closer for one observed sample. We will calculate both risks exactly and identify the parameter values at which each rule has the smaller risk.

Model, Estimators, and Risk

Definition: Suppose $X_1,\ldots,X_n$ are independent and identically distributed with $X_i\sim N(\theta,\sigma^2)$, where the unknown parameter is $\theta\in\mathbb{R}$, the variance $\sigma^2>0$ is known, and $n\geq 1$. Under squared-error loss, the risk of an estimator $\delta$ at $\theta$ is $$ R(\theta,\delta)=E_\theta[(\delta(X_1,\ldots,X_n)-\theta)^2]. $$ The sample mean is $\bar X=n^{-1}\sum_{i=1}^n X_i$. A fixed estimator, denoted $\delta_c$, reports a prespecified constant $c\in\mathbb{R}$: $\delta_c(X_1,\ldots,X_n)=c$ for every possible sample.

Risk averages the loss over all samples that could be generated at the given $\theta$. Thus the risk of a rule is a function of the unknown parameter, even though its value can be calculated from the model. The fixed estimator’s constant $c$ must be chosen without using these observations. It could, for example, be zero or a reference value specified by the problem.

Exact Risk of the Sample Mean

Theorem: Under the stated Gaussian location model, for every $\theta\in\mathbb{R}$, $$ \bar X\sim N\left(\theta,\frac{\sigma^2}{n}\right) \quad\text{and}\quad R(\theta,\bar X)=\frac{\sigma^2}{n}. $$

Proof. Independence gives $E_\theta[\bar X]=n^{-1}\sum_{i=1}^n E_\theta[X_i]=\theta$. It also gives $$ \operatorname{Var}_\theta(\bar X) =\operatorname{Var}_\theta\left(\frac{1}{n}\sum_{i=1}^n X_i\right) =\frac{1}{n^2}\sum_{i=1}^n\operatorname{Var}_\theta(X_i) =\frac{\sigma^2}{n}. $$ A linear combination of independent normal random variables is normal, so these two moments give the stated distribution. Since the sample mean is unbiased, its squared-error risk is its variance: $$ R(\theta,\bar X) =E_\theta[(\bar X-\theta)^2] =\operatorname{Var}_\theta(\bar X) =\frac{\sigma^2}{n}. $$ This expression does not depend on $\theta$. $\square$

The risk is exact for every finite sample size; it is not an approximation that requires $n$ to be large. Increasing $n$ reduces it in proportion to $1/n$, while increasing the observation variance increases it in proportion to $\sigma^2$. Normality supplies the exact normal distribution of $\bar X$, but the risk calculation itself uses only its mean and variance.

Worked Example: Risk With a Specified Sample Size and Variance

Suppose $n=4$ and each observation has variance $\sigma^2=9$. Calculate the sample mean’s risk and compare it with the fixed estimator that always reports zero.

The sample mean risk is $$ R(\theta,\bar X)=\frac{\sigma^2}{n}=\frac{9}{4}=2.25 $$ for every real value of $\theta$. The fixed estimator is $\delta_0=0$, so its error is always $0-\theta$, and $$ R(\theta,\delta_0)=E_\theta[(0-\theta)^2]=\theta^2. $$ At $\theta=1$, the risks are $2.25$ for the sample mean and $1$ for $\delta_0$, so the fixed estimator has lower risk. At $\theta=2$, they are $2.25$ and $4$, respectively, so the sample mean has lower risk. These are comparisons of expected squared errors over repeated samples, not of the two rules on a particular observed data set.

Risk of a Fixed Estimator and the Exact Comparison

Because $\delta_c$ always equals $c$, its error at a particular parameter value is the nonrandom quantity $c-\theta$. Its risk therefore has no sampling variability to average out.

Theorem: For any fixed $c\in\mathbb{R}$, the fixed estimator has risk $$ R(\theta,\delta_c)=(c-\theta)^2. $$ Writing $r=\sigma^2/n$, its risk is less than, equal to, or greater than the sample mean’s risk according as $$ |\theta-c|<\sqrt{r},\qquad |\theta-c|=\sqrt{r},\qquad |\theta-c|>\sqrt{r}, $$ respectively.

Proof. For every sample, $\delta_c-\theta=c-\theta$. Thus $$ R(\theta,\delta_c) =E_\theta[(c-\theta)^2] =(c-\theta)^2. $$ The sample mean risk is $r$, by the preceding theorem. Consequently, the fixed estimator has smaller risk exactly when $(c-\theta)^2<r$. Since both sides are nonnegative and $r>0$, this is equivalent to $|\theta-c|<\sqrt r$. Equality of risks is equivalent to $|\theta-c|=\sqrt r$; outside that interval, $(c-\theta)^2>r$, so the sample mean has smaller risk. $\square$

This comparison is pointwise in $\theta$: it says which risk is smaller at each specified parameter value. The fixed estimator’s risk is zero when $\theta=c$, then increases quadratically as $\theta$ moves away from $c$. The sample mean’s risk is a flat line at height $\sigma^2/n$. The two risks meet at $c-\sqrt{\sigma^2/n}$ and $c+\sqrt{\sigma^2/n}$.

Parameter location	Fixed estimator’s risk	Lower-risk rule
$\|\theta-c\|<\sqrt{\sigma^2/n}$	Less than $\sigma^2/n$	Fixed estimator
$\|\theta-c\|=\sqrt{\sigma^2/n}$	Equal to $\sigma^2/n$	Equal risk
$\|\theta-c\|>\sqrt{\sigma^2/n}$	Greater than $\sigma^2/n$	Sample mean

Worked Example: Finding the Full Region of Lower Risk

Let $n=9$, $\sigma^2=4$, and consider the fixed estimator $\delta_2=2$. Determine where it has lower risk than the sample mean.

The sample mean risk is $$ R(\theta,\bar X)=\frac{4}{9}. $$ The fixed estimator risk is $$ R(\theta,\delta_2)=(\theta-2)^2. $$ It is lower precisely when $$ (\theta-2)^2<\frac{4}{9} \quad\Longleftrightarrow\quad |\theta-2|<\frac{2}{3} \quad\Longleftrightarrow\quad \frac{4}{3}<\theta<\frac{8}{3}. $$ For example, at $\theta=2.5$, the fixed risk is $(2.5-2)^2=0.25$, which is below $4/9$. At $\theta=0$, its risk is $(0-2)^2=4$, which is above $4/9$. At either endpoint, $\theta=4/3$ or $\theta=8/3$, its risk is $4/9$, exactly matching the sample mean’s risk.

Worked Example: A Fixed Zero Estimate With a Different Noise Level

Suppose $n=5$ and $\sigma^2=1.25$, and compare $\bar X$ with $\delta_0=0$. Calculate both risks at $\theta=0.4$ and at $\theta=1$.

The sample mean risk at either parameter value is $$ R(\theta,\bar X)=\frac{1.25}{5}=0.25. $$ At $\theta=0.4$, the fixed estimator risk is $$ R(0.4,\delta_0)=(0-0.4)^2=0.16, $$ so the fixed estimator has lower risk at that parameter value. At $\theta=1$, it is $$ R(1,\delta_0)=(0-1)^2=1, $$ so the sample mean has lower risk there. More generally, the fixed zero estimate has lower risk exactly when $$ \theta^2<0.25 \quad\Longleftrightarrow\quad -0.5<\theta<0.5. $$ The endpoint risks agree, since $0.5^2=0.25$.

What the Comparison Does—and Does Not—Show

On the unrestricted parameter space $\mathbb{R}$, neither rule has lower risk at every parameter value. At $\theta=c$, the fixed estimator has risk zero, strictly below $\sigma^2/n>0$. But when $|\theta-c|>\sqrt{\sigma^2/n}$, the sample mean has strictly lower risk. Thus the comparison does not establish that one estimator is uniformly better; it identifies where each is better.

A fixed value can be attractive when there is a compelling reason to expect $\theta$ to be near $c$. But a lower risk near $c$ does not by itself justify choosing that rule when $\theta$ is unknown. Risk is evaluated at each parameter value, and the comparison over the full parameter space exposes the cost of using a fixed estimate when the true location is far from its target.

Common Mistakes

Using the standard deviation instead of the variance. If observations have variance $\sigma^2$, then the sample mean variance and risk are $\sigma^2/n$, not $\sigma/n$.
Calling the fixed estimator’s error random. Its error $c-\theta$ is fixed at each $\theta$, so its risk is exactly $(c-\theta)^2$.
Inferring a uniform risk ordering from one parameter value. The fixed estimator wins near $c$, but the sample mean wins sufficiently far away. A comparison at one $\theta$ cannot settle the ordering at all $\theta$.
Comparing realized errors instead of risks. For a particular sample, $(\bar X-\theta)^2$ can be larger or smaller than $(c-\theta)^2$. Risk compares their expectations under repeated sampling at the same parameter value.

Key takeaway: In the Gaussian location model, the sample mean has constant squared-error risk $\sigma^2/n$, while a fixed estimate $c$ has risk $(c-\theta)^2$. The fixed estimate wins within distance $\sqrt{\sigma^2/n}$ of $c$; the sample mean wins outside that interval.

Check Your Understanding

Use the risk formulas to answer the following questions.

If $n=8$ and $\sigma^2=16$, what is the sample mean’s risk at an arbitrary $\theta$?
For a fixed estimator $\delta_3=3$, write its risk as a function of $\theta$. At what parameter value is this risk zero?
If $n=4$, $\sigma^2=4$, and $c=1$, identify the open interval of parameter values where $\delta_c$ has lower risk than $\bar X$.
Can a fixed estimator have lower risk than the sample mean for every $\theta\in\mathbb{R}$? Explain using the two risk functions.
Why does observing one sample with $\bar X$ close to $\theta$ not establish that the sample mean has lower risk than a fixed estimator?

Parameter location	Fixed estimator’s risk	Lower-risk rule
\(\|\theta-c\|<\sqrt{\sigma^2/n}\)	Less than \(\sigma^2/n\)	Fixed estimator
\(\|\theta-c\|=\sqrt{\sigma^2/n}\)	Equal to \(\sigma^2/n\)	Equal risk
\(\|\theta-c\|>\sqrt{\sigma^2/n}\)	Greater than \(\sigma^2/n\)	Sample mean