Variance as the Best Mean-Squared Constant Approximation

Why the Mean Is the Best Constant

Suppose a random quantity $X$ must be represented by one fixed number $a$, with error measured by the squared difference $(X-a)^2$. Which number gives the smallest expected squared error? The answer is the mean, but the reason is more informative than a differentiation rule: for every choice of $a$, its expected squared error separates into an unavoidable part, the variance, and an additional nonnegative penalty for choosing $a$ away from the mean.

This result concerns approximation by constants, not by functions of other variables. Its proof uses only the definitions of expectation and variance and an expansion of a square. The moment condition matters: the expected squared error must be finite for the decomposition to be used as an ordinary equality of finite quantities.

Definition: Let $X$ be a real-valued random variable with finite variance, and write $\mu=E[X]$. For a constant $a\in\mathbb{R}$, define its mean-squared error as $R(a)=E[(X-a)^2]$. The problem is to find the constant $a$ that minimizes $R(a)$.

Here finite variance means $E[(X-\mu)^2]<\infty$, with $\mu$ finite. In particular, $E[X^2]<\infty$: since $X^2\leq 2(X-\mu)^2+2\mu^2$, taking expectations gives a finite upper bound for $E[X^2]$. Thus $R(a)$ is finite for every real $a$, because $(X-a)^2\leq 2X^2+2a^2$.

The Mean-Squared Error Decomposition

The key is to express the error relative to $a$ as the sum of the deviation from the mean and the difference between the mean and $a$. The cross term vanishes after taking expectation because the deviation from the mean has expectation zero.

Theorem: If $X$ has finite variance and $\mu=E[X]$, then for every constant $a\in\mathbb{R}$, $$ E[(X-a)^2]=\operatorname{Var}(X)+(\mu-a)^2. $$

Proof. Write $X-a=(X-\mu)+(\mu-a)$, and expand:

(X-a)^2=(X-\mu)^2+2(\mu-a)(X-\mu)+(\mu-a)^2.

All terms are integrable: $E[(X-\mu)^2]$ is finite by assumption, $E[X-\mu]=0$, and $\mu-a$ is a finite constant. Taking expectations term by term therefore gives

$$ \begin{aligned} E[(X-a)^2] &=E[(X-\mu)^2]+2(\mu-a)E[X-\mu]+(\mu-a)^2\\ &=\operatorname{Var}(X)+2(\mu-a)\cdot 0+(\mu-a)^2\\ &=\operatorname{Var}(X)+(\mu-a)^2. \end{aligned} $$ This proves the decomposition. \(\square\)

The two terms have different roles. The variance does not depend on $a$; it is the minimum error that remains even when the constant is chosen optimally. The squared distance $(\mu-a)^2$ is the extra error from using a constant other than the mean.

Corollary: If $X$ has finite variance, $R(a)=E[(X-a)^2]$ has a unique minimum over $a\in\mathbb{R}$, attained at $a=\mu=E[X]$. The minimum value is $\operatorname{Var}(X)$.

Proof. By the theorem, $R(a)=\operatorname{Var}(X)+(\mu-a)^2$. A square is nonnegative, so $R(a)\geq\operatorname{Var}(X)$. Equality holds exactly when $(\mu-a)^2=0$, which is equivalent to $a=\mu$. Thus the mean attains the minimum, and no other constant does. $\square$

A useful way to read the result is as an exact excess-error formula:

R(a)-R(\mu)=(a-\mu)^2.

This says not only that the mean is best, but also precisely how much expected squared error is added by any alternative choice. For example, choosing a constant one unit away from the mean increases the error by one squared unit, regardless of the distribution's shape.

Worked Calculations

Worked Example: A Three-Point Distribution

Let $X$ take the values $0,2,5$ with probabilities $0.2,0.5,0.3$, respectively. Find the best constant and its minimum mean-squared error. Then calculate the error at $a=1$.

First calculate the mean:

\mu=E[X]=0(0.2)+2(0.5)+5(0.3)=0+1+1.5=2.5.

The second moment and variance are

E[X^2]=0^2(0.2)+2^2(0.5)+5^2(0.3)=0+2+7.5=9.5, $$ $$ \operatorname{Var}(X)=E[X^2]-\mu^2=9.5-(2.5)^2=9.5-6.25=3.25.

The decomposition gives $R(a)=3.25+(2.5-a)^2$. Hence the unique best constant is $a=2.5$, and its error is $R(2.5)=3.25$. At $a=1$, the error is

$$ R(1)=3.25+(2.5-1)^2=3.25+2.25=5.5. $$

Direct calculation checks the result: $E[(X-1)^2]=0.2(1)+0.5(1)+0.3(16)=0.2+0.5+4.8=5.5$.

Worked Example: A Normal Distribution

Suppose $X\sim N(10,4)$, where the second parameter is the variance. Find the best constant approximation and compare its error with the error from using $a=8$.

For this distribution, $E[X]=10$ and $\operatorname{Var}(X)=4$. Therefore

$$ R(a)=4+(10-a)^2. $$

The unique minimizer is $a=10$, with minimum mean-squared error $R(10)=4$. Choosing $a=8$, which is two units below the mean, gives

$$ R(8)=4+(10-8)^2=4+4=8. $$

Thus this choice adds $4$ units of expected squared error to the minimum. The minimum itself is not zero: even centering the approximation at the mean does not eliminate the variability of $X$.

Worked Example: Unequal Probabilities and a Misleading Midpoint

Let $X=2$ with probability $0.75$ and $X=8$ with probability $0.25$. Find the best constant. Compare it with the midpoint of the two possible values, and verify the error by direct expansion.

The mean is the probability-weighted average:

\mu=2(0.75)+8(0.25)=1.5+2=3.5.

The second moment is $E[X^2]=4(0.75)+64(0.25)=3+16=19$. Hence

\operatorname{Var}(X)=19-(3.5)^2=19-12.25=6.75.

So $a=3.5$ uniquely minimizes the error, and the minimum is $6.75$. The midpoint of the possible values is $5$, not $3.5$. Its error is

$$ R(5)=6.75+(5-3.5)^2=6.75+2.25=9. $$

For a direct check at any $a$,

\begin{aligned} R(a)&=0.75(2-a)^2+0.25(8-a)^2\\ &=0.75(a^2-4a+4)+0.25(a^2-16a+64)\\ &=a^2-7a+19\\ &=(a-3.5)^2+6.75. \end{aligned}

The midpoint would be the mean only if the two values had equal probabilities. The minimizing constant depends on the distribution's probabilities, not just its range.

Conditions, Interpretation, and Common Pitfalls

Finite variance is not a decorative technical assumption. The proof expands expectations of squared terms and uses finite quantities. If $X$ does not have a finite second moment, the finite-valued decomposition above cannot simply be invoked. In fact, when $E[X^2]=\infty$, $E[(X-a)^2]=\infty$ for every finite $a$: the inequality $X^2\leq 2(X-a)^2+2a^2$ shows that a finite value of $E[(X-a)^2]$ would force $E[X^2]$ to be finite. In that case every constant has infinite mean-squared error, so there is no unique finite-error minimizer of the kind established here.

The result also clarifies why squared error and absolute error lead to different notions of a best constant. This tutorial establishes the minimizer for expected squared error only; it does not imply that the mean minimizes every possible measure of error.

Mistake: dropping the cross term without explanation. The term is $2(\mu-a)E[X-\mu]$, which vanishes because $E[X-\mu]=0$. It is not zero before expectation is taken.
Mistake: claiming the minimum error is zero. At $a=\mu$, the error is $\operatorname{Var}(X)$, which is positive whenever $X$ is not almost surely constant.
Mistake: choosing the midpoint of the smallest and largest possible values. The best constant is the probability-weighted mean. A midpoint agrees with it only in particular distributions, such as an equally weighted two-point distribution.
Mistake: using the formula without checking moments. The decomposition as a finite-error identity requires finite variance. Infinite second moments cannot be treated as if all terms in the expansion were finite.

Key takeaway: For a finite-variance random variable, every constant $a$ has mean-squared error $\operatorname{Var}(X)+(a-E[X])^2$. The mean uniquely minimizes this error, and the excess error from any other choice is exactly the squared distance from the mean.

Check Your Understanding

Use the decomposition to answer these questions. When doing calculations, show how the mean and variance enter the result.

If $E[X]=4$ and $\operatorname{Var}(X)=9$, what is $E[(X-a)^2]$ as a function of $a$, and where is it minimized?
For the same $X$, how much larger is the mean-squared error at $a=1$ than at the minimizing constant?
A random variable takes values $1$ and $7$ with probabilities $0.5$ and $0.5$. Find its best constant and minimum error.
In the decomposition proof, why does the cross term have expectation zero?
Why does finite variance ensure that $E[(X-a)^2]$ is finite for every real constant $a$?

Variance as the Best Mean-Squared Constant Approximation

What You'll Learn