UMVU Estimation from Normal Samples

Why Normal-Model UMVU Estimation Needs Both Statistics

For a normal sample with unknown mean and variance, estimating one parameter can require information about both. The sample mean alone, for example, is unbiased for the population mean but its square is not unbiased for the squared mean: its expectation includes a contribution from sampling variance. The sample variance supplies the adjustment needed to remove that contribution.

The central tool is the complete sufficient statistic. Once an unbiased estimator is expressed as a function of that statistic, the Lehmann–Scheffé theorem establishes that it is the unique uniformly minimum-variance unbiased estimator, or UMVU estimator. This tutorial develops that method for the mean, variance, and several functions of the two parameters.

Model and notation: Let $X_1,\ldots,X_n$ be an independent random sample from $N(\mu,\sigma^2)$, where $\mu$ is any real number and $\sigma^2>0$. Define $$ \bar{X}=\frac{1}{n}\sum_{i=1}^{n}X_i,\qquad S^2=\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar{X})^2 \quad (n\geq 2). $$ Here $S^2$ is the unbiased sample variance. The parameter $\sigma^2$ denotes the population variance, not the standard deviation.

Complete Sufficiency in the Normal Sample

The joint density of the sample is

f(x_1,\ldots,x_n) =(2\pi\sigma^2)^{-n/2} \exp\left\{-\frac{1}{2\sigma^2}\sum_{i=1}^{n}x_i^2 +\frac{\mu}{\sigma^2}\sum_{i=1}^{n}x_i -\frac{n\mu^2}{2\sigma^2}\right\}.

The data enter this density through $\sum X_i$ and $\sum X_i^2$. By the factorization theorem, the statistic

T=\left(\sum_{i=1}^{n}X_i,\ \sum_{i=1}^{n}X_i^2\right)

is sufficient for $(\mu,\sigma^2)$. It is also complete for the model in which both parameters are unknown. To see the relevant structure, the two natural parameters in the density are $\eta_1=\mu/\sigma^2$ and $\eta_2=-1/(2\sigma^2)$. As $\mu$ ranges over the real numbers and $\sigma^2>0$, $(\eta_1,\eta_2)$ ranges over the open set $\mathbb{R}\times(-\infty,0)$. The full, regular exponential-family completeness result therefore applies. In brief, if an integrable function of $T$ has expectation zero for every parameter value, its transform against the exponential-family density is zero throughout this open natural-parameter set; uniqueness of the multivariate Laplace transform implies the function is zero almost surely.

For $n\geq2$, $T$ and $(\bar X,S^2)$ determine each other, since $\sum X_i=n\bar X$ and $\sum X_i^2=(n-1)S^2+n\bar X^2$. Thus either representation can be used. The result used below is the Lehmann–Scheffé theorem: an unbiased estimator that is a function of a complete sufficient statistic is the unique UMVU estimator of its target, among unbiased estimators with finite variance.

UMVU Estimators of the Mean and Variance

The sample mean is unbiased for $\mu$, because $E[\bar X]=\mu$, and is a function of $T$. The sample variance is unbiased for $\sigma^2$, because $E[S^2]=\sigma^2$, and is also a function of $T$. The theorem therefore identifies both as UMVU estimators.

\widehat{\mu}_{\mathrm{UMVU}}=\bar X,\qquad \widehat{\sigma^2}_{\mathrm{UMVU}}=S^2.

The condition $n\geq2$ is needed for the displayed definition of $S^2$ and for estimating $\sigma^2$ by this method. When $n=1$, $\bar X=X_1$ remains unbiased for $\mu$, but the sample variance formula is not defined.

Worked Example: Estimating the Mean and Variance

A sample of $n=4$ independent observations comes from $N(\mu,\sigma^2)$, with both parameters unknown. The observed values are $8,10,12,14$. Find the UMVU estimators of $\mu$ and $\sigma^2$.

Method. Calculate $\bar X$ and the sample variance $S^2$. Since both are unbiased functions of the complete sufficient statistic, they are the required UMVU estimators.

The sample mean is $$ \bar X=\frac{8+10+12+14}{4}=\frac{44}{4}=11. $$ The deviations from 11 are $-3,-1,1,3$, so their squared sum is $$ (-3)^2+(-1)^2+1^2+3^2=9+1+1+9=20. $$ Thus $$ S^2=\frac{20}{4-1}=\frac{20}{3}. $$ The UMVU estimates are $11$ for $\mu$ and $20/3$ for $\sigma^2$, or $6.67$ when rounded to two decimals. The calculation checks directly against the definitions: the deviations sum to zero, and their squared sum is 20.

Common mistake avoided. Dividing the squared deviations by $n=4$ gives 5, but this is not the unbiased sample variance. The denominator for $S^2$ is $n-1=3$, giving $20/3$.

Estimating Functions of the Parameters

For a function such as $\mu^2$, $\bar X$ cannot simply be squared and assumed unbiased. The normal-sample identity

E[\bar X^2]=\operatorname{Var}(\bar X)+(E[\bar X])^2 =\frac{\sigma^2}{n}+\mu^2

shows exactly what must be corrected. Since $E[S^2/n]=\sigma^2/n$, subtraction removes the extra term. The result is an unbiased function of the complete sufficient statistic:

\widehat{\mu^2}_{\mathrm{UMVU}}=\bar X^2-\frac{S^2}{n}.

This is also the average of all distinct pairwise products in the sample:

\bar X^2-\frac{S^2}{n} =\frac{\left(\sum_{i=1}^n X_i\right)^2-\sum_{i=1}^n X_i^2}{n(n-1)} =\frac{1}{n(n-1)}\sum_{i\ne j}X_iX_j.

Each product $X_iX_j$, for $i\ne j$, has expectation $E[X_i]E[X_j]=\mu^2$ by independence. This gives a second justification of unbiasedness. Completeness then gives the UMVU conclusion.

Worked Example: Estimating the Squared Mean

A sample of $n=4$ independent observations comes from $N(\mu,\sigma^2)$, with both parameters unknown. The observed values are $8,10,12,14$, as in the preceding calculation. Estimate $\mu^2$ by its UMVU estimator.

Method. Use $\bar X^2-S^2/n$, rather than the naive square $\bar X^2$. From the data, $\bar X=11$ and $S^2=20/3$. Therefore $$ \widehat{\mu^2}_{\mathrm{UMVU}} =11^2-\frac{20/3}{4} =121-\frac{5}{3} =\frac{363-5}{3} =\frac{358}{3}. $$

The estimate is $119.33$, rounded to two decimals. As a check, the pairwise-product form gives $(8\cdot10+8\cdot12+8\cdot14+10\cdot12+10\cdot14+12\cdot14)/6=716/6=358/3$, the same value.

Common mistake avoided. Using $\bar X^2=121$ alone ignores the $\sigma^2/n$ term in its expectation. Subtracting $S^2/n=5/3$ corrects this bias.

Another useful target is $\mu\sigma^2$. In a normal sample, $\bar X$ and $S^2$ are independent, with $E[\bar X]=\mu$ and $E[S^2]=\sigma^2$. Hence

E[\bar X S^2]=E[\bar X]E[S^2]=\mu\sigma^2.

Because their product is a function of the complete sufficient statistic, the UMVU estimator is $\bar X S^2$. The independence statement is specific to the normal-sample model; this argument should not be carried over to arbitrary distributions.

Worked Example: Estimating the Product of Mean and Variance

Five independent observations come from $N(\mu,\sigma^2)$, with both parameters unknown. The values are $3,5,5,7,10$. Find the UMVU estimate of $\mu\sigma^2$.

Method. Use $\bar X S^2$, whose unbiasedness follows from independence of the sample mean and sample variance under normal sampling.

The sample mean is $$ \bar X=\frac{3+5+5+7+10}{5}=\frac{30}{5}=6. $$ The deviations are $-3,-1,-1,1,4$, and the sum of their squares is $$ 9+1+1+1+16=28. $$ Therefore $S^2=28/(5-1)=7$, and the estimate is $$ \bar X S^2=6\cdot7=42. $$ The deviations sum to zero and their squared sum is 28, confirming both summary calculations. The UMVU estimate is $42$.

Common mistake avoided. The variance estimate is $28/4$, not $28/5$. The unbiasedness argument also depends on normal sampling; multiplying two unbiased estimators does not generally produce an unbiased estimator of the product of their targets.

Estimating the Square of the Variance

The same strategy works for $\sigma^4=(\sigma^2)^2$, but the expectation of $(S^2)^2$ must first be adjusted. Under normal sampling, the standard sampling-distribution result is

Q=\frac{(n-1)S^2}{\sigma^2}\sim\chi^2_{n-1}.

A chi-square random variable with $\nu$ degrees of freedom has second moment $E[Q^2]=\nu(\nu+2)$. Letting $\nu=n-1$ gives

E[(S^2)^2] =\frac{\sigma^4}{(n-1)^2}E[Q^2] =\frac{\sigma^4(n-1)(n+1)}{(n-1)^2} =\frac{n+1}{n-1}\sigma^4.

Therefore the unbiased complete-sufficient-statistic function, and hence the UMVU estimator, is

\widehat{\sigma^4}_{\mathrm{UMVU}} =\frac{n-1}{n+1}(S^2)^2,\qquad n\geq2.

Worked Example: Estimating the Fourth Power of the Standard Deviation Parameter

Four independent observations come from $N(\mu,\sigma^2)$, with both parameters unknown. The observed values are $2,4,6,8$. Estimate $\sigma^4$, the square of the population variance.

Method. First compute $S^2$, then apply $\frac{n-1}{n+1}(S^2)^2$. The sample mean is $(2+4+6+8)/4=5$. The squared deviations from 5 sum to $9+1+1+9=20$, so $$ S^2=\frac{20}{4-1}=\frac{20}{3}. $$ With $n=4$, the UMVU estimate is $$ \widehat{\sigma^4}_{\mathrm{UMVU}} =\frac{3}{5}\left(\frac{20}{3}\right)^2 =\frac{3}{5}\cdot\frac{400}{9} =\frac{80}{3} =26.67, $$ rounded to two decimals. The squared deviations sum to 20 and $S^2=20/3$; substituting this value into the estimator gives the displayed result.

Common mistake avoided. Using $(S^2)^2=400/9$ alone is biased upward for $\sigma^4$: its expectation is $(5/3)\sigma^4$ when $n=4$. Multiplication by $3/5$ removes that factor.

How to Apply the Method Reliably

Specify the target and model.
Write the target parameter function, such as $\mu^2$ or $\mu\sigma^2$, and confirm the observations are an independent normal sample with the stated unknown parameters.

Find an unbiased starting expression.
Use expectations to identify corrections. For example, $E[\bar X^2]=\mu^2+\sigma^2/n$, so $\bar X^2$ needs a variance adjustment to estimate $\mu^2$.

Express the result using the complete sufficient statistic.
Rewrite the estimator in terms of $\bar X$ and $S^2$, which are equivalent to $(\sum X_i,\sum X_i^2)$ for $n\geq2$.

Check unbiasedness and apply Lehmann–Scheffé.
Verify that the estimator's expectation equals the target. A function of the complete sufficient statistic that is unbiased is the unique UMVU estimator.

Common Exam Traps and Scope

The method is powerful, but each conclusion depends on its assumptions and on distinguishing the parameter from its estimator. In particular, $\sigma^2$ is the variance, $S^2$ is its estimator, and $\sigma^4$ is the square of the variance. The factor used for $\sigma^4$ is not the same as the correction used for $\mu^2$.

Squaring an unbiased estimator does not usually preserve unbiasedness. Here $E[\bar X^2]=\mu^2+\sigma^2/n$, so the extra term must be removed.
Use the right denominator. $S^2$ divides the squared deviations by $n-1$, not by $n$.
State the model behind independence. The independence of $\bar X$ and $S^2$, used for $\mu\sigma^2$, holds for a normal sample.
Check the sample-size condition. The displayed $S^2$-based estimators require $n\geq2$.
Do not confuse unbiasedness with a finite-sample observed value. Unbiasedness is a statement about the estimator's expectation over repeated samples, not a guarantee that one estimate equals the unknown parameter.

Key takeaway: In the normal model with both parameters unknown, $(\sum X_i,\sum X_i^2)$ is complete and sufficient. Find an unbiased function of this statistic for the target; the Lehmann–Scheffé theorem then identifies it as the UMVU estimator.

Check Your Understanding

For each question, assume an independent sample from a normal distribution and that both $\mu$ and $\sigma^2$ are unknown. Use $S^2$ with denominator $n-1$.

A sample of size 3 has values $1,3,5$. Find the UMVU estimates of $\mu$ and $\sigma^2$. (Answer: $\bar X=3,\ S^2=4$.)
A sample of size 4 has values $2,4,6,8$. Find the UMVU estimate of $\mu^2$. (Answer: $70/3$, or $23.33$ to two decimals.)
A sample of size 4 has values $1,3,5,7$. Find the UMVU estimate of $\mu\sigma^2$. (Answer: $80/3$, or $26.67$ to two decimals.)
A sample of size 3 has values $0,2,4$. Find the UMVU estimate of $\sigma^4$. (Answer: $8$.)
A sample of size 3 has values $2,4,6$. Find the UMVU estimate of $\mu^2+\sigma^2$. (Answer: $56/3$, or $18.67$ to two decimals.)

What You'll Learn