Conditional Expectations for Normal-Sample Estimators

Why Condition in a Normal Sample?

For a normal sample, a sufficient statistic collects the sample information relevant to the unknown parameter or parameters. An unbiased estimator may nevertheless depend on the observations in a way that adds avoidable variation. Conditioning that estimator on a sufficient statistic averages over sample arrangements that carry no additional information about the parameters. In a normal sample, permutation symmetry often makes this conditional expectation calculable without finding a complicated conditional density.

Definition: Let $W$ be an unbiased estimator of a parameter function $\tau$, and let $T$ be a sufficient statistic. The Rao–Blackwellized estimator is $W^* = E[W\mid T]$. It is unbiased for $\tau$, and its variance is no greater than the variance of $W$, provided $W$ has finite variance.

The focus here is on calculating $E[W\mid T]$ in normal samples. The conditional expectation is a function of the observed value of $T$, not a new random draw. If $W$ is already a function of $T$, conditioning does not change it: $E[W\mid T]=W$ with probability one.

Normal-Sample Sufficient Statistics and Symmetry

Suppose $X_1,\ldots,X_n$ are independent and identically distributed as $N(\mu,\sigma^2)$, where $\sigma^2>0$. If $\sigma^2$ is known and $\mu$ is unknown, the sample sum $T_1=\sum_{i=1}^n X_i$ is sufficient for $\mu$. If both $\mu$ and $\sigma^2$ are unknown, the pair $T=(T_1,T_2)$, with $T_2=\sum_{i=1}^n X_i^2$, is sufficient for $(\mu,\sigma^2)$. These statements follow by factoring the joint density: its dependence on the sample values occurs through the sum alone in the known-variance model, and through the sum and sum of squares in the unknown-variance model.

Both sufficient statistics are unchanged by a permutation of the observations. The normal-sample joint density is also unchanged by such a permutation. Consequently, conditional on either sufficient statistic, the observations are symmetric: each $X_i$ has the same conditional expectation, and each distinct pair $X_i,X_j$ has the same conditional expectation of its product. This symmetry is the main calculation tool.

E[X_i\mid T_1]=\frac{T_1}{n}, \qquad E[X_i^2\mid T_1,T_2]=\frac{T_2}{n}.

For the first identity, all $E[X_i\mid T_1]$ are equal, and their sum is $E[\sum_i X_i\mid T_1]=T_1$. For the second, all $E[X_i^2\mid T_1,T_2]$ are equal, and their sum is $E[\sum_i X_i^2\mid T_1,T_2]=T_2$. Dividing each sum by $n$ gives the results.

Theorem: If $W$ is unbiased for $\tau$ and has finite variance, then $E[W\mid T]$ is also unbiased for $\tau$, and $$ \operatorname{Var}(W) = \operatorname{Var}(E[W\mid T]) + E[\operatorname{Var}(W\mid T)]. $$ Thus conditioning cannot increase variance. It leaves variance unchanged precisely when $\operatorname{Var}(W\mid T)=0$ with probability one, equivalently when $W$ is already determined by $T$ with probability one.

The identity is the law of total variance applied to $W$. In particular, the second term is nonnegative. Also, iterated expectation gives $E[E[W\mid T]]=E[W]=\tau$, which proves that the conditioned estimator remains unbiased. A strict variance reduction occurs when the conditional variance is positive with positive probability.

Conditioning an Estimator of the Mean

In the normal model with unknown $\mu$, each $X_i$ is unbiased for $\mu$. When $\sigma^2$ is known, condition on $T_1$; when both parameters are unknown, condition on $(T_1,T_2)$. In either case, the symmetry argument gives $E[X_1\mid T]=T_1/n=\bar X$. By contrast, $\bar X$ itself is already a function of the sufficient statistic, so conditioning $\bar X$ leaves it unchanged.

Worked Example: Replacing One Observation with the Sample Mean

Let $X_1,\ldots,X_4$ be an independent sample from $N(\mu,9)$, with $\mu$ unknown and variance known. Use $W=X_1$ to estimate $\mu$. For the observed sample $(2,4,6,8)$, find the Rao–Blackwellized estimate and compare the estimator variances. Name the method and a common error it avoids.

Method: condition on the sufficient sum. Here $T_1=\sum_i X_i$ is sufficient for $\mu$, and $E[X_1\mid T_1]=T_1/4$. The observed sum is $T_1=2+4+6+8=20$, so the conditioned estimate is $20/4=5$. The original estimate from the first observation is $2$.

Both estimators are unbiased: $E[X_1]=\mu$, and $E[\bar X]=\mu$. Their variances are $\operatorname{Var}(X_1)=9$ and $\operatorname{Var}(\bar X)=9/4=2.25$. Conditioning therefore reduces variance by $9-2.25=6.75$. The result is consistent with the variance decomposition because the conditional estimator is less variable than a single observation.

Check. The four observations sum to 20, and the conditioned estimate is their mean, 5. A common error is to treat the observed value $X_1=2$ as if it were the conditional expectation; conditioning averages over the possible arrangements given the sufficient sum.

Conditioning Products to Estimate the Squared Mean

When both normal parameters are unknown, an unbiased starting estimator of $\mu^2$ is $X_1X_2$, since independence gives $E[X_1X_2]=E[X_1]E[X_2]=\mu^2$. To condition it on $T=(T_1,T_2)$, use symmetry across all ordered pairs of distinct observations. The sum of their products is $T_1^2-T_2$, because expanding $T_1^2$ gives the sum of squares plus all distinct cross-products.

E[X_1X_2\mid T_1,T_2] = \frac{T_1^2-T_2}{n(n-1)}.

There are $n(n-1)$ ordered pairs with distinct indices, and each has the same conditional expected product. Their conditional expectations sum to $T_1^2-T_2$; dividing by the number of pairs gives the formula.

Worked Example: Conditioning a Product Estimator of the Squared Mean

Let $X_1,\ldots,X_4$ be independent observations from $N(\mu,\sigma^2)$, with both parameters unknown. Use $W=X_1X_2$ to estimate $\mu^2$. For the observed sample $(1,3,4,6)$, calculate the conditioned estimate and verify the pair-product calculation. State the common mistake avoided.

Method: average over distinct pairs using the sufficient statistics. The starting estimator is unbiased because $E[X_1X_2]=\mu^2$. The sufficient-statistic values are $T_1=1+3+4+6=14$ and $T_2=1^2+3^2+4^2+6^2=1+9+16+36=62$.

The conditional expectation is $(T_1^2-T_2)/(4\cdot3)=(14^2-62)/12=(196-62)/12=134/12=67/6$, or approximately $11.1667$. This also follows by summing the six unordered pair products: $1\cdot3+1\cdot4+1\cdot6+3\cdot4+3\cdot6+4\cdot6=3+4+6+12+18+24=67$. Their average is $67/6$, as required.

Check. The conditional estimate is a function of $T_1,T_2$ and remains unbiased by iterated expectation. A common mistake is to multiply $E[X_1\mid T]$ by $E[X_2\mid T]$. Conditional expectations do not generally turn a product into a product of conditional expectations; that incorrect calculation gives $(14/4)^2=12.25$, not $67/6$.

Conditioning a Difference to Estimate the Variance

For $i\ne j$, the difference $X_i-X_j$ has mean zero and variance $2\sigma^2$. Therefore $W=(X_1-X_2)^2/2$ is unbiased for $\sigma^2$. Conditional symmetry lets us average this estimator over all unordered pairs. The sum of squared pairwise differences satisfies $\sum_{i<j}(X_i-X_j)^2=nT_2-T_1^2$. This follows by expanding each square: each $X_i^2$ appears $n-1$ times, while the cross-products combine to give the stated expression.

There are $n(n-1)/2$ unordered pairs. Averaging their squared differences and then dividing by two gives \[ E\left[\frac{(X_1-X_2)^2}{2}\,\middle|\,T_1,T_2\right] = \frac{nT_2-T_1^2}{n(n-1)} = \frac{T_2-T_1^2/n}{n-1}. \] The last expression is the usual unbiased sample variance. Here it arises as a conditional expectation, not as an assumption about which estimator is best among all possible estimators.

Worked Example: Conditioning a Pairwise Difference

Let $X_1,\ldots,X_4$ be independent observations from $N(\mu,\sigma^2)$, with both parameters unknown. Start with $W=(X_1-X_2)^2/2$, an unbiased estimator of $\sigma^2$. For the observed sample $(1,2,5,8)$, calculate its conditional expectation using the sufficient statistics, and check it by averaging over all pairs.

Method: average the pairwise squared differences. The sufficient-statistic values are $T_1=1+2+5+8=16$ and $T_2=1+4+25+64=94$. Thus the conditioned estimate is $(4\cdot94-16^2)/(4\cdot3)=(376-256)/12=120/12=10$.

For a direct check, the six squared differences are $(1-2)^2=1$, $(1-5)^2=16$, $(1-8)^2=49$, $(2-5)^2=9$, $(2-8)^2=36$, and $(5-8)^2=9$. Their sum is $1+16+49+9+36+9=120$. The average of the six squared differences is $120/6=20$; dividing by two gives $10$, matching the conditional expectation.

Check. The result is also $(T_2-T_1^2/4)/3=(94-64)/3=10$. The starting estimator has expectation $\operatorname{Var}(X_1-X_2)/2=2\sigma^2/2=\sigma^2$. A common mistake is to average the squared differences but forget the original factor of one-half.

When Conditioning Changes Nothing—and When It Matters

A conditioned estimator is unchanged when its original value is already determined by the sufficient statistic. For instance, with an unknown-parameter normal sample, $\bar X=T_1/n$ is a function of $T_1$, so $E[\bar X\mid T_1,T_2]=\bar X$. In contrast, $X_1$ generally varies across sample arrangements with the same sum, and $X_1X_2$ generally varies across arrangements with the same sum and sum of squares. Conditioning replaces that arrangement-specific value with its conditional average.

This distinction is useful when assessing an estimator. First check whether it is unbiased for the stated target. Then identify a sufficient statistic for the model and calculate the conditional expectation. Finally, use the variance decomposition to determine whether the change is strictly beneficial or leaves the estimator unchanged. Sufficiency alone does not make every estimator a function of the sufficient statistic, and conditioning does not mean that the observations become conditionally independent.

Key takeaway: In a normal sample, permutation symmetry turns conditional expectations into averages across observations or pairs. An estimator already determined by the sufficient statistic is unchanged; otherwise, conditioning preserves unbiasedness and can remove variation.

Check Your Understanding

For each problem, state the sufficient-statistic calculation and use symmetry where needed.

Suppose $X_1,\ldots,X_5$ are independent $N(\mu,4)$ observations, with known variance. An unbiased estimator of $\mu$ is $X_1$. If the observed sample sum is 30, what is the Rao–Blackwellized estimate? (Answer: $30/5=6$.)
Suppose $X_1,X_2,X_3$ are independent $N(\mu,\sigma^2)$ observations with both parameters unknown. The observed values are $0,2,4$, and $X_1^2$ is used to estimate $\mu^2+\sigma^2$. Find its conditional expectation given the sufficient statistics. (Answer: $T_2/3=(0+4+16)/3=20/3$.)
For four independent $N(\mu,\sigma^2)$ observations with both parameters unknown, use $X_1X_2$ to estimate $\mu^2$. If the observations are $1,3,5,7$, calculate the conditioned estimate from $T_1$ and $T_2$. (Answer: $(16^2-84)/(4\cdot3)=172/12=43/3$.)
For three independent $N(\mu,\sigma^2)$ observations with both parameters unknown, use $(X_1-X_2)^2/2$ to estimate $\sigma^2$. Given observations $1,3,5$, compute the conditional estimate using the sufficient statistics. (Answer: $(35-9^2/3)/(3-1)=(35-27)/2=4$.)
In an independent normal sample with unknown mean and variance, an estimator is $\bar X$. Explain whether conditioning it on $(T_1,T_2)$ changes it. (Answer: It is unchanged because $\bar X=T_1/n$ is already a function of the sufficient statistic.)

What You'll Learn