Why Completeness Turns a Good Estimator into the Only Best One
An unbiased estimator can often be improved by conditioning on a sufficient statistic: the conditional expectation keeps its target expectation while removing variation that is irrelevant to the parameter. But this does not, by itself, explain when the resulting estimator is the best unbiased estimator for every parameter value, or whether a different estimator could do just as well. The Lehmann–Scheffé theorem answers both questions when the sufficient statistic is also complete.
The theorem is a bridge between three ideas. Unbiasedness specifies the target, sufficiency captures the sample information about the parameter, and completeness rules out distinct unbiased functions of that statistic. Together, these conditions yield a uniformly minimum-variance unbiased estimator, or UMVU estimator.
Sufficiency means that the conditional distribution of the sample given \(T\) does not depend on the unknown parameter. Equivalently, in the usual dominated setting, the joint density or probability mass function factors into a product with the data depending on \(\theta\) only through \(T\). Completeness is a separate condition: it says no nonzero function of \(T\) can have expectation zero throughout the model.
The Lehmann–Scheffé Theorem
The theorem has two conclusions to establish. First, \(g(T)\) has variance no greater than any competing unbiased estimator. Second, any estimator that attains this minimum must agree with \(g(T)\) almost surely. The first conclusion uses conditioning and the variance decomposition; the second uses completeness.
Let \(U\) be any unbiased estimator of \(\delta(\theta)\) with finite variance, and define \(h(T)=E_\theta[U\mid T]\). Sufficiency ensures this conditional expectation can be represented as a function of \(T\) that does not depend on \(\theta\).
Since \(E_\theta[h(T)]=E_\theta[U]=\delta(\theta)=E_\theta[g(T)]\), the function \(h(T)-g(T)\) has expectation zero for every \(\theta\). Completeness gives \(h(T)=g(T)\) almost surely.
The conditional variance identity gives \(\operatorname{Var}_\theta(U)=E_\theta[\operatorname{Var}_\theta(U\mid T)]+\operatorname{Var}_\theta(E_\theta[U\mid T])\). Substituting \(E_\theta[U\mid T]=g(T)\) shows \(\operatorname{Var}_\theta(U)\geq\operatorname{Var}_\theta(g(T))\).
This also proves uniqueness. If \(U\) is itself UMVU, its variance cannot be strictly larger than that of \(g(T)\) at any parameter value. The variance decomposition then requires \(\operatorname{Var}_\theta(U\mid T)=0\) almost surely, so \(U=E_\theta[U\mid T]=g(T)\) almost surely. Thus no genuinely different estimator can achieve the same minimum throughout the model.
Worked Applications
Worked Example: Estimating the Exponential Mean
Model and assumptions. Let \(X_1,X_2,X_3\) be independent exponential random variables with rate \(\lambda>0\), each having density \(f(x;\lambda)=\lambda e^{-\lambda x}\) for \(x>0\). The target is \(1/\lambda\), the mean of one observation. Show that the sample mean is the unique UMVU estimator, and compute it for observations \(0.4, 0.7, 0.9\).
Method. Set \(T=X_1+X_2+X_3\). The joint density is \(\lambda^3 e^{-\lambda T}\) on the parameter-independent support \(x_1,x_2,x_3>0\); the factorization criterion therefore makes \(T\) sufficient. Also, \(T\) has a gamma distribution with shape \(3\) and rate \(\lambda\), with density \(\lambda^3 t^2e^{-\lambda t}/2\) for \(t>0\).
To see completeness, suppose \(E_\lambda[a(T)]=0\) for every \(\lambda>0\). The gamma density gives \(\int_0^\infty a(t)t^2e^{-\lambda t}\,dt=0\) for every \(\lambda>0\). Uniqueness of the Laplace transform implies \(a(t)t^2=0\) almost everywhere, hence \(a(T)=0\) almost surely. Thus \(T\) is complete.
The gamma mean is \(E_\lambda[T]=3/\lambda\), so \(g(T)=T/3\) is unbiased for \(1/\lambda\). The Lehmann–Scheffé theorem makes it the unique UMVU estimator. Its variance is \(\operatorname{Var}_\lambda(T/3)=3/\lambda^2\,/\,9=1/(3\lambda^2)\).
For the observations, \(T=0.4+0.7+0.9=2.0\), so \(g(T)=2.0/3=0.6667\), rounded to four decimal places. The result is checked by summing the three observations and dividing by three. The competing unbiased estimator \(X_1\) has variance \(1/\lambda^2\), which exceeds \(1/(3\lambda^2)\). A common mistake is to claim that \(X_1\) is UMVU simply because it is unbiased; conditioning it on the complete sufficient statistic produces the lower-variance sample mean.
Worked Example: A Uniform Maximum for a Squared Parameter
Model and assumptions. Let \(X_1,X_2,X_3\) be independent observations from the continuous uniform distribution on \((0,\theta)\), where \(\theta>0\). The target is \(\theta^2\). Find the unique UMVU estimator and evaluate it for observations \(1.2, 2.0, 1.5\).
Method. Let \(T=\max(X_1,X_2,X_3)\). The joint density is \(\theta^{-3}\) when all observations lie between \(0\) and \(\theta\), and is zero otherwise. Its dependence on the data is through the maximum, so \(T\) is sufficient.
The maximum has density \(3t^2/\theta^3\) for \(0<t<\theta\). To verify completeness, suppose \(E_\theta[a(T)]=0\) for all \(\theta>0\). Then \(\int_0^\theta a(t)t^2\,dt=0\) for every \(\theta>0\). Differentiating this identity at points where the integral is differentiable gives \(a(\theta)\theta^2=0\) almost everywhere. Therefore \(a(T)=0\) almost surely, proving completeness.
For \(k> -3\), direct integration gives \(E_\theta[T^k]=\int_0^\theta t^k(3t^2/\theta^3)\,dt=3\theta^k/(k+3)\). In particular, \(E_\theta[T^2]=3\theta^2/5\). Hence \(g(T)=(5/3)T^2\) is unbiased for \(\theta^2\), and the theorem makes it the unique UMVU estimator.
The observed maximum is \(T=2.0\), so the estimate is \((5/3)(2.0)^2=(5/3)(4)=20/3=6.67\), rounded to two decimal places. As a check on the moments, \(E_\theta[g(T)]=(5/3)(3\theta^2/5)=\theta^2\). Its variance is \((25/9)E_\theta[T^4]-\theta^4=(25/9)(3\theta^4/7)-\theta^4=4\theta^4/21\). A common mistake is to use \(T^2\) without its factor \(5/3\); its expectation is \(3\theta^2/5\), not \(\theta^2\).
Worked Example: Rao–Blackwellization in a Geometric Model
Model and assumptions. Let \(X_1,X_2\) be independent geometric random variables with parameter \(p\), where \(0<p<1\), supported on \(1,2,\ldots\), and with \(P_p(X_i=x)=p(1-p)^{x-1}\). Estimate \(1/p\), and evaluate the UMVU estimator for observations \(X_1=2\) and \(X_2=4\).
Method. Let \(T=X_1+X_2\). The joint probability is \(p^2(1-p)^{T-2}\), so \(T\) is sufficient. Its probability mass function is \(P_p(T=t)=(t-1)p^2(1-p)^{t-2}\), for integers \(t\geq2\).
For completeness, suppose \(E_p[a(T)]=0\) for every \(0<p<1\). Write \(q=1-p\). Dividing the expectation by \(p^2\) gives \(\sum_{j=0}^\infty a(j+2)(j+1)q^j=0\) for \(0<q<1\). A power series that vanishes on this interval has every coefficient equal to zero. Thus \(a(t)=0\) for all \(t\geq2\), and \(T\) is complete.
Since \(E_p[X_1]=1/p\), \(X_1\) is an unbiased starting estimator. Given \(T=t\), the possible ordered pairs \((X_1,X_2)\) are \((1,t-1),(2,t-2),\ldots,(t-1,1)\); each has the same conditional probability because their joint probabilities are all \(p^2(1-p)^{t-2}\). Therefore \(E_p[X_1\mid T=t]=(1+2+\cdots+(t-1))/(t-1)=t/2\). The conditional estimator \(T/2\) is unbiased, and completeness and sufficiency make it the unique UMVU estimator.
For the observed data, \(T=2+4=6\), giving \(T/2=3\). The estimate is checked against the sample mean, which is also \((2+4)/2=3\). At \(p=1/2\), \(\operatorname{Var}(X_1)=(1-p)/p^2=2\), while \(\operatorname{Var}(T/2)=2\operatorname{Var}(X_1)/4=1\). A common mistake is to assert that the original unbiased estimator \(X_1\) is already UMVU; its conditional expectation given the complete sufficient statistic has smaller variance.
What the Theorem Does—and Does Not—Guarantee
The theorem gives a decisive procedure: establish sufficiency and completeness for \(T\), construct a function \(g(T)\), and verify its expectation equals the target for every parameter value. Once these conditions hold, no separate comparison with every conceivable unbiased estimator is needed. The proof already compares against an arbitrary competitor through conditional expectation.
The conditions are essential. Sufficiency alone does not imply completeness, and an unbiased function of a sufficient statistic is not automatically UMVU if completeness has not been established. Nor does the theorem declare \(g(T)\) best among biased estimators, or among estimators for a different target. Its guarantee concerns unbiased estimators of the specified target with finite variance under the model.
A frequent exam error is to prove only that \(g(T)\) is unbiased and then conclude it is UMVU. Unbiasedness checks the target but says nothing alone about variance. Another error is to show a variance reduction for one competitor and infer global optimality. The theorem’s proof requires an arbitrary unbiased \(U\), conditions on \(T\), invokes completeness to identify the conditional estimator, and then applies the variance decomposition. Keeping those steps distinct makes the argument both rigorous and efficient.
Check Your Understanding
For each problem, check both the target expectation and the role of the complete sufficient statistic before applying the theorem.
- Let \(X_1,\ldots,X_4\) be independent exponential observations with rate \(\lambda>0\). Use \(T=\sum X_i\) to estimate \(1/\lambda\). Find the UMVU estimate for observations \(0.2,0.5,0.7,0.6\). (Answer: \(0.5000\))
- For two independent uniform observations on \((0,\theta)\), use the maximum to estimate \(\theta\). What is the UMVU estimate when the observations are \(2.4\) and \(1.8\)? (Answer: \(3.60\))
- For three independent geometric observations with support \(1,2,\ldots\) and success probability \(p\), estimate \(1/p\) using the complete sufficient sum. Evaluate the estimate for observations \(1,3,2\). (Answer: \(2\))
- For three independent exponential observations with rate \(\lambda\), estimate \(1/\lambda^2\) using the complete sufficient sum \(T\). Since \(E_\lambda[T^2]=12/\lambda^2\), find the UMVU estimate when the observations are \(0.5,1.0,1.5\). (Answer: \(0.7500\))
- For two independent uniform observations on \((0,\theta)\), compare the UMVU estimator of \(\theta^2\) with the unbiased estimator \(4X_1X_2\), using observations \(X_1=1\), \(X_2=3\). Give both estimates, then state which has variance no greater for every \(\theta\). (Answer: UMVU estimate \(18\); competing estimate \(12\); the UMVU estimator has variance no greater for every \(\theta\).)