Tutorials › VEE Mathematical Statistics › Worked UMVU Problems for Binomial, Poisson, and Normal Samples

Home  /  Tutorials  /  VEE Mathematical Statistics / Sufficiency and Optimal Estimation

VEE Mathematical Statistics · Sufficiency and Optimal Estimation · Tutorial 519 of 880

Worked UMVU Problems for Binomial, Poisson, and Normal Samples

Work through multi-step UMVU problems by finding a complete sufficient statistic, Rao–Blackwellizing an unbiased estimator, and applying the Lehmann–Scheffé theorem.

Advanced 12 min read

What You'll Learn

  • Identify complete sufficient statistics in binomial, Poisson, and normal samples
  • Choose unbiased starting estimators for nonlinear functions of model parameters
  • Calculate conditional expectations given sufficient statistics
  • Apply the Rao–Blackwell and Lehmann–Scheffé theorems to establish UMVU status
  • Detect common errors involving conditional probabilities and sample-size factors

From an Unbiased Estimator to a UMVU Estimator

An unbiased estimator is not automatically the best unbiased estimator: different unbiased statistics for the same parameter can have different variances. In an integrated estimation problem, the useful sequence is to find an unbiased starting statistic, condition it on a sufficient statistic, and then check whether that sufficient statistic is complete. When the conditions hold, the conditional expectation is not merely an improvement; it is the unique uniformly minimum-variance unbiased estimator.

This tutorial applies that sequence to binomial, Poisson, and normal samples. The main work is often choosing a starting estimator whose conditional expectation is tractable, then getting the conditioning calculation and its sample-size factors right.

Definition: An estimator \(U\) of a parameter function \(g(\theta)\) is unbiased if \(E_\theta[U]=g(\theta)\) for every parameter value \(\theta\). An unbiased estimator \(U^*\) is uniformly minimum-variance unbiased (UMVU) if, for every unbiased estimator \(U\) with finite variance, \(\operatorname{Var}_\theta(U^*)\leq \operatorname{Var}_\theta(U)\) for every \(\theta\).

A statistic \(T\) is sufficient for a parameter if, once \(T\) is known, the conditional distribution of the sample does not depend on that parameter. It is complete if any function \(h(T)\) with \(E_\theta[h(T)]=0\) for every allowed parameter value must satisfy \(h(T)=0\) with probability one for every such value. These definitions lead to the two results used throughout the examples.

Theorem: The Rao–Blackwell theorem states that if \(U\) is an unbiased estimator of \(g(\theta)\) and \(T\) is sufficient, then \(E[U\mid T]\) is unbiased for \(g(\theta)\) and has variance no greater than that of \(U\). If \(T\) is also complete, the Lehmann–Scheffé theorem says that any unbiased function of \(T\) is the unique UMVU estimator of its target.

Worked Problems in Three Sampling Models

For each model, state the assumptions first. The conditional calculations depend on them: in particular, the observations must be independent and identically distributed, and the sufficient statistic and its parameter space must match the stated model.

Worked Example: Estimating a Binomial Success-Failure Probability

Let \(X_1,\ldots,X_5\) be independent Bernoulli observations with common success probability \(p\), where \(0<p<1\). Estimate \(p(1-p)\) from the observed sample \((1,0,1,0,0)\), and establish that the estimator is UMVU.

Method and starting estimator. Since \(X_1\) and \(X_2\) are independent, \(U=X_1(1-X_2)\) is unbiased: \(E_p[U]=E_p[X_1]E_p[1-X_2]=p(1-p)\). The sum \(T=\sum_{i=1}^{5}X_i\) is sufficient and complete for \(p\) in this binomial model.

Condition on the sufficient statistic. Given \(T=k\), all arrangements of the \(k\) successes among the five positions are equally likely. The chance that the first observation is a success is \(k/5\). Given that, there are \(5-k\) failures among the four remaining positions, so the chance that the second observation is a failure is \((5-k)/4\). Therefore

$$ E[X_1(1-X_2)\mid T=k] =\frac{k}{5}\frac{5-k}{4} =\frac{k(5-k)}{20}. $$

Evaluate and conclude. The observed total is \(T=2\), so the conditional estimator is \(\frac{2(5-2)}{20}=\frac{6}{20}=0.30\). It is a function of the complete sufficient statistic and is unbiased by Rao–Blackwellization. Lehmann–Scheffé therefore establishes that \(T(5-T)/20\) is UMVU for \(p(1-p)\).

Common mistake avoided: using \(k/5\) for the conditional chance of two specified observations being success and failure. The second draw is from the four remaining positions, so its denominator is 4, not 5.

Worked Example: Estimating a Zero-Count Probability in a Poisson Sample

Let \(X_1,\ldots,X_4\) be independent observations from a Poisson distribution with parameter \(\lambda>0\). Estimate \(e^{-\lambda}\) from \((0,1,2,1)\), and show that the result is UMVU.

Method and starting estimator. For a Poisson variable, \(P_\lambda(X_1=0)=e^{-\lambda}\). Thus \(U=I(X_1=0)\), where \(I(\cdot)\) is 1 when its condition holds and 0 otherwise, is unbiased for \(e^{-\lambda}\). The total \(T=\sum_{i=1}^{4}X_i\) is sufficient and complete for \(\lambda\).

Find the conditional distribution. Given \(T=t\), the four counts have a multinomial distribution with total \(t\) and equal cell probabilities \(1/4\). Equivalently, \(X_1\mid T=t\) has a binomial distribution with \(t\) trials and success probability \(1/4\). Consequently,

$$ E[I(X_1=0)\mid T=t] =P(X_1=0\mid T=t) =\left(1-\frac14\right)^t =\left(\frac34\right)^t. $$

Evaluate and conclude. The observed total is \(T=0+1+2+1=4\). The conditional estimator is \(\left(\frac34\right)^4=\frac{81}{256}=0.31640625\), or \(0.3164\) to four decimal places. It is unbiased and is a function of the complete sufficient total, so the Lehmann–Scheffé theorem makes \((3/4)^T\) UMVU for \(e^{-\lambda}\).

Common mistake avoided: using \(e^{-T/4}\) as the conditional estimate. The requested estimator is the conditional probability of a zero count in one specified cell; the multinomial calculation gives \((3/4)^T\), not an exponential plug-in estimate.

Worked Example: Estimating the Square of the Mean in a Normal Sample

Let \(X_1,X_2,X_3\) be independent observations from \(N(\mu,\sigma^2)\), with both \(\mu\) and \(\sigma^2\) unknown. From the data \((1,3,5)\), estimate \(\mu^2\) and establish that the estimator is UMVU.

Method and starting estimator. Independence gives \(E[X_1X_2]=E[X_1]E[X_2]=\mu^2\), so \(U=X_1X_2\) is unbiased. A complete sufficient statistic for the normal model with both parameters unknown is the pair \((T_1,T_2)=(\sum_i X_i,\sum_i X_i^2)\). This follows from the full two-parameter exponential-family representation: its natural parameter space \(\{(\eta_1,\eta_2):\eta_1\in\mathbb R,\ \eta_2<0\}\) is open.

Condition using symmetry. Given \((T_1,T_2)\), the observations are exchangeable, so every distinct pair has the same conditional expectation. Expanding the square gives \(T_1^2=T_2+2\sum_{i<j}X_iX_j\). There are \(3\) distinct pairs when \(n=3\), and each has conditional expectation \(E[X_1X_2\mid T_1,T_2]\). Therefore

$$ E[X_1X_2\mid T_1,T_2] =\frac{T_1^2-T_2}{3(3-1)} =\frac{T_1^2-T_2}{6}. $$

Evaluate and conclude. For the data, \(T_1=1+3+5=9\) and \(T_2=1^2+3^2+5^2=35\). The conditional estimator is \(\frac{9^2-35}{6}=\frac{81-35}{6}=\frac{46}{6}=\frac{23}{3}\), or \(7.6667\) to four decimal places. It is the conditional expectation of an unbiased estimator and is a function of the complete sufficient statistic, so it is UMVU for \(\mu^2\).

Common mistake avoided: estimating \(\mu^2\) with \(\bar X^2\) and calling it unbiased. In fact, \(E[\bar X^2]=\mu^2+\sigma^2/3\) here; the conditional calculation accounts for the sample variation rather than leaving that positive bias in the estimate.

Why the Conditional Calculation Works

The three problems share a short justification. Suppose \(U\) is unbiased for \(g(\theta)\) and \(T\) is sufficient. The law of iterated expectation gives

$$ E_\theta[E_\theta(U\mid T)] =E_\theta[U] =g(\theta). $$

Thus conditioning preserves unbiasedness. The variance reduction follows from the conditional variance decomposition:

$$ \operatorname{Var}_\theta(U) =E_\theta[\operatorname{Var}_\theta(U\mid T)] +\operatorname{Var}_\theta(E_\theta[U\mid T]) \geq \operatorname{Var}_\theta(E_\theta[U\mid T]). $$

The inequality holds because conditional variance is nonnegative. If \(T\) is complete as well as sufficient, \(E[U\mid T]\) is an unbiased function of \(T\); Lehmann–Scheffé then gives uniqueness and minimum variance across all unbiased estimators. The completeness check is essential: Rao–Blackwellization alone guarantees a variance improvement or tie, but does not by itself establish that the result is UMVU.

1
Choose the target and model.
Write down the parameter function and the sampling assumptions, including the parameter space and whether any normal variance is known or unknown.
2
Find a simple unbiased starting statistic.
Indicators and products of independent observations are often useful when the target is a probability or a power of a mean.
3
Condition on a complete sufficient statistic.
Derive the conditional expectation carefully, then substitute the observed sufficient statistic.
4
State the conclusion with its justification.
Unbiasedness follows from iterated expectation; UMVU status follows when the result is a function of a complete sufficient statistic.

Common Exam Traps

  • Skipping the model assumptions. A conditional distribution such as the multinomial allocation in the Poisson example relies on independent Poisson observations with a common rate.
  • Confusing plug-in estimates with unbiased estimates. Substituting a sample mean into a nonlinear target does not generally preserve unbiasedness; for example, \(\bar X^2\) is biased for \(\mu^2\).
  • Dropping a factor in a conditional probability. For two specified positions in a binomial sample, the second conditional draw is from the remaining positions, not from the original sample size.
  • Claiming UMVU without completeness. A sufficient statistic permits Rao–Blackwellization. Completeness is the additional condition that supports the Lehmann–Scheffé conclusion.
Key takeaway: For these standard models, identify the complete sufficient statistic, construct a convenient unbiased starting estimator, and calculate its conditional expectation. Verify the target and the conditioning factors before invoking Lehmann–Scheffé.

Check Your Understanding

For each problem, identify the starting estimator or conditional calculation and verify the stated model conditions before concluding that an estimator is UMVU.

  1. Let \(X_1,\ldots,X_6\) be independent Bernoulli observations with success probability \(p\). Use \(X_1X_2\) to estimate \(p^2\). If the observed total is 4, what is the Rao–Blackwellized estimate? (Answer: \(4(4-1)/(6\cdot5)=0.4\).)
  2. Let \(X_1,\ldots,X_5\) be independent Poisson observations with common parameter \(\lambda\). Use the indicator that both \(X_1\) and \(X_2\) are zero to estimate \(e^{-2\lambda}\). If the total count is 3, what is its conditional expectation? (Answer: \((1-2/5)^3=27/125=0.216\).)
  3. For independent Bernoulli observations with common probability \(p\), estimate \(p(1-p)\) from \(n=4\) observations when the observed total is 2. Give the conditional estimator and identify the theorem supporting UMVU status. (Answer: \(2(4-2)/(4\cdot3)=1/3\); Lehmann–Scheffé, since the binomial total is complete sufficient.)
  4. Let \(X_1,X_2,X_3\) be independent Poisson variables with common parameter \(\lambda\), and estimate \(\lambda^2\) using \(X_1(X_1-1)\). If the observed total is 5, find the conditional estimate. (Answer: \(5(5-1)/3^2=20/9\).)
  5. Let \(X_1,\ldots,X_4\) be independent \(N(\mu,\sigma^2)\) observations with both parameters unknown. For data \((2,4,5,9)\), use the conditional expectation of \(X_1X_2\) given \((\sum X_i,\sum X_i^2)\) to estimate \(\mu^2\). (Answer: \((20^2-126)/(4\cdot3)=137/6\approx22.8333\), which is UMVU under the stated normal model.)

Previous

Next

Back to Tutorials
Clinical Biostats