Checking Efficiency in Standard Estimation Problems

What It Means to Check Efficiency

An estimator can be unbiased without using the available sample information as effectively as it could. The Cramér–Rao lower bound gives a benchmark: under suitable regularity conditions, it is a lower bound on the variance of any unbiased estimator of a parameter. To check efficiency, calculate the estimator’s variance under the model, calculate the bound using the full sample’s Fisher information, and compare the two.

The comparison is meaningful only when its assumptions hold. In this tutorial, the ordinary bound is used for an unbiased estimator of a single scalar parameter. A biased estimator requires a different bound, and models whose support depends on the parameter may violate regularity conditions. Those cases should not be treated as routine applications of the formula.

Definition: Suppose $T$ is an unbiased estimator of a scalar parameter $\theta$, and the model satisfies the regularity conditions for the Cramér–Rao bound. If the Fisher information in the sample is $I_n(\theta)$, then $$ \operatorname{Var}(T)\geq \frac{1}{I_n(\theta)}. $$ The estimator $T$ is efficient in the Cramér–Rao sense if its variance equals this bound. In this tutorial, its relative efficiency is defined as $$ \frac{1/I_n(\theta)}{\operatorname{Var}(T)}. $$ For an estimator that satisfies the bound, this ratio is at most 1; it equals 1 precisely when the estimator attains the bound.

For $n$ independent observations from the same regular model, the sample information is $I_n(\theta)=nI_1(\theta)$, where $I_1(\theta)$ is the information in one observation. This additivity lets us compare the variance of a sample-based estimator with the information in all $n$ observations, not merely the information in the observations used by the estimator.

Why the Bound Provides a Benchmark

A short justification explains the comparison. Let $L(\theta)$ be the likelihood and let $U(\theta)=\frac{\partial}{\partial\theta}\log L(\theta)$ be the score. Under the regularity conditions, $E[U(\theta)]=0$, $\operatorname{Var}(U(\theta))=I_n(\theta)$, and differentiation can be interchanged with expectation. If $T$ is unbiased for $\theta$, then $E[T]=\theta$, so differentiating this expectation gives $$ \operatorname{Cov}(T,U(\theta)) =E[TU(\theta)] =\frac{\partial}{\partial\theta}E[T] =1. $$ The Cauchy–Schwarz inequality therefore implies $$ 1=\operatorname{Cov}(T,U(\theta))^2 \leq \operatorname{Var}(T)\operatorname{Var}(U(\theta)) =\operatorname{Var}(T)I_n(\theta). $$ Rearranging yields $\operatorname{Var}(T)\geq 1/I_n(\theta)$. Equality requires the estimator’s centered value to be proportional to the score. Thus, equality is a specific condition to check, not a consequence of unbiasedness alone.

Worked Checks in Standard Models

Worked Example: A Sample Proportion in a Bernoulli Model

A hypothetical quality audit records whether each of 10 independently selected components passes inspection. Model each result as Bernoulli with pass probability $p$, where $0<p<1$. Suppose 3 components pass. Check whether the sample proportion is efficient for estimating $p$.

Method. First verify unbiasedness and calculate the estimator’s variance. Then calculate the total Fisher information and compare the variance with the ordinary Cramér–Rao bound.

Let $X_i=1$ for a pass and $X_i=0$ otherwise. The estimator is $\hat p=\bar X=\frac{1}{10}\sum_{i=1}^{10}X_i$. Since $E[X_i]=p$, $$ E[\hat p]=\frac{1}{10}\sum_{i=1}^{10}E[X_i] =\frac{10p}{10}=p. $$ It is unbiased. The observed value is $\hat p=3/10=0.3$. For the theoretical variance comparison, use the model parameter $p$, not the observed value $0.3$: $$ \operatorname{Var}(\hat p) =\frac{\operatorname{Var}(X_i)}{10} =\frac{p(1-p)}{10}. $$ For one Bernoulli observation, the log-likelihood is $x\log p+(1-x)\log(1-p)$. Its score is $\frac{x}{p}-\frac{1-x}{1-p}$, and the Fisher information is $I_1(p)=1/[p(1-p)]$. Independence gives $$ I_{10}(p)=\frac{10}{p(1-p)},\qquad \frac{1}{I_{10}(p)}=\frac{p(1-p)}{10}. $$ The estimator’s variance equals the bound for every $0<p<1$. Therefore $\hat p$ is efficient in the Cramér–Rao sense.

Check. The variance and bound have the same expression, and $3/10$ is a possible observed proportion in a sample of size 10. Common mistake avoided: substituting the observed proportion into the variance and treating the resulting estimate as the theoretical variance. Efficiency is established by comparing model expressions, not by the value observed in one sample.

Worked Example: Using Only Part of a Normal Sample

Suppose $X_1,\ldots,X_8$ are independent observations from $N(\mu,9)$, where the variance is known and $\mu$ is unknown. A proposed estimator averages only the first four observations: $T=(X_1+X_2+X_3+X_4)/4$. Check its efficiency and compare it with the full sample mean.

Method. Verify unbiasedness, calculate the variance of $T$, and use the information from all eight observations to obtain the bound.

Each observation has mean $\mu$, so $$ E[T]=\frac{4\mu}{4}=\mu. $$ Thus $T$ is unbiased. Independence and the known variance give $$ \operatorname{Var}(T) =\frac{1}{4^2}(4\cdot 9) =\frac{36}{16} =\frac{9}{4} =2.25. $$ For one $N(\mu,9)$ observation, the Fisher information about $\mu$ is $1/9$. All eight observations contribute, so $$ I_8(\mu)=\frac{8}{9},\qquad \frac{1}{I_8(\mu)}=\frac{9}{8}=1.125. $$ Since $9/4>9/8$, $T$ does not attain the bound. Its relative efficiency is $$ \frac{9/8}{9/4}=\frac{1}{2}=0.5. $$ The full sample mean has variance $9/8$, so it attains the bound and is efficient.

Check. The proposed estimator has variance twice the bound, and the full sample mean’s variance equals the bound. Common mistake avoided: calculating information from only the first four observations because only those observations appear in $T$. The bound for this model and sample uses all eight independent observations available.

Worked Example: The Sample Mean for an Exponential Model

A hypothetical monitoring study records five independent waiting times, each with an exponential distribution having mean $\theta>0$. For this model, the density is $f(x;\theta)=\frac{1}{\theta}e^{-x/\theta}$ for $x>0$. The recorded waiting times are 1, 2, 3, 4 and 5 time units. Check whether the sample mean is efficient for estimating $\theta$.

Method. Verify unbiasedness and calculate the sample mean’s variance. Then compute the information for the parameterization by the mean $\theta$.

The observed estimator is $$ \bar X=\frac{1+2+3+4+5}{5}=\frac{15}{5}=3. $$ For an exponential distribution with mean $\theta$, $E[X_i]=\theta$ and $\operatorname{Var}(X_i)=\theta^2$. Hence $$ E[\bar X]=\frac{5\theta}{5}=\theta,\qquad \operatorname{Var}(\bar X)=\frac{\theta^2}{5}. $$ The log-density for one observation is $-\log\theta-X_i/\theta$. Its score is $-1/\theta+X_i/\theta^2$. The score has mean zero and variance $1/\theta^2$, so $I_1(\theta)=1/\theta^2$. For five independent observations, $$ I_5(\theta)=\frac{5}{\theta^2},\qquad \frac{1}{I_5(\theta)}=\frac{\theta^2}{5}. $$ The sample mean’s variance equals the bound, so it is efficient. The calculation uses the mean parameter $\theta$; changing to a rate parameter would require changing both the information calculation and the parameter being estimated.

Check. The observations give a sample mean of 3, while the theoretical variance and bound both equal $\theta^2/5$. Common mistake avoided: confusing the observed estimate with its sampling variance, or using an information formula for a differently parameterized exponential model.

A Practical Comparison and Its Limits

In standard problems, the comparison is often easiest to organize in a few steps. Keep the model parameter, the proposed estimator and the number of independent observations explicit throughout.

Check the scope.
Confirm that the model is regular for the parameter value of interest and that the estimator is unbiased for that parameter. The ordinary bound used here does not cover biased estimators.

Find the estimator’s variance.
Use its sampling distribution or the model’s moments. Do not confuse a calculated estimate from one sample with the estimator’s variance across repeated samples.

Use total sample information.
For independent observations, add their Fisher information. Take the reciprocal to get the ordinary lower bound.

Compare and conclude.
If the variance equals the bound, the estimator is efficient in the Cramér–Rao sense. If the variance is larger, it is not; the relative efficiency defined above quantifies the comparison.

The phrase “in the Cramér–Rao sense” matters. Attaining the bound establishes that an unbiased estimator has the smallest variance allowed by this inequality under the stated model. It does not establish that the same estimator is best under every criterion, nor does it compare it with biased estimators. Also, the bound may be unattainable: an unbiased estimator can have variance strictly greater than the bound.

A regularity check is essential. For example, a uniform distribution on $(0,\theta)$ has support that changes with $\theta$. The usual fixed-support regularity conditions used in the score argument do not hold, so the ordinary Cramér–Rao formula cannot simply be applied to that model. Likewise, unbiasedness is not optional: a variance comparison alone does not justify using the ordinary bound for a biased estimator.

Key takeaway: A valid efficiency check is a three-way match: the estimator must be unbiased for the stated parameter, the model must satisfy the relevant regularity conditions, and its variance must be compared with the bound calculated from all available sample information. Equality establishes Cramér–Rao efficiency; a larger variance does not.

Check Your Understanding

For each problem, check unbiasedness where relevant, calculate the variance and bound, and decide whether the estimator is efficient in the Cramér–Rao sense.

In a hypothetical seed test, 20 independent seeds each germinate with probability $p$, where $0<p<1$. The estimator is the sample proportion. Calculate its variance and the bound, and determine whether it is efficient. (Answer: Both are $p(1-p)/20$; the sample proportion is efficient.)
Six independent counts follow a Poisson distribution with parameter $\lambda>0$. The observed counts are 1, 2, 0, 3, 2 and 4, and the estimator is their sample mean. Find the estimate, the variance, and the bound, and determine efficiency. (Answer: The estimate is $2$; the variance and bound are both $\lambda/6$, so the estimator is efficient.)
Suppose $X_1,\ldots,X_{10}$ are independent $N(\mu,4)$ observations, with known variance 4. An estimator averages only the first five observations. Find its variance, the bound based on all 10 observations, and its relative efficiency. (Answer: Variance $4/5=0.8$; bound $4/10=0.4$; relative efficiency $0.5$, so it is not efficient.)
Four independent waiting times have exponential distributions with mean $\theta>0$. Consider $T=X_1$ as an estimator of $\theta$. Is it unbiased? Compare its variance with the bound based on all four observations. (Answer: It is unbiased; its variance is $\theta^2$, the bound is $\theta^2/4$, and its relative efficiency is $1/4$, so it is not efficient.)
In a Bernoulli model with $p=0.25$, 12 independent observations are available. An estimator averages only the first three outcomes. Find its variance, the bound based on all 12 observations, and the relative efficiency. (Answer: Variance $0.25(0.75)/3=0.0625$; bound $0.25(0.75)/12=0.015625$; relative efficiency $0.25$, so it is not efficient.)

What You'll Learn