Why Efficiency Calculations Need a Careful Check
A small variance can sound like proof that an estimator is good. It is not. Before interpreting a variance or applying a Cramér-Rao lower bound, check what quantity the estimator targets, whether bias matters to the comparison, how much information the full sample contains, and whether the model satisfies the bound’s conditions. Missing any one of these checks can produce a precise-looking but invalid conclusion.
This tutorial gives a practical diagnostic framework. In the discussion, an estimator is a statistic computed from the observed sample, and its performance is evaluated under a specified model and parameter value. An estimator’s bias and variance can depend on the unknown parameter; statements about performance must therefore identify the target and the model.
The decomposition follows by writing \(T-g(\theta)=(T-E_\theta[T])+(E_\theta[T]-g(\theta))\), squaring, and taking expectations. The cross term has expectation zero because \(E_\theta[T-E_\theta[T]]=0\). Thus variance measures spread around the estimator’s own mean, while squared bias measures the distance between that mean and the target.
First Check: Are the Estimators Aiming at the Same Target?
A variance comparison is meaningful as a direct comparison of precision only when the estimators target the same quantity under the same model. If two estimators target different functions of a parameter, a smaller variance does not make one preferable: the estimator may simply be estimating a different quantity or using a different scale.
Worked Example: Variances for Different Targets
Suppose \(X_1,\ldots,X_{25}\) are independent Bernoulli observations with success probability \(p=0.4\). Compare \(T_1=\bar X\) with \(T_2=2\bar X\), where \(\bar X\) is the sample proportion of successes. Diagnose the claim that the estimator with smaller variance is necessarily the better estimator.
Method. Calculate each estimator’s expectation and variance, then identify its target.
For a Bernoulli observation, \(E_p[X_i]=p\) and \(\operatorname{Var}_p(X_i)=p(1-p)=0.4(0.6)=0.24\). Independence gives $$ E_p[\bar X]=p=0.4,\qquad \operatorname{Var}_p(\bar X)=\frac{0.24}{25}=0.0096. $$ Thus \(T_1\) is unbiased for \(p\), and its variance is \(0.0096\).
For \(T_2=2\bar X\), linearity gives $$ E_p[T_2]=2p=0.8,\qquad \operatorname{Var}_p(T_2)=4\operatorname{Var}_p(\bar X) =4(0.0096)=0.0384. $$ So \(T_2\) is unbiased for \(2p\), not for \(p\). Its variance is larger, partly because its target is on twice the scale. The two variances do not compare the precision of two estimators of the same target.
Check. At \(p=0.4\), the targets are \(p=0.4\) and \(2p=0.8\), respectively. The calculation is internally consistent; the error would be treating these as competing estimators of one quantity.
Common mistake avoided: Ranking estimators only by their numerical variances without first checking what each estimates.
Second Check: Does Bias Matter to the Comparison?
For unbiased estimators of the same target, comparing variances is equivalent to comparing mean squared errors. If an estimator is biased, however, variance alone does not determine its total squared-error performance. A biased estimator can have smaller mean squared error than an unbiased alternative at a particular parameter value, but the comparison must state that value and should not be mistaken for a uniform conclusion over the whole parameter space.
Worked Example: A Biased Estimator With Lower Mean Squared Error
Let \(X_1,\ldots,X_4\) be independent observations from \(N(\mu,4)\), with the variance known to be \(4\). At the parameter value \(\mu=1\), compare \(T_1=\bar X\) and \(T_2=0.8\bar X\) as estimators of \(\mu\).
Method. Find bias and variance for each estimator, then combine them using the mean squared error identity.
The sample mean has \(E_\mu[\bar X]=\mu\) and \(\operatorname{Var}_\mu(\bar X)=4/4=1\). Therefore, at \(\mu=1\), \(T_1\) is unbiased and $$ \operatorname{MSE}_1(T_1)=1+0^2=1. $$
For \(T_2=0.8\bar X\), $$ E_\mu[T_2]=0.8\mu,\qquad \operatorname{Bias}_\mu(T_2)=0.8\mu-\mu=-0.2\mu. $$ At \(\mu=1\), its bias is \(-0.2\). Its variance is $$ \operatorname{Var}_1(T_2)=(0.8)^2\operatorname{Var}_1(\bar X) =0.64(1)=0.64. $$ Hence $$ \operatorname{MSE}_1(T_2)=0.64+(-0.2)^2 =0.64+0.04=0.68. $$ At this parameter value, \(T_2\) has lower mean squared error, despite being biased. This is a pointwise comparison: for other values of \(\mu\), its squared bias changes.
Check. The MSE decomposition gives \(0.64+0.04=0.68\), and \(0.68<1\). The comparison uses the same target, \(\mu\), and the same model.
Common mistake avoided: Calling an estimator more efficient just because it has smaller variance, while ignoring its bias when comparing overall squared error.
Third Check: Is the Information for the Whole Sample?
Fisher information measures how much the likelihood changes with a parameter. For independent observations, information adds. A frequent error is to use the information from one observation when a bound is required for the full sample, losing a factor of the sample size.
Worked Example: Restoring the Missing Sample-Size Factor
Suppose \(X_1,\ldots,X_{20}\) are independent Bernoulli observations with unknown success probability \(p\), and consider the bound at \(p=0.3\) for an unbiased estimator of \(p\). Find the total Fisher information and the Cramér-Rao lower bound. Then diagnose the result obtained by using only one observation’s information.
Method. Differentiate the log-likelihood for one observation to obtain its information, add over the 20 independent observations, and take the reciprocal.
For a single observation \(X\), the log-likelihood is \(X\log p+(1-X)\log(1-p)\). Its second derivative is $$ -\frac{X}{p^2}-\frac{1-X}{(1-p)^2}. $$ The one-observation Fisher information, the negative expected second derivative, is $$ I_1(p)=\frac{p}{p^2}+\frac{1-p}{(1-p)^2} =\frac{1}{p}+\frac{1}{1-p} =\frac{1}{p(1-p)}. $$ At \(p=0.3\), this is \(1/(0.3\cdot0.7)=1/0.21=100/21\), approximately \(4.7619\). Thus the sample information is $$ I_{20}(p)=20\left(\frac{100}{21}\right) =\frac{2000}{21}\approx95.2381. $$ The bound is therefore $$ \operatorname{Var}_{0.3}(T)\geq \frac{1}{I_{20}(0.3)} =\frac{21}{2000}=0.0105. $$
If one incorrectly uses \(I_1\) as though it were the full-sample information, the resulting bound is \(1/I_1=0.21\), twenty times too large. As a check, the sample proportion has variance \(0.3(0.7)/20=0.0105\), exactly the correct bound.
Common mistake avoided: Forgetting that independent observations contribute additive information, or comparing an estimator’s sample variance with a one-observation bound.
When the Cramér-Rao Bound Applies
The Cramér-Rao bound is not an automatic lower limit for every estimator in every model. Its standard derivation uses regularity conditions: the probability model must be differentiable in the parameter, the support must not vary with the parameter, differentiation and integration must be interchangeable under suitable conditions, and the score must have finite variance. These assumptions ensure, in particular, that the score has mean zero and that the covariance argument used in the derivation is valid. The precise technical conditions can be stated in different equivalent forms, but they must justify those steps.
For a scalar parameter \(\theta\), write the score as \(U_\theta(X)=\partial\log f(X;\theta)/\partial\theta\), where \(f\) is the density or probability mass function. Under the regularity conditions, \(E_\theta[U_\theta]=0\) and \(\operatorname{Var}_\theta(U_\theta)=I(\theta)\). If \(T\) is unbiased for \(\theta\), then \(E_\theta[T]=\theta\). Differentiating this expectation under the integral gives \(\operatorname{Cov}_\theta(T,U_\theta)=1\). The covariance inequality then yields the bound:
The key is not merely the final inequality. The identity \(\operatorname{Cov}_\theta(T,U_\theta)=1\) depends on unbiasedness for the target \(\theta\) and on differentiating the expectation correctly. If the target is \(g(\theta)\), the corresponding covariance is \(g'(\theta)\), giving the scalar bound \(\operatorname{Var}_\theta(T)\geq [g'(\theta)]^2/I(\theta)\) under the same regularity requirements and when \(T\) is unbiased for \(g(\theta)\).
In a multiparameter model, use the full information matrix rather than treating its diagonal entries as if each were an isolated scalar information. For an unbiased estimator of \(g(\boldsymbol{\theta})\), the corresponding variance bound is \(\nabla g(\boldsymbol{\theta})^\mathsf{T}I(\boldsymbol{\theta})^{-1}\nabla g(\boldsymbol{\theta})\), assuming the matrix is invertible and the regularity conditions hold. Off-diagonal information matters; a diagonal-only calculation can give the wrong bound.
Worked Example: A Bound That Fails for a Moving Support
Let \(X_1,X_2\) be independent uniform observations on \((0,\theta)\), with \(\theta>0\). Let \(M=\max(X_1,X_2)\) and \(T=3M/2\). Check the mean and variance of \(T\), then explain why a tempting score calculation does not give a valid Cramér-Rao bound.
Method. Calculate from the distribution of the maximum, then inspect the model’s support before using a score-based bound.
For \(0<m<\theta\), independence gives \(P_\theta(M\leq m)=(m/\theta)^2\). Differentiating, \(f_M(m)=2m/\theta^2\). Therefore $$ E_\theta[M]=\int_0^\theta m\frac{2m}{\theta^2}\,dm =\frac{2}{\theta^2}\frac{\theta^3}{3} =\frac{2\theta}{3}, $$ and $$ E_\theta[M^2]=\int_0^\theta m^2\frac{2m}{\theta^2}\,dm =\frac{2}{\theta^2}\frac{\theta^4}{4} =\frac{\theta^2}{2}. $$ Thus \(\operatorname{Var}_\theta(M)=\theta^2/2-(2\theta/3)^2=\theta^2/18\). It follows that $$ E_\theta[T]=\frac32\frac{2\theta}{3}=\theta,\qquad \operatorname{Var}_\theta(T)=\left(\frac32\right)^2\frac{\theta^2}{18} =\frac{\theta^2}{8}. $$
A tempting calculation takes the log-density of the two observations inside their support, \(-2\log\theta\), and differentiates it to get a score \(-2/\theta\). Squaring suggests information \(4/\theta^2\) and the bound \(\theta^2/4\). But the support is \(0<x_1,x_2<\theta\), which depends on \(\theta\). The regularity conditions behind the ordinary score calculation fail; in particular, the density’s changing boundary cannot be ignored when differentiating an expectation. The apparent bound would also contradict the calculated variance \(\theta^2/8<\theta^2/4\). That contradiction is a warning to check assumptions, not evidence that the estimator’s variance calculation is wrong.
Check. The maximum has mean \(2\theta/3\), so scaling it by \(3/2\) makes \(T\) unbiased. The model’s parameter-dependent support invalidates the tempting ordinary Cramér-Rao calculation.
Common mistake avoided: Differentiating only the density’s formula inside its support and overlooking that the support itself changes with the parameter.
A Practical Diagnostic Checklist
Before concluding that an estimator is efficient, or that a calculated bound proves it cannot be improved, work through the checks in order. Efficiency in the Cramér-Rao sense requires an unbiased estimator of the stated target whose variance attains a valid bound. Attaining the bound is a stronger claim than simply having low variance.
Write the estimand as \(g(\theta)\), not merely as “the parameter,” and verify that every estimator in the comparison targets that same quantity.
Calculate bias and variance. Use mean squared error when the comparison concerns squared-error loss and estimators may be biased.
For independent observations, add their information. In a multiparameter model, use the information matrix and the target’s gradient.
Check regularity, including parameter-independent support, before interpreting the result as a lower bound.
Attaining a valid Cramér-Rao bound establishes minimum variance among the relevant unbiased estimators under the stated model; it does not automatically settle every possible loss-based comparison.
For a regular scalar model, equality in the Cramér-Rao inequality requires the centered estimator to be proportional to the score: \(T-E_\theta[T]=c(\theta)U_\theta(X)\) almost surely, for an appropriate constant \(c(\theta)\). Thus a claim of efficiency needs both a valid bound and a reason the estimator attains it. A variance that happens to equal a number obtained from an invalid bound is not evidence of efficiency.
Check Your Understanding
For each problem, identify the relevant diagnostic and check the final calculation.
- For \(n=16\) independent Bernoulli observations with \(p=0.25\), find the total Fisher information and the Cramér-Rao bound for an unbiased estimator of \(p\). (Answer: \(I_{16}=256/3\approx85.3333\); bound \(=3/256\approx0.0117\).)
- For \(n=9\) independent \(N(\mu,9)\) observations, find the variance of \(\bar X\) and the bound for an unbiased estimator of \(\mu\). Does \(\bar X\) attain it? (Answer: variance \(=1\); bound \(=1\); yes.)
- An estimator \(T\) has bias \(0.3\) and variance \(0.64\) at a stated parameter value. Find its mean squared error at that value. (Answer: \(0.64+0.3^2=0.73\).)
- In a uniform \((0,\theta)\) model, take \(n=3\), \(M=\max(X_1,X_2,X_3)\), and \(T=4M/3\). State its mean and variance, and say whether the ordinary Cramér-Rao bound is applicable. (Answer: \(E_\theta[T]=\theta\); \(\operatorname{Var}_\theta(T)=\theta^2/15\); the ordinary bound is not applicable because the support depends on \(\theta\).)
- Two estimators have variances \(0.04\) and \(0.09\), but one estimates \(\theta\) and the other estimates \(3\theta\). Is the smaller variance sufficient to rank them as estimators of the same target? (Answer: no; they target different quantities, so the variance comparison alone does not rank their performance for one common estimand.)