Equality Conditions for Asymptotic Efficiency

Why Equality Requires More Than a Small Variance

The scalar information bound gives a lower limit on the asymptotic variance of a regular estimator. But the bound also has a precise equality condition: it is attained only when the estimator’s first-order fluctuation is aligned with the likelihood score. This condition is useful because it turns an abstract variance comparison into a direct calculation involving an influence function and a score.

We work with independent observations \(X_1,\ldots,X_n\) from a scalar parametric model with density or mass function \(f_\theta\), where \(\theta\) lies in an open subset of the real line. The results concern regular estimators that are asymptotically linear. They do not assert that every estimator with a small variance must be regular, or that every regular estimator has an influence function.

Score, Influence Function, and Regularity

Definition: At a fixed parameter value \(\theta\), the score for one observation is \(s_\theta(x)=\partial \log f_\theta(x)/\partial\theta\), where this derivative exists. The Fisher information for one observation is \(I(\theta)=E_\theta[s_\theta(X)^2]\). Assume the model is differentiable in quadratic mean at \(\theta\), \(E_\theta[s_\theta(X)]=0\), and \(0<I(\theta)<\infty\).

An estimator \(\hat\theta\) is asymptotically linear at \(\theta\) with influence function \(\psi_\theta\) if

\sqrt n(\hat\theta-\theta) = \frac{1}{\sqrt n}\sum_{i=1}^n \psi_\theta(X_i)+o_{P_\theta}(1), \qquad E_\theta[\psi_\theta(X)]=0, \qquad E_\theta[\psi_\theta(X)^2]<\infty.

The influence function describes the estimator’s first-order response to individual observations. Under the stated moment conditions, the central limit theorem gives asymptotic variance \(E_\theta[\psi_\theta(X)^2]\).

For the equality result, regularity means that under local parameter changes \(\theta+h/\sqrt n\), the limiting distribution of \(\sqrt n\{\hat\theta-(\theta+h/\sqrt n)\}\) is the same as its limiting distribution at \(\theta\), for each fixed real \(h\). We also require the usual local asymptotic normality conditions and that the asymptotic linear representation transfers to these local alternatives. Differentiability in quadratic mean supplies local asymptotic normality; the representation and its remainder are understood to satisfy the corresponding contiguity conditions.

In this setting, regularity imposes a normalization on the influence function:

Regularity identity: For a regular asymptotically linear estimator of \(\theta\), \(E_\theta[\psi_\theta(X)s_\theta(X)]=1\).

To see why, let \(\Delta_n=n^{-1/2}\sum_{i=1}^n s_\theta(X_i)\). Local asymptotic normality gives the log likelihood ratio for \(\theta+h/\sqrt n\) against \(\theta\) as \(h\Delta_n-\tfrac12h^2 I(\theta)+o_{P_\theta}(1)\). The joint central limit theorem for the sums of \(\psi_\theta(X_i)\) and \(s_\theta(X_i)\), followed by Le Cam’s third lemma, says that under this local alternative the limiting mean of the normalized influence sum shifts by \(hE_\theta[\psi_\theta s_\theta]\). Centering the estimator at the local target subtracts \(h\). Regularity requires the resulting limiting law not to shift, so \(h\{E_\theta[\psi_\theta s_\theta]-1\}=0\) for every \(h\), which gives the identity.

The Exact Equality Condition

Theorem: Under the preceding assumptions, the asymptotic variance of a regular asymptotically linear estimator satisfies \(E_\theta[\psi_\theta(X)^2]\geq 1/I(\theta)\). Equality holds if and only if \(\psi_\theta(X)=s_\theta(X)/I(\theta)\), \(P_\theta\)-almost surely.

Proof. The regularity identity and the definition of information give

\begin{aligned} E_\theta\left[\left\{\psi_\theta(X)-\frac{s_\theta(X)}{I(\theta)}\right\}^2\right] &=E_\theta[\psi_\theta(X)^2] -\frac{2}{I(\theta)}E_\theta[\psi_\theta(X)s_\theta(X)] +\frac{1}{I(\theta)^2}E_\theta[s_\theta(X)^2]\\ &=E_\theta[\psi_\theta(X)^2]-\frac{2}{I(\theta)}+\frac{1}{I(\theta)}\\ &=E_\theta[\psi_\theta(X)^2]-\frac{1}{I(\theta)}. \end{aligned}

The left side is nonnegative, proving the bound. It is zero exactly when the squared difference is zero almost surely, which is equivalent to \(\psi_\theta=s_\theta/I(\theta)\) almost surely. This proves both the inequality and its equality condition. \(\square\)

The calculation also gives a useful measure of excess asymptotic variance: it is exactly the mean squared distance between the influence function and the normalized score. Merely having a score correlated with the influence function is not enough. The two must agree almost surely after the score is divided by the information.

Checking the Condition in Parametric Models

Worked Example: Normal Mean with Known Variance

Suppose \(X_i\sim N(\mu,\sigma^2)\), with known \(\sigma^2>0\), and the target is \(\mu\). The one-observation log density, up to a constant, is \(- (x-\mu)^2/(2\sigma^2)\). Differentiating with respect to \(\mu\) gives

s_\mu(x)=\frac{x-\mu}{\sigma^2}, \qquad I(\mu)=E_\mu[s_\mu(X)^2] =\frac{E_\mu[(X-\mu)^2]}{\sigma^4} =\frac{1}{\sigma^2}.

For the sample mean \(\bar X\), the exact representation is \(\sqrt n(\bar X-\mu)=n^{-1/2}\sum_{i=1}^n(X_i-\mu)\). Thus its influence function is \(\psi_\mu(x)=x-\mu\). The normalized score is

\frac{s_\mu(x)}{I(\mu)} = \frac{(x-\mu)/\sigma^2}{1/\sigma^2} =x-\mu =\psi_\mu(x).

The equality condition holds, and the asymptotic variance is \(E_\mu[(X-\mu)^2]=\sigma^2=1/I(\mu)\). The sample mean attains the scalar bound.

Worked Example: Bernoulli Probability

Let \(X_i\sim\operatorname{Bernoulli}(p)\), with \(0<p<1\), and estimate \(p\) by \(\hat p=\bar X\). For one observation, the log likelihood is \(x\log p+(1-x)\log(1-p)\). Its derivative is

s_p(x)=\frac{x}{p}-\frac{1-x}{1-p} =\frac{x-p}{p(1-p)}.

Since \(E_p[(X-p)^2]=p(1-p)\), the information is \(I(p)=E_p[s_p(X)^2]=1/\{p(1-p)\}\). Also, \(\sqrt n(\bar X-p)=n^{-1/2}\sum_{i=1}^n(X_i-p)\), so \(\psi_p(x)=x-p\). Therefore

\frac{s_p(x)}{I(p)} = \frac{x-p}{p(1-p)}p(1-p) =x-p =\psi_p(x).

The estimator attains the bound \(1/I(p)=p(1-p)\), which is also its exact variance multiplied by \(n\). The restriction \(0<p<1\) matters: at the boundary, the usual interior score and regularity argument do not apply in the same way.

Worked Example: Exponential Rate

Suppose \(X_i\) has density \(f_\lambda(x)=\lambda e^{-\lambda x}\) for \(x\geq 0\), with \(\lambda>0\). The score and information are

s_\lambda(x)=\frac{1}{\lambda}-x, \qquad I(\lambda)=\operatorname{Var}_\lambda(X)=\frac{1}{\lambda^2}.

Consider \(\hat\lambda=1/\bar X\). Since \(E_\lambda[X]=1/\lambda\), the first-order Taylor expansion of \(g(y)=1/y\) at \(y=1/\lambda\), where \(g'(1/\lambda)=-\lambda^2\), gives

\sqrt n(\hat\lambda-\lambda) = \frac{1}{\sqrt n}\sum_{i=1}^n \left\{-\lambda^2\left(X_i-\frac{1}{\lambda}\right)\right\} +o_{P_\lambda}(1).

Thus \(\psi_\lambda(x)=\lambda-\lambda^2x\). The normalized score is \(\ s_\lambda(x)/I(\lambda)=\lambda^2(1/\lambda-x)=\lambda-\lambda^2x\), exactly the same influence function. Its variance is \(\lambda^4\operatorname{Var}(X)=\lambda^4/\lambda^2=\lambda^2=1/I(\lambda)\); the estimator attains the bound.

When an Estimator Misses the Bound

Worked Example: Adding an Orthogonal Fluctuation

Return to the normal model \(X_i\sim N(\mu,\sigma^2)\) with known \(\sigma^2\), and fix a nonzero constant \(c\). Define

\tilde\mu=\bar X+c(S_n^2-\sigma^2), \qquad S_n^2=\frac1n\sum_{i=1}^n(X_i-\bar X)^2.

The added term converges to zero, so \(\tilde\mu\) is consistent. To obtain its influence function, write \(S_n^2=n^{-1}\sum_{i=1}^n(X_i-\mu)^2-(\bar X-\mu)^2\). The last term is \(O_{P_\mu}(n^{-1})\), so it is negligible at the \(n^{-1/2}\) scale. The influence function is consequently \(\psi_\mu(x)=(x-\mu)+c\{(x-\mu)^2-\sigma^2\}\).

For a centered normal variable, the third moment is zero, so \(\operatorname{Cov}_\mu(X-\mu,(X-\mu)^2-\sigma^2)=0\). The variance of \((X-\mu)^2-\sigma^2\) is \(2\sigma^4\). Hence

E_\mu[\psi_\mu(X)^2] =\sigma^2+2c^2\sigma^4 >\sigma^2 =\frac{1}{I(\mu)}.

This estimator is regular: the added sample-variance term is unchanged by a shift in \(\mu\), and the same first-order representation applies under local mean changes. Its influence function has the required covariance with the score, but it is not equal to the normalized score; the extra orthogonal component strictly increases variance.

Interpretation and a Common Mistake

The score summarizes how the likelihood changes under a small parameter perturbation. A regular estimator must respond to that perturbation with the correct first-order amount, expressed by \(E_\theta[\psi_\theta s_\theta]=1\). Among all square-integrable influence functions with this response, the normalized score is the one with the smallest variance. Any component orthogonal to the score contributes variance without helping estimate the parameter.

Common mistake: concluding that an estimator attains the bound because its influence function is proportional to the score. Proportionality alone is not enough: the constant must be exactly \(1/I(\theta)\). If \(\psi_\theta=a\,s_\theta\), the regularity identity requires \(aI(\theta)=1\), hence \(a=1/I(\theta)\). Another mistake is to apply the result to an estimator without establishing regularity and an asymptotically linear representation. The theorem is a statement about that specified class, not an unconditional claim about every estimator.

Key takeaway: For a regular asymptotically linear estimator of a scalar parameter, the excess asymptotic variance over \(1/I(\theta)\) equals \(E_\theta[(\psi_\theta-s_\theta/I(\theta))^2]\). The bound is attained exactly when this difference vanishes almost surely.

Check Your Understanding

Use the equality condition and its assumptions to answer each question.

What identity does local regularity impose on the covariance between the influence function and the score?
If \(\psi_\theta=a\,s_\theta\), what value of \(a\) is required for a regular estimator to attain the bound?
In the Bernoulli example, identify the score, the information, and the normalized score.
Why does adding a mean-zero component orthogonal to the score increase asymptotic variance without changing the regularity identity?
Why is asymptotic linearity, together with regularity, part of the theorem’s scope?