Godambe Information and Efficiency of Estimating Functions

Why Estimating Functions Have Their Own Information

An estimating equation can target the same parameter in several ways. One choice may react strongly to a change in the parameter but also vary greatly from sample to sample; another may be less variable but less sensitive. Precision depends on both features. Godambe information combines them to quantify the information carried by an estimating function.

The comparison is useful because an estimating function need not be a likelihood score. Robust procedures, for example, can use bounded functions of the observations. Their sensitivity and variability can be compared with those of the score, making the efficiency cost of a choice explicit.

Definition: Let \(X\) have density or mass function \(f(x;\theta)\), where \(\theta\) is a \(p\)-dimensional parameter. A \(p\)-dimensional function \(\psi(X;\theta)\) is an unbiased estimating function if \(E_\theta[\psi(X;\theta)]=0\) for every \(\theta\) in the parameter space. Its sensitivity and variability are, respectively,

$$ A(\theta)=-E_\theta\!\left[\frac{\partial\psi(X;\theta)}{\partial\theta^T}\right], \qquad B(\theta)=\operatorname{Var}_\theta(\psi(X;\theta)). $$ When \(A\) and \(B\) are nonsingular, its Godambe information is \(G(\theta)=A(\theta)^T B(\theta)^{-1}A(\theta)\).

For a scalar parameter and scalar estimating function, this reduces to \(G=A^2/B\). The minus sign in the definition of sensitivity makes \(A\) positive for common equations such as \(\psi(X;\theta)=X-\theta\).

For independent observations \(X_1,\ldots,X_n\), an estimator \(\hat\theta\) may be defined by \(\sum_{i=1}^n\psi(X_i;\hat\theta)=0\). Under the usual consistency, differentiability, nonsingularity, and finite-second-moment conditions for Taylor linearization and the central limit theorem, its asymptotic covariance is

\operatorname{Avar}(\hat\theta) = \frac{1}{n}A^{-1}BA^{-T} = \frac{1}{n}G^{-1}.

Here \(A^{-T}\) means \((A^{-1})^T\). This is the sandwich covariance: variability \(B\) is between the two sensitivity factors. The formula concerns the first-order asymptotic covariance of a regular root of the estimating equation, not a universal finite-sample variance formula.

The Information Bound for Unbiased Estimating Functions

The likelihood score provides a benchmark. Write \(S(X;\theta)=\partial\log f(X;\theta)/\partial\theta\), and let \(I(\theta)=E_\theta[S S^T]\) be the Fisher information for one observation. Assume the model has common support, the density is differentiable in \(\theta\), differentiation can be interchanged with integration for the expectations below, and \(S\) has finite second moments. In particular, these conditions give \(E_\theta[S]=0\). Assume also that \(\psi\) is unbiased, has finite second moments, and can be differentiated under the expectation so that its sensitivity is defined by the displayed derivative.

Theorem: Under these conditions,

B-AI^{-1}A^T \ \text{is positive semidefinite}.

If \(A\) and \(B\) are nonsingular, then

$$ G=A^TB^{-1}A\ \leq\ I, \qquad A^{-1}BA^{-T}\ \geq\ I^{-1}. $$ For matrices, \(C\geq D\) means \(C-D\) is positive semidefinite. Thus the covariance bound applies to the asymptotic covariance of a regular estimator based on the estimating equation.

Proof. Differentiate the unbiasedness identity \(E_\theta[\psi(X;\theta)]=0\) with respect to \(\theta\). Differentiating both the function and the distribution gives

0 = E_\theta\!\left[\frac{\partial\psi(X;\theta)}{\partial\theta^T}\right] + E_\theta[\psi(X;\theta)S(X;\theta)^T].

Consequently \(E_\theta[\psi S^T]=A\). Since \(E_\theta[\psi]=E_\theta[S]=0\), these are also the cross-covariance matrices. Define the residual \(R=\psi-AI^{-1}S\). Its covariance is positive semidefinite, and direct expansion yields

\begin{aligned} \operatorname{Var}_\theta(R) &=B-AI^{-1}A^T-AI^{-1}A^T+AI^{-1}II^{-1}A^T\\ &=B-AI^{-1}A^T\ \geq\ 0. \end{aligned}

This proves the first inequality. Premultiplying by \(A^{-1}\) and postmultiplying by \(A^{-T}\) gives \(A^{-1}BA^{-T}\geq I^{-1}\). To obtain the information form, set \(C=B^{-1/2}AI^{-1/2}\). The first inequality implies \(C^TC\leq I_p\), so the largest singular value of \(C\) is at most one and \(CC^T\leq I_p\). Equivalently, \(B^{-1/2}AI^{-1}A^TB^{-1/2}\leq I_p\), which rearranges to \(A^TB^{-1}A\leq I\). \(\square\)

The proof also identifies equality. The residual has zero covariance precisely when \(R=0\) almost surely, which means \(\psi=AI^{-1}S\) almost surely. In other words, equality holds when the estimating function contains no component, beyond a nonsingular linear transformation, that is unrelated to the score.

Worked Comparisons in a Normal Location Model

Take independent observations from \(X\sim N(\mu,4)\), with the variance known and \(\mu\) unknown. The Fisher information for one observation is \(I=1/4\). We compare three unbiased estimating functions for \(\mu\). In each case, the information and variance calculations refer to one observation; the asymptotic covariance of the estimator has an additional factor \(1/n\).

Worked Example: The Score Attains the Bound

The score for \(\mu\) is \(\psi_s(X;\mu)=(X-\mu)/4\). It is unbiased because \(E_\mu[X-\mu]=0\). Its derivative with respect to \(\mu\) is \(-1/4\), so \(A_s=1/4\). Since \(\operatorname{Var}_\mu(X)=4\),

B_s=\operatorname{Var}_\mu\!\left(\frac{X-\mu}{4}\right) =\frac{4}{16}=\frac14, \qquad G_s=\frac{(1/4)^2}{1/4}=\frac14.

Thus \(G_s=I\), and the asymptotic variance of the score-based estimator is \(G_s^{-1}/n=4/n\). Solving the sample score equation gives \(\hat\mu=\bar X\). This choice attains the information bound.

Worked Example: A Cubic Estimating Function

Consider \(\psi_c(X;\mu)=(X-\mu)^3\). Symmetry of the normal distribution gives \(E_\mu[\psi_c]=0\). Its derivative is \(-3(X-\mu)^2\), so \(A_c=3E[(X-\mu)^2]=12\). A centered normal variable with variance \(4\) has sixth moment \(15(4^3)=960\); therefore

B_c=E_\mu[(X-\mu)^6]=960, \qquad G_c=\frac{12^2}{960}=\frac{3}{20}.

The corresponding asymptotic variance is \(G_c^{-1}/n=20/(3n)\), larger than the score-based \(4/n\). Its relative efficiency against the score is \(G_c/I=(3/20)/(1/4)=3/5\). The estimator is defined by \(\sum_i(X_i-\hat\mu)^3=0\); it is not generally the sample mean, despite targeting the same location parameter.

Worked Example: A Bounded Sign Estimating Function

Now use \(\psi_{\mathrm{sgn}}(X;\mu)=\operatorname{sign}(X-\mu)\), where the sign is \(1\) for a positive argument and \(-1\) for a negative argument. The event \(X=\mu\) has probability zero. The function is unbiased at the true \(\mu\), and

E_\mu[\psi_{\mathrm{sgn}}(X;\mu)] =P_\mu(X>\mu)-P_\mu(X<\mu)=0, \qquad B_{\mathrm{sgn}}=E_\mu[\psi_{\mathrm{sgn}}^2]=1.

Although the function is not differentiable at \(X=\mu\), its expectation is differentiable: \(E_\mu[\psi_{\mathrm{sgn}}(X;\mu)]=1-2F_\mu(\mu)\) when the argument \(\mu\) is viewed as the candidate parameter. More precisely, for a fixed true location \(\mu_0\), the expectation at candidate value \(t\) is \(1-2F_{\mu_0}(t)\); its derivative at \(t=\mu_0\) is \(-2f_{\mu_0}(\mu_0)\). Thus the sensitivity is \(2f_\mu(\mu)=1/\sqrt{2\pi}\), and

G_{\mathrm{sgn}}=\frac{(1/\sqrt{2\pi})^2}{1} =\frac{1}{2\pi}, \qquad G_{\mathrm{sgn}}^{-1}=\;2\pi.

Its asymptotic variance is \(2\pi/n\), and its relative efficiency is \((1/(2\pi))/(1/4)=2/\pi\). The sign function is bounded, but it is less efficient than the score under this uncontaminated normal model. The calculation uses differentiability of the expected estimating function near the target; pointwise differentiability of \(\psi_{\mathrm{sgn}}\) is not required.

How to Read the Comparison

Sensitivity alone is not information: a function can be multiplied by a large constant, increasing its sensitivity and variability together. In fact, replacing \(\psi\) by \(C\psi\), for any fixed nonsingular matrix \(C\), gives \(A^*=CA\) and \(B^*=CBC^T\). Substitution shows

G^* =A^TC^T(CBC^T)^{-1}CA =A^TB^{-1}A =G.

The estimating equation itself is unchanged by this transformation, and its information is unchanged as well. What matters is the balance between sensitivity and variability, not the numerical scale used to write the equation.

The score is optimal among regular unbiased estimating functions in the sense of the theorem: no such choice has Godambe information greater than Fisher information. A non-score choice can nevertheless be useful when its behavior under departures from the assumed model matters. The sign example illustrates the trade-off: boundedness can limit the influence of extreme observations, while its lower information under the normal model yields a larger asymptotic variance.

Common Mistake

Pitfall: Treating \(B\), the variability of the estimating function, as the estimator’s covariance. \(B\) is not the asymptotic covariance of the root. The correct covariance per observation is \(A^{-1}BA^{-T}\), or equivalently \(G^{-1}\). For example, in the cubic normal calculation, \(B=960\) while the asymptotic variance per observation is \(20/3\). Sensitivity must be included on both sides of the variability matrix.

A related mistake is to conclude that two estimating functions are equally efficient just because they have the same target. Unbiasedness identifies the target; it does not determine precision. Compute both \(A\) and \(B\), then compare \(G\) or the sandwich covariance.

Key takeaway: Godambe information measures the precision of an estimating function by combining its sensitivity and variability. Under regularity conditions, its information cannot exceed Fisher information; the corresponding sandwich covariance cannot be smaller than the inverse Fisher information.

Check Your Understanding

Use the definitions and comparisons above to check your understanding.

For a scalar estimating function, write the sensitivity, variability, and Godambe information.
Why does multiplying an estimating function by a nonzero constant leave its Godambe information unchanged?
What covariance identity follows from differentiating the unbiasedness condition \(E_\theta[\psi(X;\theta)]=0\)?
In the normal model with variance \(4\), which of the score and cubic estimating functions has greater Godambe information, and by what factor?
Why does the sign estimating function’s lack of pointwise differentiability at the target not by itself prevent calculation of its sensitivity?