Semiparametric Efficiency Bounds

Why an Efficiency Bound Matters

A semiparametric model allows some parts of the data-generating process to vary flexibly while still specifying a target parameter. That flexibility affects what can be learned from a sample: two estimators of the same target may both be regular and consistent, yet one can have greater asymptotic variance than the other. The semiparametric efficiency bound describes the smallest asymptotic variance available to regular estimators under the model.

The bound is determined by the model's score directions and by how the target changes along those directions. The canonical gradient combines this information into a single function. Its squared \(L^2(P)\) norm gives the scalar bound per observation. This tutorial derives that result and shows how to use it without confusing an attainable lower bound with a guarantee about any particular estimator.

From Scores to the Canonical Gradient

Let \(P\) be the data distribution at which inference is considered, and let \(\psi(P)\) be a scalar target. A differentiable one-dimensional parametric submodel \(P_t\), with \(P_0=P\), has score \(s\): informally, \(s\) describes the local change in log density at \(t=0\). The tangent space \(\mathcal T\) is the closure, in \(L^2(P)\), of the linear span of the scores of submodels through \(P\). Scores have mean zero, so \(\mathcal T\) is a subspace of the mean-zero square-integrable functions \(L^2_0(P)\).

The target is pathwise differentiable if its derivative along each such submodel is a continuous linear function of the score. Thus there is a derivative map \(\dot\psi_P\) such that \(\left.\frac{d}{dt}\psi(P_t)\right|_{t=0}=\dot\psi_P(s)\). The inner product used for score geometry is \(\langle f,g\rangle_P=E_P[fg]\).

The canonical gradient is the tangent-space representative, not just any influence function. This distinction matters because a function orthogonal to all allowed score directions cannot change the derivative identity, but it can increase variance.

Why the Canonical Gradient Gives a Lower Bound

Suppose a regular estimator has an asymptotically linear representation \[ \hat\psi-\psi(P)=\frac{1}{n}\sum_{i=1}^n \phi(O_i)+o_P(n^{-1/2}), \] where \(O_i\) are independent observations and \(\phi\) is its influence function. Regularity requires that the estimator respond correctly to local changes in the distribution. In particular, for every score \(s\in\mathcal T\), the derivative of the target must be represented by that influence function: \(E_P[\phi s]=\dot\psi_P(s)\).

Because \(\phi_{\mathrm{eff}}\) represents the same derivative, \(E_P[(\phi-\phi_{\mathrm{eff}})s]=0\) for every \(s\in\mathcal T\). Thus the difference \(\phi-\phi_{\mathrm{eff}}\) is orthogonal to the tangent space. Since \(\phi_{\mathrm{eff}}\in\mathcal T\), the two terms are orthogonal in the variance calculation. This gives the key decomposition:

The cross-product is zero because \(\phi-\phi_{\mathrm{eff}}\) is orthogonal to every element of \(\mathcal T\), including \(\phi_{\mathrm{eff}}\). Under the asymptotic linear representation, the estimator's asymptotic variance is \(E_P[\phi^2]/n\). Therefore the lower bound is \(E_P[\phi_{\mathrm{eff}}^2]/n\). Equality holds exactly when the estimator's influence function equals the canonical gradient in \(L^2(P)\).

Here \(\operatorname{avar}\{\sqrt n(\hat\psi-\psi(P))\}\) denotes the asymptotic variance of the scaled estimation error. Equivalently, the leading asymptotic variance of \(\hat\psi\) itself is the bound divided by \(n\). The result concerns regular estimators in the model; it does not rule out an irregular procedure behaving unusually at an isolated distribution.

Using Orthogonality to Assess an Estimator

All influence functions for a pathwise differentiable target have the form \(\phi_{\mathrm{eff}}+r\), where \(r\) is orthogonal to \(\mathcal T\). Such an \(r\) has no effect on the derivative along any allowed model path. It does, however, add \(E_P[r^2]\) to the asymptotic variance. Consequently, calculating an estimator's influence function and comparing it with the canonical gradient is a direct efficiency check.

This calculation also clarifies what a bound does not say. The existence of a canonical gradient establishes the minimum variance allowed by the local model geometry. It does not establish that a proposed estimation procedure has the canonical gradient, or even that a particular estimator achieves the bound. Those claims require an analysis of that estimator's regularity and influence function.

Vector-Valued Targets

For a target with \(k\) components, the canonical gradient is a \(k\)-vector \(\boldsymbol{\phi}_{\mathrm{eff}}\). Its covariance matrix is the efficiency bound per observation. If a regular estimator has influence vector \(\boldsymbol{\phi}\), write \(\boldsymbol{\phi}=\boldsymbol{\phi}_{\mathrm{eff}}+\boldsymbol r\), where each component of \(\boldsymbol r\) is orthogonal to \(\mathcal T\). The cross-covariance terms vanish, giving \(\operatorname{Var}_P(\boldsymbol{\phi})=\operatorname{Var}_P(\boldsymbol{\phi}_{\mathrm{eff}})+E_P[\boldsymbol r\boldsymbol r^\mathsf{T}]\). The final matrix is positive semidefinite, so the comparison is in the positive-semidefinite, or Loewner, order.

Common Mistakes and Practical Interpretation

• Calling every influence function canonical. An influence function can include a component orthogonal to the tangent space. It represents the derivative, but it need not minimize variance.
• Forgetting the sample-size scaling. \(E_P[\phi_{\mathrm{eff}}^2]\) is the bound for the asymptotic variance of the scaled error \(\sqrt n(\hat\psi-\psi(P))\). For the estimator itself, divide by \(n\).
• Claiming an estimator is efficient because it is consistent. Consistency alone does not imply regularity or bound attainment. Check the estimator's influence function against the canonical gradient.
• Treating the bound as a finite-sample guarantee. It is a local asymptotic lower bound for regular procedures under the specified model, not a promise about finite-sample variance.