Why a Scalar Target Has Its Own Bound
A model may have several unknown parameters, but a scientific question often concerns only one number determined by them: a contrast, a ratio, a product, or a predicted quantity. Estimating that scalar target does not require every parameter to be estimated with equal precision. Its first-order precision depends on how the target changes in each parameter direction and on how much information the data provide in those directions.
The resulting bound is a quadratic form: the target’s gradient on both sides of the inverse Fisher information matrix. It is not generally enough to inspect one diagonal entry of the inverse information, because changes in different parameters may be correlated.
Conditions and the Scalar Bound
Assume observations \(X_1,\ldots,X_n\) are independent and identically distributed from a regular parametric model with an open parameter space near the true \(\theta\). Write \(s_\theta(X)=\nabla_\theta\log f_\theta(X)\) for the column score vector. Assume the score has mean zero and finite, positive-definite information \(I(\theta)=E_\theta[s_\theta(X)s_\theta(X)^T]\). Also assume \(g\) is differentiable at \(\theta\).
For a regular asymptotically linear estimator \(T_n\) of \(g(\theta)\), write its first-order expansion as \(\sqrt n\{T_n-g(\theta)\}=n^{-1/2}\sum_{i=1}^n\psi_\theta(X_i)+o_p(1)\). Here the scalar influence function \(\psi_\theta\) has mean zero and finite variance. Regularity requires its first-order response to local changes in \(\theta\) to match the response of the target. In score form, this is the identity \(E_\theta[\psi_\theta(X)s_\theta(X)]=\nabla g(\theta)\). This identity is the key condition behind the bound: it says the estimator responds correctly to each local parameter direction.
The proof isolates the part of the influence function that is explained by the score. Define \(a=I(\theta)^{-1}\nabla g(\theta)\), and subtract the score-based component from the influence function: \(R=\psi_\theta(X)-a^Ts_\theta(X)\). The score identity gives \(E_\theta[R s_\theta(X)]=\nabla g(\theta)-I(\theta)a=0\). Thus \(R\) is uncorrelated with every component of the score. Since the score has mean zero, expanding the variance gives \[ \begin{aligned} \operatorname{Var}_\theta(\psi_\theta(X)) &=\operatorname{Var}_\theta(a^Ts_\theta(X)+R)\\ &=a^TI(\theta)a+\operatorname{Var}_\theta(R)\\ &=\nabla g(\theta)^TI(\theta)^{-1}\nabla g(\theta) +\operatorname{Var}_\theta(R). \end{aligned} \] The last term is nonnegative, proving the inequality. It is zero exactly when \(R=0\) almost surely, which gives the stated equality condition. \(\square\)
The bound is a first-order statement about regular estimators; it is not a claim that every estimator’s finite-sample variance must obey this inequality. Equality describes an efficient influence function: its entire first-order variation is the score component needed to track the target. The theorem does not assert that an estimator attaining equality exists in every model.
Worked Calculations
Worked Example: A Linear Contrast of Two Normal Means
Suppose \(X_i\sim N_2(\theta,\Sigma)\), where \(\theta=(\theta_1,\theta_2)^T\) and the known covariance matrix is \(\Sigma=\begin{pmatrix}4&1\\1&2\end{pmatrix}\). Find the bound for \(g(\theta)=\theta_1+2\theta_2\).
For a multivariate normal observation with known covariance, the score is \(s_\theta(X)=\Sigma^{-1}(X-\theta)\), so \(I(\theta)=\Sigma^{-1}\) and \(I(\theta)^{-1}=\Sigma\). The target gradient is \(c=\nabla g=(1,2)^T\). Therefore, \[ \begin{aligned} \nabla g^T I^{-1}\nabla g &=c^T\Sigma c\\ &=(1,2)\begin{pmatrix}4&1\\1&2\end{pmatrix}\begin{pmatrix}1\\2\end{pmatrix}\\ &=(1,2)\begin{pmatrix}6\\5\end{pmatrix}=16. \end{aligned} \] The bound for the variance of a regular estimator is \(16/n\) to first order. Indeed, \(\bar X_1+2\bar X_2\) has variance \(\{4+4(1)+4(2)\}/n=16/n\), so it attains the bound.
Worked Example: A Ratio of Two Multinomial Probabilities
Each observation falls in one of three categories with probabilities \(p,q,r=1-p-q\), where \(p>0\), \(q>0\), and \(r>0\). Take \(\theta=(p,q)^T\) and estimate \(g(p,q)=p/q\). Calculate the bound at \(p=0.2\), \(q=0.3\).
For one categorical observation, the information matrix and its inverse are \[ I(\theta)= \begin{pmatrix} 1/p+1/r&1/r\\ 1/r&1/q+1/r \end{pmatrix}, \qquad I(\theta)^{-1}= \begin{pmatrix} p(1-p)&-pq\\ -pq&q(1-q) \end{pmatrix}. \] The inverse can be checked by multiplying the two matrices. The gradient is \(\nabla g=(1/q,-p/q^2)^T\). Substitution and simplification give \[ \begin{aligned} \nabla g^T I^{-1}\nabla g &=\frac{p(1-p)}{q^2} +\frac{2p^2}{q^2} +\frac{p^2(1-q)}{q^3}\\ &=\frac{p(p+q)}{q^3}. \end{aligned} \] At \(p=0.2\), \(q=0.3\), this is \(0.2(0.5)/(0.3)^3=0.1/0.027\approx3.704\). Thus the first-order variance bound is approximately \(3.704/n\). This target depends on both category probabilities; ignoring their joint information would generally give the wrong result.
Worked Example: A Product of a Normal Mean and Variance
Let \(X_i\sim N(\mu,v)\), with both \(\mu\) and \(v>0\) unknown, and consider \(g(\mu,v)=\mu v\). Find the bound at \(\mu=1\), \(v=4\), and check it using a natural estimator.
The per-observation information for \((\mu,v)^T\) is diagonal: \(I(\mu,v)=\operatorname{diag}(1/v,1/(2v^2))\), so \(I^{-1}=\operatorname{diag}(v,2v^2)\). Since \(\nabla g=(v,\mu)^T\), \[ \nabla g^TI^{-1}\nabla g =v^3+2\mu^2v^2. \] At \((\mu,v)=(1,4)\), the value is \(4^3+2(1)^2(4^2)=64+32=96\); the first-order variance bound is \(96/n\).
For a check, estimate \(g\) by \(\bar X S_n^2\), where \(S_n^2=n^{-1}\sum_i(X_i-\bar X)^2\). Its first-order influence function is \(v(X-\mu)+\mu\{(X-\mu)^2-v\}\). Under the normal model, the two terms have covariance zero because the centered third moment is zero; their variances are \(v^3\) and \(2\mu^2v^2\). The estimator therefore has first-order variance \(v^3+2\mu^2v^2\), attaining the bound. At the stated parameter values this is \(96\).
Using the Bound When Parameters Are Estimated Together
A useful consequence connects this scalar result to the matrix bound. Suppose a regular estimator \(\hat\theta\) has \(\sqrt n(\hat\theta-\theta)\xrightarrow{d}N_p(0,V)\), where \(V-I(\theta)^{-1}\) is positive semidefinite. If \(g\) is differentiable, a first-order Taylor expansion gives \[ \sqrt n\{g(\hat\theta)-g(\theta)\} =\nabla g(\theta)^T\sqrt n(\hat\theta-\theta)+o_p(1). \] Thus the limiting variance for the plug-in estimator is \(\nabla g^TV\nabla g\). Because a positive-semidefinite matrix has nonnegative quadratic forms, \[ \nabla g^TV\nabla g =\nabla g^TI^{-1}\nabla g+ \nabla g^T(V-I^{-1})\nabla g \ \geq\ \nabla g^TI^{-1}\nabla g. \] This proves that the scalar bound is the relevant projection of the vector bound in the particular direction that changes the target. The plug-in estimator need not attain it if the vector estimator has excess variance in that direction.
When some coordinates are nuisance parameters, they remain part of \(\theta\) and of the information matrix. The gradient records how the target changes with those coordinates; the off-diagonal entries of \(I^{-1}\) account for their statistical dependence with the parameters of direct interest. The scalar quadratic form automatically includes both effects.
Common Pitfall and Interpretation
A frequent mistake is to treat the bound as just the inverse information for the parameter that appears most directly in the target. That shortcut drops both the target’s sensitivity to each coordinate and the dependence represented by off-diagonal information terms. The correction is to calculate the full gradient and the full inverse matrix, then evaluate the quadratic form. For a linear target \(g(\theta)=c^T\theta\), the gradient is \(c\), so the bound is \(c^TI^{-1}c\), not a single diagonal entry unless the target and information structure justify that simplification.
The bound answers a focused question: how variable must a regular first-order estimator of this particular scalar target be? A small value signals that the target is well informed in the direction that matters; a large value can arise because the target is sensitive to poorly informed parameter combinations. This makes the calculation useful before choosing an estimator, and it provides a benchmark for judging an estimator’s first-order efficiency.
Check Your Understanding
Use the gradient and the full inverse information matrix in each calculation.
- If \(I^{-1}=\begin{pmatrix}3&1\\1&2\end{pmatrix}\) and \(g(\theta)=\theta_1-\theta_2\), what is the first-order bound?
- For the multinomial ratio \(p/q\), why does the gradient have two components even though the target is scalar?
- State the condition on an influence function for equality in the scalar information bound.
- Why can a plug-in estimator have a larger first-order variance than the scalar bound even when its parameter estimator is regular?
- In the normal product example, which model property makes the two terms in the influence function uncorrelated?