Efficiency Under Nested Model Restrictions

Why Restrictions Can Change Precision

A statistical model describes which probability distributions are regarded as possible. A more restrictive model rules out some distributions that a broader model allows. If the restriction is correct, it can reduce uncertainty: fewer possible local changes in the data distribution need to be accommodated by an estimator. The precise first-order consequence is a change in the tangent space and, consequently, in the canonical gradient and efficiency bound.

This comparison is local. It concerns two models at the same distribution \(P\), and it concerns the same parameter in both models. A smaller bound is a statement about the best attainable asymptotic variance under the smaller model if its restrictions hold—not a guarantee that every estimator will be more precise in finite samples.

Definition: Let \(\mathcal M_0\subseteq\mathcal M_1\) be two models, and let \(P\in\mathcal M_0\). The tangent space \(T_i\) at \(P\) is the closed linear span in \(L_2(P)\) of scores of differentiable parametric submodels through \(P\) that lie in \(\mathcal M_i\). Because every path in \(\mathcal M_0\) is also a path in \(\mathcal M_1\), \(T_0\subseteq T_1\). Scores and tangent-space elements have mean zero and finite second moment under \(P\).

For a real-valued parameter \(\psi(P)\), pathwise differentiability means that along every such path with score \(s\), the derivative of the parameter can be represented as \(E_P[\phi s]\) for a mean-zero, square-integrable function \(\phi\). A function with this representation is a gradient. Among gradients in a model's tangent space, the canonical gradient is the unique one with the smallest \(L_2(P)\) norm. Its variance is the model's efficiency bound for one observation; for an independent sample of size \(n\), the corresponding lower bound on asymptotic variance at the \(\sqrt n\) scale is the same variance.

Definition: Write \(\phi_1\) for the canonical gradient of \(\psi\) in the larger model \(\mathcal M_1\), and \(\phi_0\) for its canonical gradient in the smaller model \(\mathcal M_0\). The efficiency bounds are \(E_P[\phi_1^2]\) and \(E_P[\phi_0^2]\), respectively. These quantities are compared at the common distribution \(P\).

The Projection Relationship

Suppose \(\psi\) is pathwise differentiable in the larger model. The larger-model canonical gradient \(\phi_1\) represents the derivative along every score in \(T_1\), and therefore also along every score in \(T_0\). The smaller model, however, asks for a representing gradient only on \(T_0\). Orthogonally projecting \(\phi_1\) onto \(T_0\) preserves all its inner products with elements of \(T_0\), while removing components that are irrelevant to derivatives along the restricted paths.

\phi_0=\Pi_{T_0}\phi_1

Here \(\Pi_{T_0}\) denotes orthogonal projection in \(L_2(P)\). This equation relies on the models being nested at \(P\), on the parameter being the same in both models, and on pathwise differentiability in the larger model. The projection itself is in \(T_0\), so it is the canonical gradient there: it represents the derivative on \(T_0\), and it has minimum norm among gradients that do so.

Restrict the derivative identity.
For every \(s\in T_0\), the larger-model gradient satisfies \(E_P[\phi_1s]=\dot\psi_P(s)\), where \(\dot\psi_P(s)\) is the derivative along score \(s\).

Project onto the smaller tangent space.
By the defining property of projection, \(E_P[(\phi_1-\Pi_{T_0}\phi_1)s]=0\) for every \(s\in T_0\). Thus the projection has the same inner products with restricted scores as \(\phi_1\) does.

Use the canonical-gradient criterion.
The projection lies in \(T_0\) and represents the derivative on \(T_0\). It is therefore the smaller-model canonical gradient.

The Pythagorean identity for an orthogonal projection gives the bound comparison:

E_P[\phi_1^2] = E_P[\phi_0^2]+E_P[(\phi_1-\phi_0)^2], \qquad E_P[\phi_0^2]\leq E_P[\phi_1^2].

The reduction is exactly the squared \(L_2(P)\) norm of the part of the larger-model canonical gradient orthogonal to \(T_0\). If \(\phi_1\in T_0\), the projection changes nothing and the bounds are equal. Thus a model can be strictly smaller yet give no first-order precision gain for a particular parameter. The removed directions matter only to the extent that the larger-model gradient has a component in them.

Worked Example: A Two-Direction Projection

Consider a local model in which \(T_1\) is spanned by two orthonormal, mean-zero score directions \(u_1\) and \(u_2\), so \(E_P[u_j^2]=1\) and \(E_P[u_1u_2]=0\). Suppose the restricted model has \(T_0\) spanned only by \(u_1\). For a parameter whose larger-model canonical gradient is \(\phi_1=2u_1+3u_2\), find both canonical gradients and bounds.

Since projection onto the span of \(u_1\) retains the \(u_1\) component and removes the orthogonal \(u_2\) component,

\phi_0=\Pi_{T_0}\phi_1=2u_1.

The larger-model bound is

E_P[\phi_1^2] =E_P[(2u_1+3u_2)^2] =4E_P[u_1^2]+12E_P[u_1u_2]+9E_P[u_2^2] =4+0+9=13.

The restricted-model bound is \(E_P[\phi_0^2]=4E_P[u_1^2]=4\). The difference is \(13-4=9=E_P[(3u_2)^2]\). The restriction removes a direction that contributes to the larger-model gradient, so the bound falls by exactly that component's variance.

A Distributional Restriction Can Improve a Bound

The projection calculation is useful when a restriction changes the tangent space in directions relevant to the target. The following example compares an unrestricted distribution model with a normal model. It is a local efficiency comparison at a standard normal distribution, which belongs to both models.

Worked Example: The Third Moment Under a Normal Restriction

Let \(X\) have distribution \(P=N(0,1)\), and let the target be the raw third moment \(\psi(P)=E_P[X^3]\). Compare the unrestricted model for the distribution of \(X\) with the model that restricts \(X\) to be normal with unknown mean \(\mu\) and variance \(\sigma^2\).

Unrestricted model. The canonical gradient for an expectation \(E_P[g(X)]\) is \(g(X)-E_P[g(X)]\). Here \(g(X)=X^3\), and \(E_P[X^3]=0\), so \(\phi_1=X^3\). Its bound is the sixth moment of a standard normal variable:

E_P[\phi_1^2]=E_P[X^6]=15.

Normal model. At \((\mu,\sigma^2)=(0,1)\), the scores for \(\mu\) and \(\sigma^2\) are \(S_\mu=X\) and \(S_{\sigma^2}=(X^2-1)/2\). Thus the tangent space is spanned by these two scores. Under a normal distribution, the target is \(\psi(\mu,\sigma^2)=\mu^3+3\mu\sigma^2\), so at \((0,1)\) its derivatives with respect to \(\mu\) and \(\sigma^2\) are 3 and 0. The candidate canonical gradient \(3X\) represents those derivatives:

E_P[3X S_\mu]=E_P[3X^2]=3, \qquad E_P[3X S_{\sigma^2}] =E_P\!\left[\frac{3X(X^2-1)}{2}\right]=0.

It belongs to the normal tangent space, so it is the normal-model canonical gradient. Its bound is

$$ E_P[(3X)^2]=9E_P[X^2]=9. $$

The normal restriction lowers the bound from 15 to 9 at this \(P\). Geometrically, \(X^3\) projects onto the normal tangent space as \(3X\): it has inner product 3 with \(X\), and inner product 0 with \((X^2-1)/2\). This comparison assumes that the normal restriction is correct; it does not say that a normal-model analysis is safe for arbitrary distributions.

When the Gain Is Zero—and When It Is Not Valid

The projection result distinguishes two questions that are easy to conflate: whether a restriction is genuinely informative for a target at \(P\), and whether the restriction is true. The former is answered by the component \(\phi_1-\phi_0\). The latter is a property of the data-generating distribution and cannot be established merely by calculating a smaller model-based bound.

For example, in the normal comparison above, the unrestricted mean has canonical gradient \(X-\mu\). At a normal \(P\), this gradient already belongs to the normal tangent space. Projecting it onto that space leaves it unchanged. Restricting to normal distributions therefore gives no first-order reduction in the bound for the mean at that \(P\), even though it does reduce the bound for the third moment.

Worked Example: A False Normal Restriction Changes the Target

Suppose the true distribution instead assigns probability \(2/3\) to \(X=-1\) and probability \(1/3\) to \(X=2\). Calculate its first three raw moments, then compare the target \(E[X^3]\) with the value implied by fitting a normal distribution using the mean and variance.

The mean is

E[X]=\frac{2}{3}(-1)+\frac{1}{3}(2)=0.

The second and third raw moments are

E[X^2]=\frac{2}{3}(1)+\frac{1}{3}(4)=2, \qquad E[X^3]=\frac{2}{3}(-1)+\frac{1}{3}(8)=2.

A normal distribution with the same mean and variance has \(\mu=0\) and \(\sigma^2=2\). Its implied third raw moment is \(\mu^3+3\mu\sigma^2=0\), not 2. A procedure that imposes normality and estimates the normal-model target would therefore target a different value from the true third moment in this example. The true distribution is outside the normal model, so the smaller normal-model efficiency bound does not describe regular inference for \(E[X^3]\) at this distribution.

This distinction matters in applications because a restriction may look attractive precisely when it yields a smaller standard error. But a smaller model-based standard error accounts only for sampling variability under that model. It does not account for bias from using a false model, or for a change in the estimand. For an efficiency comparison to support inference about the stated target, the data-generating distribution must lie in the restricted model and the target must be defined consistently there.

Common Pitfalls and Practical Interpretation

Comparing bounds at different distributions. The projection argument compares nested tangent spaces at the same \(P\). Bounds evaluated at different distributions need not obey the projection inequality.
Treating fewer parameters as an automatic gain. A smaller tangent space only reduces the bound if the larger-model canonical gradient has a nonzero component orthogonal to the smaller tangent space.
Calling a smaller model-based standard error evidence that the restriction is true. Efficiency calculations are conditional on the model. They do not validate its assumptions.
Confusing the efficiency bound with an estimator's actual variance. The bound is a lower limit for regular estimators under the specified model. An inefficient estimator can have larger variance, and finite-sample performance can differ.
Ignoring regularity. The projection comparison presumes a pathwise differentiable target in the larger model. If that condition fails, the usual larger-model canonical gradient and bound are not available in the stated form.

Key takeaway: At a common distribution in nested models, the restricted-model canonical gradient is the projection of the larger-model canonical gradient onto the smaller tangent space. The bound falls by the squared norm of the removed component—but the gain is meaningful for inference only when the restriction is valid and the target is unchanged.

Check Your Understanding

Use the tangent-space projection relationship to answer the following.

In nested models at the same \(P\), why does projecting the larger-model canonical gradient onto the smaller tangent space preserve derivatives along restricted paths?
If \(\phi_1=\phi_0+r\), where \(\phi_0\in T_0\) and \(r\) is orthogonal to \(T_0\), express the difference in efficiency bounds in terms of \(r\).
What condition on \(\phi_1\) makes the efficiency bounds equal even if \(T_0\) is a proper subset of \(T_1\)?
In the standard-normal third-moment example, what are the unrestricted and normal-model efficiency bounds, and what restriction produces their difference?
Why does a smaller normal-model bound not justify using it when the true distribution is the two-point distribution in the misspecification example?