Singular Jacobians and Weak Identification

Why the Population Jacobian Matters

For an estimator defined by estimating equations, the population equation describes how the model responds to changes in the parameter. Its Jacobian measures that response locally. If the Jacobian has a direction in which the response is zero, or nearly zero, sample fluctuations can produce a large change in the estimated parameter. The usual root-\(n\) approximation and sandwich covariance rely on being able to invert this local response.

A singular Jacobian is a warning, not a complete diagnosis. It can accompany nonidentification, but a parameter may still be locally identified through higher-order changes in the population equation. A nearly singular Jacobian can produce weak identification: the equation changes so little in some direction that ordinary sample noise dominates its local signal.

Definition: Let \(\Psi(\theta)=E[\psi(Z,\theta)]\) be a population estimating function, and let \(\theta_0\) satisfy \(\Psi(\theta_0)=0\). The root \(\theta_0\) is locally identified by the equation if there is a neighborhood \(U\) of \(\theta_0\) such that \(\Psi(\theta)=0\) for \(\theta\in U\) implies \(\theta=\theta_0\). If \(\Psi\) is differentiable at \(\theta_0\), its population Jacobian is \(A=\partial\Psi(\theta_0)/\partial\theta'\). A small minimum singular value of \(A\), relative to the scale of sampling noise, indicates weak local sensitivity.

When \(A\) is nonsingular, the inverse function theorem gives a sufficient condition for local identification. In regular settings it also lets us translate a small empirical error in the estimating equation into a small parameter error. When \(A\) is singular, that translation cannot be made by the usual first-order argument. The equation may still identify the parameter, but its first derivative does not reveal how.

The Regular Root-\(n\) Argument

The standard asymptotic result makes the role of invertibility explicit. Write \(\Psi_n(\theta)=n^{-1}\sum_{i=1}^n\psi(Z_i,\theta)\), and suppose \(\hat\theta\) solves \(\Psi_n(\hat\theta)=0\). For clarity, consider a fixed-dimensional parameter and independent, identically distributed observations. Assume \(\Psi(\theta_0)=0\), \(\hat\theta\xrightarrow{p}\theta_0\), and \(A\) is nonsingular. Also assume a central limit theorem \(\sqrt n\,\Psi_n(\theta_0)\xrightarrow{d}N(0,B)\), where \(B=\operatorname{Var}(\psi(Z,\theta_0))\), and a local expansion \[ \Psi_n(\hat\theta)=\Psi_n(\theta_0)+A(\hat\theta-\theta_0)+r_n, \] where, for \(\hat\theta\) in a neighborhood shrinking in probability to \(\theta_0\), \(r_n=o_p(\|\hat\theta-\theta_0\|)\). To establish the root-\(n\) rate, the expansion must hold uniformly enough in that neighborhood to give the usual local derivative control.

Theorem: Under the conditions just stated, if \(\hat\theta\) solves the sample equation, then \[ \sqrt n(\hat\theta-\theta_0)=-A^{-1}\sqrt n\,\Psi_n(\theta_0)+o_p(1) \xrightarrow{d}N(0,A^{-1}B A^{-T}). \]

Proof. Since \(A\) is nonsingular, there is a constant \(c>0\) such that \(\|Av\|\ge c\|v\|\) for every vector \(v\). The estimating equation and expansion give \(A(\hat\theta-\theta_0)=-\Psi_n(\theta_0)-r_n\). The central limit theorem implies \(\Psi_n(\theta_0)=O_p(n^{-1/2})\). By consistency and the stipulated local remainder control, with probability tending to one the remainder can be bounded by \(\|r_n\|\le (c/2)\|\hat\theta-\theta_0\|\). Therefore \[ c\|\hat\theta-\theta_0\| \le \|\Psi_n(\theta_0)\|+\|r_n\| \le \|\Psi_n(\theta_0)\|+(c/2)\|\hat\theta-\theta_0\|, \] so \(\hat\theta-\theta_0=O_p(n^{-1/2})\). The remainder is then \(o_p(n^{-1/2})\). Multiplying the expansion by \(\sqrt n\) and rearranging yields the stated linear representation. Applying the central limit theorem gives the stated normal limit and covariance. \(\square\)

This argument depends on \(A^{-1}\) both to control the parameter error and to express its limiting distribution. If \(A\) is singular, this proof stops; it does not follow that every estimator fails or that the parameter is unidentified. If \(A\) is nearly singular, its inverse can greatly amplify the empirical fluctuation. In the regular case, the covariance \(A^{-1}BA^{-T}\) is the sandwich covariance. The same instability affects its sample estimate when the estimated Jacobian is poorly conditioned.

What Singularity Does—and Does Not—Tell Us

A zero derivative can occur either when the population equation has several nearby roots or when it has a unique root whose identifying signal appears only at higher order. These possibilities have different implications and cannot be distinguished from the Jacobian alone.

Proposition: A singular Jacobian at a root does not, by itself, determine whether that root is locally identified. In one dimension, the functions \(\Psi_1(\theta)=\theta^2\) and \(\Psi_2(\theta)=0\) both satisfy \(\Psi_j(0)=0\) and \(\Psi_j'(0)=0\). The root \(0\) is locally identified for \(\Psi_1\), but not for \(\Psi_2\).

Proof. For \(\Psi_1\), the equation \(\theta^2=0\) implies \(\theta=0\), so zero is the only root in every neighborhood of zero. For \(\Psi_2\), every \(\theta\) is a root, so zero is not isolated. Both derivatives at zero are zero. Thus the same singular derivative is compatible with either conclusion. \(\square\)

Worked Example: Unique Identification With a Singular Derivative

Let \(X_1,\ldots,X_n\) be independent \(N(0,1)\) observations, let the parameter space be \(\theta\ge 0\), and define the population moment \(\Psi(\theta)=\theta^2\). Its only root is \(\theta_0=0\), so the parameter is identified by this equation. But \(\Psi'(0)=0\).

Use the sample moment \(\Psi_n(\theta)=\theta^2-\bar X\), where \(\bar X=n^{-1}\sum_i X_i\), and minimize \(\Psi_n(\theta)^2\) over \(\theta\ge0\). If \(\bar X\ge0\), the minimum is zero at \(\hat\theta=\sqrt{\bar X}\). If \(\bar X<0\), then \(\theta^2-\bar X=\theta^2+|\bar X|\) is minimized at \(\theta=0\). Hence \(\hat\theta=\sqrt{\max(\bar X,0)}\).

Since \(\sqrt n\,\bar X\sim N(0,1)\) exactly, write \(Z_n=\sqrt n\,\bar X\). Then \[ n^{1/4}\hat\theta=\sqrt{\max(Z_n,0)}. \] This has a nonnormal limit with probability \(1/2\) at zero and positive values with probability \(1/2\). In particular, the estimator is not root-\(n\) regular: when \(Z_n>0\), \(\sqrt n\,\hat\theta=n^{1/4}\sqrt{Z_n}\), which does not remain bounded in probability. Identification holds, but the quadratic population signal produces a slower, nonstandard rate.

Weak Identification and the Size of the Inverse

For a nonsingular matrix \(A\), its minimum singular value \(s_{\min}(A)\) satisfies \(\|A^{-1}\|=1/s_{\min}(A)\) under the Euclidean operator norm. Thus a small \(s_{\min}(A)\) means there is a unit direction \(v\) along which the population equation changes by very little: \(\|Av\|\) is small. Sample noise in that direction can then translate into a large parameter error.

For a sequence of models indexed by \(n\), let the Jacobian be \(A_n\). The first-order term in the estimator is \(-A_n^{-1}\Psi_n(\theta_0)\). Its magnitude depends on both the inverse Jacobian and the noise in the corresponding directions. Consequently, small singular values are most meaningful relative to the covariance \(B_n=\operatorname{Var}(\psi(Z,\theta_0))\), not in isolation. Standard root-\(n\) inference requires more than invertibility for each \(n\): the linear approximation and the behavior of \(A_n^{-1}\sqrt n\,\Psi_n(\theta_0)\) must remain suitable for the claimed limit.

Worked Example: A Weak Direction in a Triangular Sequence

For each sample size \(n\), observe independent vectors \(\varepsilon_i\sim N(0,I_2)\), and solve \(\Psi_n(\theta)=n^{-1}\sum_{i=1}^n(\theta_1-\varepsilon_{i1},\,\delta_n\theta_2-\varepsilon_{i2})'=0\), where \(\delta_n=n^{-\alpha}\) and \(0<\alpha<1/2\). The population root is \(\theta_0=(0,0)'\), and the Jacobian is \(A_n=\operatorname{diag}(1,\delta_n)\). Its minimum singular value is \(\delta_n\), which tends to zero.

Solving the two equations gives \(\hat\theta_1=\bar\varepsilon_1\) and \(\hat\theta_2=\bar\varepsilon_2/\delta_n\). Since \(\sqrt n\,\bar\varepsilon_2\sim N(0,1)\), \[ \hat\theta_2=n^{-1/2+\alpha}Z,\qquad Z\sim N(0,1). \] The second coordinate converges to zero, but at the slower rate \(n^{1/2-\alpha}\), not at the root-\(n\) rate. Indeed \(\sqrt n\,\hat\theta_2=n^\alpha Z\), whose variance diverges. If instead \(\delta_n=c/\sqrt n\) for a fixed \(c>0\), then \(\hat\theta_2=Z/c\), which does not converge to zero. In that sequence the parameter is not consistently estimated in the weak direction.

Sandwich Inference and a Scaling Caution

For a fixed, nonsingular Jacobian and a valid root-\(n\) linear representation, the sandwich covariance is \(A^{-1}BA^{-T}\). If \(A\) is singular, this expression is not defined. Replacing the inverse by a generalized inverse does not automatically restore the usual result: the estimating equation may not identify all parameter directions, and the resulting distribution may have a different rate or form. If \(A\) is nearly singular, finite-sample covariance estimates can be highly sensitive to small errors in the estimated Jacobian.

Worked Example: A Small Derivative That Is Only a Rescaling

Let \(X_i\) have mean \(\mu\) and finite variance \(\sigma^2\), and use the scalar estimating function \(\psi_c(X,\theta)=c(\theta-X)\), where \(c>0\). The population root is \(\theta_0=\mu\); its Jacobian is \(A=c\), and \(B=\operatorname{Var}(\psi_c(X,\mu))=c^2\sigma^2\). The sandwich variance is therefore \[ A^{-1}BA^{-1}=c^{-1}(c^2\sigma^2)c^{-1}=\sigma^2. \]

The sample equation is \(c(\theta-\bar X)=0\), so its solution is \(\hat\theta=\bar X\) for every \(c>0\), however small. If \(c\) varies with \(n\) but remains positive, the estimate and its distribution are unchanged; the score noise shrinks in exactly the same proportion as the derivative. Thus a small raw Jacobian alone does not establish weak identification. Assess its small singular values in relation to the variability and scaling of the estimating function.

Common Mistake

Common mistake: “The Jacobian is singular, so the parameter is not identified.” Singularity only says that the first derivative fails to distinguish at least one direction. The equation \(\Psi(\theta)=\theta^2\) has a unique root at zero despite its zero derivative there. Conversely, a singular Jacobian can arise from a whole set of roots, as for \(\Psi(\theta)=0\). Check the roots of the population equation and, when needed, examine higher-order behavior rather than inferring identification from the derivative alone.

The practical lesson is to inspect the population equation, its local sensitivity, and the noise together. A singular Jacobian rules out the usual inverse-Jacobian proof as written; it does not settle identification. A nearly singular Jacobian can weaken the rate, destabilize conventional standard errors, or, in a sequence of models, prevent consistency. Before using sandwich inference, verify that the relevant Jacobian is invertible and sufficiently stable for the approximation being claimed.

Check Your Understanding

Use the population equation and its local behavior to answer each question.

What does local identification by an estimating equation mean?
Why does a singular Jacobian prevent the usual inverse-Jacobian linear representation?
How can \(\Psi(\theta)=\theta^2\) have a locally identified root even though its derivative at that root is zero?
In the triangular-sequence example, what is the rate of convergence of the weakly identified coordinate when \(\delta_n=n^{-\alpha}\)?
Why does a small Jacobian by itself not establish weak identification?
What is the main concern with applying a generalized inverse to a singular Jacobian and then reporting a conventional sandwich covariance?