Identification and Regularity for Estimating Equations

Why Identification and Regularity Matter

An estimating equation gives a way to define a parameter through a population condition. But writing down an equation does not, by itself, ensure that it defines one meaningful target or that small changes in the equation lead to small changes in its root. These are separate issues: identification concerns which parameter the population equation selects, while regularity concerns how the equation behaves near that parameter.

The distinction matters before studying the large-sample behavior of an estimator. A population equation with several roots does not uniquely specify a target. A unique root may still be poorly behaved if the equation is flat there. The conditions developed here make these issues precise without yet establishing sample-to-population convergence.

Population Equations, Roots, and Identification

Let \(W\) be a random observation, let the parameter space be \(\Theta\subseteq\mathbb{R}^p\), and let \(\psi(W,\theta)\in\mathbb{R}^p\) be an estimating function. Assume its coordinates have finite expectations at the parameter values under consideration. Define the population estimating function

A population target \(\theta_0\) satisfies \(\Psi(\theta_0)=0\). For independent observations \(W_1,\ldots,W_n\), the corresponding empirical equation is \(n^{-1}\sum_{i=1}^n\psi(W_i,\theta)=0\). The population equation defines the target; the empirical equation is what an estimator may solve.

A useful stronger condition is separation. It requires not only that other parameter values are not roots, but that parameters a fixed distance from the target cannot make the population equation arbitrarily close to zero. Using the Euclidean norm, for each \(\varepsilon>0\), separation means

whenever the set over which the infimum is taken is nonempty. This condition is especially useful when an empirical equation is only approximately solved: a small population residual cannot occur far from the target. Separation is not automatic from uniqueness on an unbounded parameter space.

Continuity and Separation on Compact Parameter Spaces

Continuity is a practical route to separation when the parameter space is compact. The key point is that the continuous residual norm attains a minimum on a closed set that excludes the target.

Proof. Fix \(\varepsilon>0\) and define \(C_\varepsilon=\{\theta\in\Theta:\|\theta-\theta_0\|\geq\varepsilon\}\). This set is closed in the compact set \(\Theta\), so it is compact. If \(C_\varepsilon\) is empty, there is nothing to check. Otherwise, the continuous function \(\theta\mapsto\|\Psi(\theta)\|\) attains its minimum on \(C_\varepsilon\). That minimum cannot be zero: a zero would be a root distinct from \(\theta_0\), contrary to uniqueness. Therefore the minimum is strictly positive, which is the separation condition. \(\square\)

Compactness is a sufficient condition, not the definition of separation. If \(\Theta\) is not compact, a sequence of parameters can move farther away while its residual tends to zero, even though no member of the sequence is a root. In that setting, separation must be checked directly or obtained from other assumptions.

Continuity and Differentiation of the Population Function

A population function is an expectation, so its regularity depends on the random estimating function. One standard sufficient condition for continuity near \(\theta_0\) is that \(\psi(W,\theta)\) is almost surely continuous in \(\theta\) and is bounded in norm by an integrable random variable throughout a neighborhood. Specifically, suppose some neighborhood \(U\) of \(\theta_0\) and some \(G(W)\) satisfy \(\|\psi(W,\theta)\|\leq G(W)\) for all \(\theta\in U\), with \(E[G(W)]<\infty\). If \(\theta_k\to\theta\in U\), continuity gives \(\psi(W,\theta_k)\to\psi(W,\theta)\) almost surely, and dominated convergence gives \(\Psi(\theta_k)\to\Psi(\theta)\).

Differentiability also requires care: differentiating the formula for \(\psi\) does not automatically justify differentiating its expectation. A sufficient local condition is that \(\psi(W,\theta)\) is almost surely continuously differentiable on a neighborhood \(U\), and that its derivative matrix \(D_\theta\psi(W,\theta)\) has an integrable envelope there. Then

To see why the interchange is justified, for a coordinate direction \(e_j\) and scalar \(h\), the fundamental theorem of calculus writes the difference quotient as

As \(h\to0\), the right side converges almost surely to \(D_\theta\psi(W,\theta)e_j\). Its norm is bounded by the integrable derivative envelope. Dominated convergence therefore allows the limit to pass through the expectation, giving the corresponding column of \(J(\theta)\). This argument applies coordinate by coordinate.

Nonsingular Jacobians and Local Identification

Assume now that \(\Theta\) contains an open neighborhood of \(\theta_0\), that \(\Psi(\theta_0)=0\), and that \(\Psi\) is continuously differentiable there. Its Jacobian \(J(\theta_0)\) is nonsingular if it has an inverse. This condition is local: it controls the behavior near \(\theta_0\), not the existence of other roots elsewhere in \(\Theta\).

Proof. Let \(s>0\) be the smallest singular value of \(J(\theta_0)\). Then \(\|J(\theta_0)v\|\geq s\|v\|\) for every vector \(v\). By continuity of \(J\), choose a ball around \(\theta_0\), contained in the differentiability domain, on which \(\|J(\theta)-J(\theta_0)\|_{\mathrm{op}}<s/2\). The ball is convex. For any \(\theta_1,\theta_2\) in it, set \(v=\theta_1-\theta_2\). The fundamental theorem of calculus along the line segment gives

The norm of the integral is at most \((s/2)\|v\|\). The triangle inequality therefore yields

If \(\theta_1\ne\theta_2\), the right side is positive, so their images differ. Thus \(\Psi\) is one-to-one on the ball. Since \(\Psi(\theta_0)=0\), no other point in that ball is a root. \(\square\)

The theorem gives more than uniqueness in a neighborhood: it gives a lower bound on how much the population equation changes when its argument changes locally. This is the local stability associated with a nonsingular Jacobian. It does not imply global identification, and it does not establish that a sample equation has a root near \(\theta_0\).

Common Mistake: Treating the Conditions as Interchangeable

• Wrong reasoning: “The Jacobian is nonsingular at the target, so the target is identified.” Nonsingularity ensures local uniqueness only. The equation \(\theta^2-1=0\) has nonsingular derivatives at both roots, yet it has two roots on \(\mathbb{R}\).
• Correction: Check global uniqueness, or use a separation condition, to rule out other roots across the parameter space. Check the Jacobian separately for local regularity.
• Wrong reasoning: “The target is the unique root, so its behavior is regular.” The equation \(\theta^3=0\) has a unique root but a zero derivative there.
• Correction: Uniqueness does not rule out a flat root. Inspect the Jacobian and the conditions that justify its existence and continuity.

These checks matter because later arguments about estimation use different ingredients for different purposes. Identification says what parameter the population equation selects. Separation rules out parameters far away with nearly zero population residual. Continuity supports such separation under compactness, and a nonsingular Jacobian gives local control of the equation. None of these conditions alone proves that an empirical solution exists or converges to the target.