Asymptotic Linearization Of Location-Scatter M-Estimators

Why Linearize a Joint Location-Scatter Estimator?

A joint location-scatter M-estimator is defined by several estimating equations that must hold together. Its location component and scatter component are generally dependent: a perturbation in location can affect the scatter equation, and both components are computed from the same observations. As a result, the uncertainty of the joint estimator cannot generally be obtained by analyzing each equation separately.

The central idea is to expand the empirical estimating equation around the population parameter. When the sample derivative is close to the population derivative along the segment joining the population parameter to the estimator, this expansion gives a first-order representation of the estimation error. A multivariate central limit theorem then gives the asymptotic distribution. The derivative describes sensitivity to parameter changes; the covariance of the estimating function describes its sampling variability.

Joint Estimating-Equation Setup

The population parameter \(\eta_0\) is specified by \(G(\eta_0)=0\). For this parameter to be identified by the equations, it must be the unique population zero in the parameter space. The equations need not be likelihood scores: in robust M-estimation they can instead be chosen to limit the effect of large residuals or outlying observations.

For the asymptotic results below, let \(\dot G(\eta)\) denote the \(q\times q\) derivative matrix of \(G\), with rows corresponding to equation components and columns to parameter components. The corresponding sample derivative is \(\dot G_n(\eta)\). Define the sensitivity matrix and score covariance by

When \(B\) is finite, the variance is a covariance matrix. It includes off-diagonal terms between location and scatter scores; those terms should not be discarded unless a calculation or a symmetry argument shows they are zero.

Consistency Conditions

A standard sufficient route to consistency separates identification from empirical approximation. Suppose the parameter space \(\Theta\) is compact, \(G\) is continuous on \(\Theta\), and \(\eta_0\) is its unique zero. Suppose also that

Then any sequence \(\hat{\eta}\in\Theta\) satisfying \(G_n(\hat{\eta})=o_p(1)\) is consistent for \(\eta_0\). In applications, a uniform law of large numbers can establish the displayed convergence when suitable measurability and integrability conditions hold. For instance, a common sufficient framework is a measurable class of functions \(\{\psi(\,\cdot\,;\eta):\eta\in\Theta\}\) with an integrable envelope and conditions ensuring a uniform law of large numbers for that class.

Proof. Fix \(\epsilon>0\). The set \(C_\epsilon=\{\eta\in\Theta:\|\eta-\eta_0\|\geq\epsilon\}\) is compact. If it is nonempty, continuity and uniqueness imply that \(\inf_{\eta\in C_\epsilon}\|G(\eta)\|=c_\epsilon>0\): the minimum is attained, and a zero on \(C_\epsilon\) would contradict uniqueness. On the event \(\hat{\eta}\in C_\epsilon\), the triangle inequality gives

The first term on the right is bounded by \(\sup_{\eta\in\Theta}\|G_n(\eta)-G(\eta)\|=o_p(1)\), and the second is \(o_p(1)\) by assumption. Thus the probability that their sum is at least \(c_\epsilon\) tends to zero, so \(P(\|\hat{\eta}-\eta_0\|\geq\epsilon)\to0\). If \(C_\epsilon\) is empty, the event is impossible. This holds for every \(\epsilon>0\), proving consistency. \(\square\)

Compactness is a convenient sufficient condition, not a requirement for every consistency theorem. The essential issue is separation: parameters a fixed distance from \(\eta_0\) must not have population estimating equations arbitrarily close to zero.

Linearization and Asymptotic Normality

The key regularity condition concerns how the empirical derivative behaves near the target. Assume \(\eta_0\) is an interior point, \(\hat{\eta}\xrightarrow{p}\eta_0\), and \(G_n(\hat{\eta})=o_p(n^{-1/2})\). Suppose \(G_n\) is differentiable on a neighborhood \(N\) of \(\eta_0\). Require that \(G\) be differentiable on \(N\), with \(\dot G\) continuous at \(\eta_0\), and that

Finally, assume \(A=\dot G(\eta_0)\) is nonsingular, \(E[\psi(Z;\eta_0)]=0\), and \(E\|\psi(Z;\eta_0)\|^2<\infty\). These conditions support a precise derivative argument; convergence of derivatives uniformly to a constant matrix on a fixed neighborhood is not required and generally would be false.

Proof. By consistency, with probability tending to one, \(\hat{\eta}\) and the line segment from \(\eta_0\) to \(\hat{\eta}\) lie in \(N\). The fundamental theorem of calculus applied componentwise along that segment gives

The derivative assumption and continuity at \(\eta_0\) imply \(A_n\xrightarrow{p}A\). Indeed, the norm of the difference is bounded by

The first term tends to zero in probability by assumption. The second does too: consistency makes the entire segment approach \(\eta_0\), and \(\dot G\) is continuous there. Since \(A\) is nonsingular, \(A_n\) is nonsingular with probability tending to one and \(A_n^{-1}\xrightarrow{p}A^{-1}\). Rearranging the integral expansion yields

Here the \(o_p(1)\) term accounts for \(\sqrt nG_n(\hat{\eta})=o_p(1)\). Since \(G(\eta_0)=0\), the multivariate central limit theorem gives \(\sqrt nG_n(\eta_0)\xrightarrow{d}N_q(0,B)\). Slutsky’s theorem now gives both the stated linear representation and its limiting normal distribution. \(\square\)

Worked Covariance Calculations

What the Sandwich Covariance Means

The asymptotic covariance has the form \(A^{-1}BA^{-T}\). The “meat” \(B\) measures variability in the estimating function at the target. The two “bread” factors convert fluctuations in the equations into parameter fluctuations. For joint location-scatter estimation, these are matrices for the entire stacked parameter: cross-derivatives in \(A\) and cross-covariances in \(B\) can both matter.

A frequent mistake is to claim that a uniform law of large numbers for the sample derivative makes \(\dot G_n(\eta)\) uniformly close to \(A=\dot G(\eta_0)\) throughout a fixed neighborhood of \(\eta_0\). A uniform law instead typically gives \(\dot G_n(\eta)\) close to the function \(\dot G(\eta)\). That function can vary across the neighborhood. The correct argument is the one used above: uniform convergence to \(\dot G(\eta)\), continuity of \(\dot G\) at \(\eta_0\), and consistency together show that the derivative averaged along the shrinking segment converges to \(A\). For example, the expected derivative of a normal scatter equation with respect to location can vary with \(\mu\), even if it is zero at \(\mu_0\).

This distinction is important when checking regularity conditions. It avoids imposing an unnecessarily strong condition and, more importantly, makes the Taylor argument valid for ordinary location-scatter equations whose population derivatives vary with the parameter.