Why Plausible Asymptotic Arguments Can Fail
An M-estimation calculation can look routine: show that an estimating equation is close to its population counterpart, expand it around the target, and estimate a sandwich covariance. Each step, however, needs its own conditions. Pointwise convergence may not control roots that move with sample size; a population equation may have several roots; a Taylor remainder may be small in absolute terms but not small relative to \(n^{-1/2}\); and an empirical covariance calculation may estimate the variability of an average rather than the variability of one score.
These are not merely technical distinctions. They can change whether an estimator is consistent, whether its limiting distribution is normal, and whether a reported standard error has the right scale. The diagnostics here concern estimating equations of the form \(\Psi_n(\theta)=0\), where \(\Psi_n\) is an average of individual contributions.
Failure 1: Pointwise Convergence Does Not Control Moving Roots
Pointwise convergence checks each fixed parameter value separately. Consistency, in contrast, concerns a root that can change with \(n\). The following deterministic example shows why pointwise convergence alone is not a root-control argument.
Worked Example: A Root Escapes Despite Pointwise Convergence
Let \(\Theta=\mathbb R\), and define \(\phi(t)=\max(1-|t|,0)\). Set \(\Psi(\theta)=\theta\) and \(\Psi_n(\theta)=\theta-n\phi(\theta-n)\). For any fixed \(\theta\), once \(n>|\theta|+1\), we have \(|\theta-n|>1\), so \(\phi(\theta-n)=0\) and \(\Psi_n(\theta)=\theta\). Thus \(\Psi_n(\theta)\to\Psi(\theta)\) at every fixed \(\theta\). The population equation has the unique root \(\theta_0=0\).
But at \(\theta=n\), \(\phi(0)=1\), and hence \(\Psi_n(n)=n-n=0\). Selecting this root gives \(\hat\theta_n=n\), which does not converge to zero. The discrepancy is not uniformly small: at \(\theta=n\), \(\left|\Psi_n(n)-\Psi(n)\right|=n\), so \(\sup_{\theta\in\mathbb R}|\Psi_n(\theta)-\Psi(\theta)|\ge n\).
The example lacks compactness and uniform control; it does not say that pointwise convergence always fails. It shows that a proof must rule out roots in regions that move with sample size, for example by establishing uniform convergence on a suitable parameter set and controlling roots outside it.
Proof. Fix \(\varepsilon>0\). The set \(C_\varepsilon=\{\theta\in\Theta:\|\theta-\theta_0\|\ge\varepsilon\}\) is compact. If it is empty, the claimed probability bound is immediate. Otherwise, continuity and the unique-root condition imply that \(\|\Psi(\theta)\|\) has a positive minimum on \(C_\varepsilon\); write this minimum as \(\delta_\varepsilon>0\). On the event that \(\sup_{\theta\in\Theta}\|\Psi_n(\theta)-\Psi(\theta)\|<\delta_\varepsilon/3\) and \(\|\Psi_n(\hat\theta_n)\|<\delta_\varepsilon/3\), a value \(\hat\theta_n\in C_\varepsilon\) would imply
contradicting \(\|\Psi(\hat\theta_n)\|\ge\delta_\varepsilon\). The probability of the stated event tends to one by the two assumed convergences. Therefore \(P(\|\hat\theta_n-\theta_0\|\ge\varepsilon)\to0\), proving consistency. \(\square\)
Failure 2: A Root Is Not Necessarily the Target Root
Uniform convergence cannot select a target if the population equation itself has several roots. A consistency argument needs an identification condition appropriate to the estimator’s root-selection rule. Merely saying “\(\hat\theta_n\) solves the equation” does not identify which solution it approaches.
Worked Example: Two Exact Roots
For every \(n\), let \(\Psi_n(\theta)=\Psi(\theta)=\theta(\theta-1)\), with \(\Theta=[0,1]\). The population roots are 0 and 1, and uniform convergence holds exactly because \(\Psi_n=\Psi\). The sequence \(\hat\theta_n=1\) is an exact root for every \(n\), but it does not converge to the selected target \(\theta_0=0\).
Here the failure is not approximation error: the empirical and population equations coincide. It is failure to distinguish the intended root from another root. A valid argument must establish uniqueness and separation, or state a selection rule and prove that it selects the desired root with probability tending to one.
Failure 3: A Taylor Remainder Can Survive at Root-n Scale
For asymptotic normality, it is not enough that a Taylor remainder tends to zero. If the score at the target is of order \(n^{-1/2}\), the remainder must be negligible on that same scale to recover the usual linear representation. The next example has a nonsingular derivative at the target and a root-\(n\) consistent estimator, but its local remainder changes the limiting law.
Worked Example: A Small Remainder With a First-Order Effect
Let \(X_1,\ldots,X_n\) be independent \(N(0,1)\) observations and write \(\bar X=n^{-1}\sum_{i=1}^nX_i\). Set \(c=1/4\), \(q(t)=ct^2/(1+t^2)\), and define \(\Psi_n(\theta)=\bar X-\theta+n^{-1/2}q(\sqrt n\,\theta)\). The target is \(\theta_0=0\), and \(\Psi_n(0)=\bar X\). Since \(q'(0)=0\), the derivative at the target is \(\Psi_n'(0)=-1\), which is nonsingular.
Let \(Z_n=\sqrt n\,\bar X\), which has the \(N(0,1)\) distribution for every \(n\), and put \(t_n=\sqrt n\,\hat\theta_n\). The estimating equation becomes \(Z_n-t_n+q(t_n)=0\), or \(F(t_n)=Z_n\) for \(F(t)=t-q(t)\). Now \(q'(t)=2ct/(1+t^2)^2\), and \(\sup_t|q'(t)|<1\) for \(c=1/4\). Thus \(F'(t)=1-q'(t)>0\); also \(q\) is bounded, so \(F(t)\to\pm\infty\) as \(t\to\pm\infty\). Consequently \(F\) is a strictly increasing bijection and the equation has the unique solution \(t_n=F^{-1}(Z_n)\).
Because \(Z_n\) has a fixed distribution and \(F^{-1}\) is continuous, \(t_n=O_p(1)\), so \(\hat\theta_n=O_p(n^{-1/2})\). However, the added term in the equation is \(n^{-1/2}q(\sqrt n\,\hat\theta_n)=n^{-1/2}q(t_n)\), which is not \(o_p(n^{-1/2})\): \(t_n\) has the nondegenerate distribution \(F^{-1}(Z)\), and \(q(F^{-1}(Z))\) is nonzero with probability one. Therefore the equation does not reduce at first order to \(t_n=Z_n\). Instead \(t_n\) has the distribution of \(F^{-1}(Z)\), where \(Z\sim N(0,1)\). The nonlinear, strictly increasing map \(F^{-1}\) is not affine, so this transformed normal distribution is not normal.
This example pinpoints the missing check: a derivative evaluated at the target does not by itself control the equation throughout a shrinking neighborhood of the estimator. A normality proof needs a valid local expansion whose remainder is \(o_p(n^{-1/2})\), not merely a remainder that vanishes without regard to rate.
Derivation. Substitute the root condition into the assumed expansion: \(0=\Psi_n(\theta_0)+A(\hat\theta_n-\theta_0)+o_p(n^{-1/2})\). Multiplying by \(\sqrt n\) gives \(A\sqrt n(\hat\theta_n-\theta_0)=-\sqrt n\,\Psi_n(\theta_0)+o_p(1)\). Multiplication by the fixed matrix \(A^{-1}\) proves the representation. The stated distributional limit follows from convergence in distribution and the fact that adding an \(o_p(1)\) term does not change that limit. \(\square\)
Failure 4: An Averaged Score Is Not a Score Covariance
In the sandwich covariance, \(B\) is the covariance of an individual score contribution at the target, not the covariance of the sample-average score. In common independent-observation settings, the latter is on the order of \(1/n\). Confusing the two can make an estimated sandwich variance collapse to zero.
Worked Example: Checking the Sandwich for a Normal Mean
Suppose \(X_i\) are independent \(N(\mu,1)\) observations, and define \(\psi(X_i,\theta)=X_i-\theta\). Then \(\Psi_n(\theta)=n^{-1}\sum_i(X_i-\theta)=\bar X-\theta\), whose root is \(\hat\theta=\bar X\). The derivative of each contribution with respect to \(\theta\) is \(-1\). Thus the population Jacobian is \(A=-1\), not \(1\). At the target \(\theta_0=\mu\), the score contribution has variance \(B=\operatorname{Var}(X_i-\mu)=1\). The scalar sandwich variance is therefore
For the estimator, the asymptotic variance of \(\sqrt n(\hat\theta-\mu)\) is 1, and the variance of \(\hat\theta\) is \(1/n\). The usual empirical score-covariance estimate at \(\hat\theta\) is \(\hat B=n^{-1}\sum_i(X_i-\bar X)^2\), which converges in probability to 1. By contrast, the outer product of the averaged fitted score is \(\{n^{-1}\sum_i(X_i-\bar X)\}^2=0\), because the fitted scores sum to zero. It cannot estimate \(B\). The covariance estimate must use individual score contributions (or a valid equivalent), not the aggregate estimating equation.
For a general vector estimating equation, the relevant population quantities are the Jacobian \(A=E[\partial\psi(X_i,\theta_0)/\partial\theta^T]\) and \(B=\operatorname{Var}(\psi(X_i,\theta_0))\), when these expectations exist and the differentiability and limit conditions justify their use. A plug-in estimate of \(B\) typically averages individual outer products after evaluating scores at \(\hat\theta\). Its consistency requires suitable laws of large numbers and control of the effect of substituting \(\hat\theta\); it does not follow solely from the estimator being consistent.
A Practical Diagnostic Sequence
Before accepting a consistency or normality argument, check the steps in the order they are used. These checks isolate distinct failure modes rather than treating “regularity conditions” as a single undifferentiated assumption.
Write down \(\Psi\), verify \(\Psi(\theta_0)=0\), and check that the target root is unique or that the estimator’s selection rule is justified.
Establish uniform convergence on a relevant set and a positive separation of \(\Psi\) from zero away from \(\theta_0\), or supply another valid root-control argument.
For root-\(n\) normality, verify that the remainder is \(o_p(n^{-1/2})\) and that the Jacobian used in the expansion converges to a nonsingular limit.
Confirm that the empirical quantity estimates the covariance of one score contribution and that the estimated Jacobian and score covariance both converge appropriately.
Common Pitfall
Wrong reasoning: “The empirical estimating function converges pointwise to an equation with a unique root, and its derivative at the target is nonsingular; therefore any empirical root is consistent and asymptotically normal.” The conclusion does not follow. Pointwise convergence need not control moving roots, a unique root is needed for the relevant population equation, and a derivative at one point does not bound the Taylor remainder at the estimator’s scale. Even after those issues are settled, an inconsistent estimate of \(B\) invalidates the reported sandwich covariance.
Correction: State and check separate conditions for root consistency, local linearization, the score limit, and covariance estimation. In particular, verify a root-separation or equivalent identification argument, a remainder negligible at the scale of interest, and a consistent estimate of the individual-score covariance. Also calculate derivative signs from the estimating function actually defined: for \(\psi(X_i,\theta)=X_i-\theta\), the derivative is \(-1\).
Check Your Understanding
For each question, identify which condition or calculation needs to be checked.
- Why does the escaping-root example not contradict the root-consistency theorem?
- What additional issue arises when the population estimating equation has two roots, even if empirical and population equations are identical?
- In the nonlinear-remainder example, why is a remainder that is \(O_p(n^{-1/2})\) insufficient for the usual linear representation?
- For the normal-mean estimating function \(\psi(X_i,\theta)=X_i-\theta\), what are \(A\) and \(B\), and why?
- Why does the squared averaged fitted score fail to estimate the individual-score covariance in the normal-mean example?