Asymptotic Normality of Smooth Extremum Estimators

From a Sample Optimum to a Limiting Distribution

Consistency says that an estimator approaches the target parameter as the sample grows, but it does not describe its typical sampling variation at a large, finite sample size. For a smooth extremum estimator, that variation is governed by two features: how much the sample criterion’s gradient fluctuates at the target, and how sharply the population criterion curves there. The first is summarized by a central limit theorem for the sample score; the second by the limiting Hessian.

The key step is local. At an interior optimum, the sample criterion has a zero gradient. Expanding that gradient around the population parameter relates the estimation error to the gradient evaluated at the target. If the Hessian converges to a nonsingular matrix and the score satisfies a central limit theorem, this relation yields a normal limit for the scaled estimation error.

Setup and Conditions

Let \(\theta\) be a parameter in an open subset of \(\mathbb{R}^k\). Suppose the sample criterion to be maximized is \(Q_n(\theta)=n^{-1}\sum_{i=1}^n q(W_i,\theta)\), where \(W_1,\ldots,W_n\) are identically distributed and independent observations. Let \(Q(\theta)=E[q(W_i,\theta)]\) be the population criterion, and let \(\theta_0\) be its uniquely identified maximizer. Write \(\hat{\theta}_n\) for a sample maximizer.

For the usual smooth-extremum argument, the following conditions give a useful sufficient set of assumptions. The estimator is consistent, so \(\hat{\theta}_n\to_p\theta_0\); \(\theta_0\) is interior; the sample criterion is twice continuously differentiable in a neighborhood of \(\theta_0\); and the first-order condition holds there. The sample Hessian converges to the population Hessian locally, including along the line segment between \(\theta_0\) and \(\hat{\theta}_n\). The population Hessian at \(\theta_0\) is nonsingular. Finally, the score at \(\theta_0\) has a finite covariance matrix and satisfies the multivariate central limit theorem.

The central limit theorem requires the score to be centered: \(E[s_i(\theta_0)]=0\). For an interior, differentiable population optimum this is the population first-order condition, provided differentiation and expectation can be interchanged. The covariance \(\Omega\) measures score variation, while the inverse Hessians translate that variation into movement in the parameter.

Why the Hessian and Score Produce This Limit

The derivation uses the sample first-order condition and a Taylor expansion with an intermediate Hessian. Define \(H_n^*=\int_0^1\nabla_\theta^2Q_n\big(\theta_0+t(\hat{\theta}_n-\theta_0)\big)\,dt\). Because \(\hat{\theta}_n\) maximizes a differentiable criterion at an interior point, \(\nabla_\theta Q_n(\hat{\theta}_n)=0\). The integral form of the expansion gives

If \(H_n^*\to_p H\) and \(H\) is nonsingular, then \(H_n^*\) is nonsingular with probability approaching one. Multiplying the equation by \(\sqrt{n}\) and solving gives

Since \(\nabla_\theta Q_n(\theta_0)=n^{-1}\sum_{i=1}^n s_i(\theta_0)\), the multivariate central limit theorem implies \(\sqrt{n}\nabla_\theta Q_n(\theta_0)\xrightarrow{d}N(0,\Omega)\). Slutsky’s theorem then gives a normal limit with covariance \((-H^{-1})\Omega(-H^{-T})=H^{-1}\Omega H^{-T}\). The two minus signs cancel. This also explains why a maximization problem’s negative Hessian does not make the resulting covariance negative.

Worked Scalar and Vector Calculations

Interpretation and Common Pitfalls

Asymptotic normality describes the distribution of the estimation error after scaling by \(\sqrt{n}\). In the stated result, the estimator’s large-sample covariance is approximately \(H^{-1}\Omega H^{-T}/n\), not \(H^{-1}\Omega H^{-T}\) without the division by \(n\). Units also follow from the problem: the covariance matrix has units of parameter times parameter, while the scaled error has the corresponding units multiplied by \(\sqrt{n}\).

• Using curvature but omitting score variation. The Hessian alone cannot specify sampling uncertainty. The score covariance \(\Omega\) is also required.
• Dropping one of the minus signs incorrectly. The expansion has a minus sign, but covariance multiplication introduces it on both sides: \((-H^{-1})\Omega(-H^{-T})=H^{-1}\Omega H^{-T}\).
• Reversing matrix order. In general, matrix multiplication is not commutative. Use \(H^{-1}\Omega H^{-T}\), not an arbitrarily reordered product.
• Assuming a first-order condition at a boundary. The zero-gradient condition used here requires an interior optimum. At a parameter boundary, that argument may fail.
• Calling an approximation exact in finite samples. The normal distribution is a limiting distribution; it need not describe the exact finite-sample distribution of the estimator.

The result is useful because it separates two sources of uncertainty in a precise way: random score variation and local criterion curvature. It also gives a disciplined route from an optimization problem to a large-sample distribution: establish the first-order condition, expand it, and verify the Hessian and score limits rather than inferring normality from smoothness alone.