Asymptotic Normality of Quasi-Estimators

From an Estimating Equation to a Distribution

Consistency tells us that a quasi-estimator approaches the target parameter, but it does not describe its fluctuations at the usual $n^{-1/2}$ scale. Asymptotic normality supplies that description. Its basic mechanism is a local linear approximation: random variation in the estimating equation at the true parameter is converted into variation in its root by the equation’s derivative.

The argument has two essential ingredients. A central limit theorem controls the estimating equation evaluated at the target; a stable derivative controls how much the equation changes as the parameter moves. Neither ingredient alone is enough. In particular, a derivative limit cannot replace the central limit condition.

Definition: Let $\theta_0\in\mathbb{R}^p$ be the target parameter and let $U_n(\theta)\in\mathbb{R}^p$ be an estimating function based on $n$ observations. A quasi-estimator $\hat{\theta}$ is a solution, or an approximate solution, of $U_n(\theta)=0$. It is asymptotically normal at rate $\sqrt n$ if $\sqrt n(\hat{\theta}-\theta_0)$ converges in distribution to a multivariate normal random vector.

The equation need not be a likelihood score, and the observations need not have a fully specified probability distribution. For quasi-score equations, the central limit theorem is applied to the sum of their contributions. The earlier material on consistency addresses why a root can approach $\theta_0$; here we add the conditions that identify the root’s limiting distribution.

A Linear Expansion and Its Conditions

Write $U_n'(\theta)$ for the $p\times p$ Jacobian matrix of first derivatives. The following conditions give a useful general result. They are stated directly in terms of the estimating equation, so they apply to quasi-scores as well as other smooth estimating equations.

Theorem: Suppose $p$ is fixed, $\hat{\theta}\xrightarrow{p}\theta_0$, and, with probability tending to one, the line segment joining $\theta_0$ and $\hat{\theta}$ lies in a region where $U_n$ is continuously differentiable. Assume $$ \frac{U_n(\theta_0)}{\sqrt n}\xrightarrow{d}N(0,B), \qquad \sup_{0\leq t\leq 1} \left\| \frac{1}{n}U_n'\bigl(\theta_0+t(\hat{\theta}-\theta_0)\bigr)+A \right\|\xrightarrow{p}0, $$ where $A$ and $B$ are finite $p\times p$ matrices and $A$ is nonsingular. Also suppose $U_n(\hat{\theta})=o_p(\sqrt n)$; an exact root satisfies this condition. Then $$ \sqrt n(\hat{\theta}-\theta_0) =A^{-1}\frac{U_n(\theta_0)}{\sqrt n}+o_p(1) \quad\text{and}\quad \sqrt n(\hat{\theta}-\theta_0)\xrightarrow{d}N\bigl(0,A^{-1}B A^{-T}\bigr). $$ Here $A^{-T}=(A^{-1})^T$.

The derivative condition says that the Jacobian, divided by $n$, is close to $-A$ all along the short segment between the target and the estimator. In common applications, it follows from a uniform law of large numbers for the derivative and consistency. Stating the condition along the segment makes explicit exactly what the expansion needs.

Expand the equation.
Apply the fundamental theorem of calculus along the segment from $\theta_0$ to $\hat{\theta}$.

Scale by $\sqrt n$.
The centered estimating equation is of order $\sqrt n$ by its central limit condition; the derivative is of order $n$.

Take the limit.
The derivative matrix approaches $-A$, and the central limit theorem and Slutsky’s theorem give the normal limit.

Proof. Define the integrated Jacobian $$ \bar H_n=\int_0^1\frac{1}{n}U_n'\bigl(\theta_0+t(\hat{\theta}-\theta_0)\bigr)\,dt. $$ By the assumed uniform convergence, $\bar H_n\xrightarrow{p}-A$. The fundamental theorem of calculus gives $$ \frac{U_n(\hat{\theta})}{\sqrt n} = \frac{U_n(\theta_0)}{\sqrt n} +\bar H_n\sqrt n(\hat{\theta}-\theta_0). $$ Since $A$ is nonsingular and $\bar H_n\xrightarrow{p}-A$, $\bar H_n$ is nonsingular with probability tending to one and $\bar H_n^{-1}\xrightarrow{p}-A^{-1}$. Rearranging on that event, $$ \sqrt n(\hat{\theta}-\theta_0) = -\bar H_n^{-1} \left\{ \frac{U_n(\theta_0)}{\sqrt n} -\frac{U_n(\hat{\theta})}{\sqrt n} \right\}. $$ The approximate-root condition implies $U_n(\hat{\theta})/\sqrt n=o_p(1)$. The central limit condition implies $U_n(\theta_0)/\sqrt n=O_p(1)$. Consequently, multiplying the difference between $-\bar H_n^{-1}$ and $A^{-1}$, which is $o_p(1)$, by $U_n(\theta_0)/\sqrt n=O_p(1)$ contributes only $o_p(1)$. Thus $$ \sqrt n(\hat{\theta}-\theta_0) =A^{-1}\frac{U_n(\theta_0)}{\sqrt n}+o_p(1). $$ The central limit theorem and Slutsky’s theorem now give the stated normal limit. If $Z\sim N(0,B)$, then the covariance of $A^{-1}Z$ is $A^{-1}B(A^{-1})^T=A^{-1}BA^{-T}$. $\square$

The explicit $O_p(1)$ step matters: it is the central limit condition that makes the random term being multiplied by the $o_p(1)$ matrix difference bounded in probability. This is why the theorem includes that condition rather than relying only on consistency and a derivative limit.

Worked Scalar Examples

Worked Example: A Normal Mean With a Working Variance

Let $Y_1,\ldots,Y_n$ be independent with $Y_i\sim N(\theta_0,9)$. Use the estimating equation $U_n(\theta)=\sum_{i=1}^n(Y_i-\theta)/4=0$, which uses working variance 4. Find the asymptotic distribution of its root.

Solving the equation gives $\hat{\theta}=\bar Y$, since $U_n(\theta)=0$ is equivalent to $\sum_iY_i-n\theta=0$. At the target, $U_n(\theta_0)=\sum_i(Y_i-\theta_0)/4$, so $$ \operatorname{Var}\left(\frac{U_n(\theta_0)}{\sqrt n}\right) =\frac{1}{n}\frac{n\cdot9}{16} =\frac{9}{16}. $$ The central limit theorem gives $U_n(\theta_0)/\sqrt n\xrightarrow{d}N(0,9/16)$; in this normal example the scaled sum is already normal for each $n$.

The derivative is $U_n'(\theta)=-n/4$, so $A=1/4$, and $B=9/16$. Therefore $$ \sqrt n(\hat{\theta}-\theta_0)\xrightarrow{d} N\left(0,\frac{9/16}{(1/4)^2}\right) =N(0,9). $$ The working factor $1/4$ changes the scale of the estimating equation, but it cancels from the root and from its resulting variance.

Worked Example: An Overdispersed Count Mean

Suppose independent counts have a common mean $\mu_0=4$ and variance $8$. For a concrete distribution, take a negative binomial distribution with mean $4$ and size $r=4$, whose variance is $\mu_0+\mu_0^2/r=4+16/4=8$. Use the log-mean parameter $\theta=\log\mu$ and the quasi-score $U_n(\theta)=\sum_i(Y_i-e^\theta)$. Find the limiting distribution of its root.

The root satisfies $e^{\hat{\theta}}=\bar Y$, so $\hat{\theta}=\log\bar Y$ whenever $\bar Y>0$; the probability of $\bar Y=0$ tends to zero because $\bar Y\xrightarrow{p}4$. At $\theta_0=\log4$, the summands $Y_i-4$ have mean zero and variance $8$. The central limit theorem gives $$ \frac{U_n(\theta_0)}{\sqrt n}\xrightarrow{d}N(0,8). $$ The derivative is $U_n'(\theta)=-ne^\theta$. Consistency of $\hat\theta$ and continuity of $e^\theta$ imply that the derivative divided by $n$, along the segment to $\theta_0$, converges in probability to $-4$. Thus $A=4$, $B=8$, and $$ \sqrt n(\hat{\theta}-\log4)\xrightarrow{d} N\left(0,\frac{8}{4^2}\right)=N(0,1/2). $$ This calculation uses the count variance in the central limit theorem; it does not require the negative binomial likelihood to define the quasi-estimator.

Worked Vector Example: A Two-Group Regression

Worked Example: Heteroskedastic Errors in a Balanced Design

For even $n$, observe $n/2$ independent responses at $x_i=0$ and $n/2$ at $x_i=1$. Suppose $Y_i=\theta_0+\theta_1x_i+\varepsilon_i$, with independent normal errors of mean zero and variance $1$ in group $x=0$, and variance $4$ in group $x=1$. Use the estimating equation $U_n(\theta)=\sum_i z_i(Y_i-z_i^T\theta)=0$, where $z_i=(1,x_i)^T$. Calculate the limiting covariance.

The normalized negative derivative is the design average $$ A=\frac{1}{n}\sum_i z_iz_i^T =\frac12 \begin{pmatrix}1&0\\0&0\end{pmatrix} +\frac12 \begin{pmatrix}1&1\\1&1\end{pmatrix} = \begin{pmatrix}1&1/2\\1/2&1/2\end{pmatrix}. $$ The score at the target is $\sum_i z_i\varepsilon_i$. Its normalized covariance is $$ B=\frac1n\sum_i\operatorname{Var}(\varepsilon_i)z_iz_i^T = \frac12 \begin{pmatrix}1&0\\0&0\end{pmatrix} +2\begin{pmatrix}1&1\\1&1\end{pmatrix} = \begin{pmatrix}5/2&2\\2&2\end{pmatrix}. $$ The independent normal errors have finite variances, and the two group sizes grow proportionally, so the multivariate central limit theorem applies to the normalized score. Also, $$ A^{-1}=\begin{pmatrix}2&-2\\-2&4\end{pmatrix}, \qquad A^{-1}BA^{-T} =\begin{pmatrix}2&-2\\-2&10\end{pmatrix}. $$ For example, the first product is $A^{-1}B=\begin{pmatrix}1&0\\3&4\end{pmatrix}$; multiplying by $A^{-T}=A^{-1}$ yields the displayed covariance. Hence $\sqrt n(\hat\theta-\theta_0)$ converges to a bivariate normal distribution with this covariance. The nonzero off-diagonal entry reflects the joint fluctuation of the fitted intercept and slope.

What the Result Does—and Does Not—Require

The conclusion is local: it describes the estimator’s fluctuations around a consistent target. It does not itself prove consistency, guarantee that a root exists, or ensure that a finite-sample distribution is normal. Those are separate questions. In particular, the result relies on a nonsingular limiting derivative $A$; if the estimating equation is locally flat in some direction, this $n^{-1/2}$ argument cannot identify the usual normal limit in that direction.

The matrix $A$ describes the equation’s local sensitivity, while $B$ describes the limiting variability of its value at the target. Their combination $A^{-1}BA^{-T}$ is the covariance for the limiting distribution of the estimator. It follows from propagating the central-limit fluctuation through the linear approximation, not from assuming a likelihood or invoking a likelihood-based information identity.

Common Mistake

Using only the derivative limit. It is not enough to show that $U_n'(\theta)/n$ approaches a nonsingular matrix and then replace the random inverse derivative by its limit in the expansion. That replacement multiplies the estimating equation at $\theta_0$. Without a condition such as $U_n(\theta_0)/\sqrt n=O_p(1)$, the product of an $o_p(1)$ matrix error and that random term need not be $o_p(1)$. The correction is to assume a central limit theorem for $U_n(\theta_0)/\sqrt n$, as in the theorem, or at minimum to assume that this term is bounded in probability.
Dropping the transpose. In a vector problem the covariance is $A^{-1}BA^{-T}$, not generally $A^{-1}BA^{-1}$. The two agree only under additional symmetry conditions.
Confusing the score’s covariance with the estimator’s covariance. $B$ is the limiting covariance of the normalized estimating equation. The estimator’s limiting covariance also accounts for sensitivity through $A^{-1}$ on both sides.

Key takeaway: Asymptotic normality follows when the estimating equation has a central limit fluctuation at the target, its normalized derivative stabilizes at a nonsingular limit along the estimator’s path, and the estimator is a consistent root or approximate root. The resulting covariance is $A^{-1}BA^{-T}$.

Check Your Understanding

Use the assumptions and expansion in the tutorial to answer the following.

In the theorem, what role does the central limit condition play when deriving the linear representation?
If $A=2$ and $B=12$ in a scalar estimating equation, what is the limiting variance of $\sqrt n(\hat{\theta}-\theta_0)$?
Why must the derivative condition control the Jacobian along the segment between $\theta_0$ and $\hat{\theta}$?
For the two-group example, identify which matrix summarizes the estimating equation’s variability and which summarizes its sensitivity.
Does the asymptotic normality result require a fully specified likelihood? Explain briefly.