Asymptotic Distribution of Moment Estimators

Why Moment Estimators Have an Approximate Distribution

A method-of-moments estimate is computed from sample moments, so its uncertainty begins with the sampling variability of those moments. For a large independent sample, the central limit theorem gives an approximate normal distribution for a sample moment. A first-order expansion then translates that variability into the distribution of the parameter estimate.

The central step is to account for how quickly the population moment changes with the parameter. If a small change in the parameter produces only a small change in the moment, the estimate can be sensitive to sampling noise. If the moment changes steeply, the same noise produces less parameter uncertainty.

Definition: Consider independent, identically distributed observations $X_1,\ldots,X_n$ from a model indexed by one unknown parameter $\theta$. Let $h(X)$ be a moment with finite variance, and define $g(\theta)=E_\theta[h(X)]$. The method-of-moments estimator $\hat{\theta}$ solves $\bar h=g(\hat{\theta})$, where $\bar h=n^{-1}\sum_{i=1}^n h(X_i)$.

The formula developed below requires a locally identifiable moment equation: $g$ must be differentiable near the true parameter, and $g'(\theta)$ must not be zero there. The estimator must also be a consistent solution in that neighborhood, and $\operatorname{Var}_\theta(h(X))$ must be finite. These conditions ensure that small sampling fluctuations in the moment correspond to small, approximately linear fluctuations in the estimate.

Deriving the Asymptotic Distribution

Write $\tau^2(\theta)=\operatorname{Var}_\theta(h(X))$. Under the conditions above, the central limit theorem gives the approximate distribution of the sample moment around its population value. Linearizing $g(\hat{\theta})$ around the true $\theta$ then gives the distribution of $\hat{\theta}$.

Apply the central limit theorem.
Because the observations are independent and identically distributed and $h(X)$ has finite variance, $\sqrt{n}(\bar h-g(\theta))$ converges in distribution to a normal random variable with mean zero and variance $\tau^2(\theta)$.

Expand the moment equation.
The estimator satisfies $\bar h=g(\hat{\theta})$. Differentiability and consistency give $g(\hat{\theta})=g(\theta)+g'(\theta)(\hat{\theta}-\theta)+o_p(|\hat{\theta}-\theta|)$.

Relate the moment error to the parameter error.
Rearranging the first-order expansion gives $\sqrt{n}(\hat{\theta}-\theta)$ approximately equal to $\sqrt{n}(\bar h-g(\theta))/g'(\theta)$.

The central limit theorem in Step 1 follows by applying the usual iid result to the variables $h(X_i)$, whose mean is $g(\theta)$ and variance is $\tau^2(\theta)$. For Step 3, the nonzero derivative ensures that the first-order term can be inverted; the remainder is negligible at the scale $1/\sqrt n$ for a regular, consistent solution. Dividing the limiting normal quantity by $g'(\theta)$ changes its variance by the square of that divisor.

\sqrt{n}(\hat{\theta}-\theta) \ \xrightarrow{d}\ N\left(0,\frac{\tau^2(\theta)}{[g'(\theta)]^2}\right)

Theorem: Under the stated scalar moment-estimation conditions, the large-sample distribution of $\hat{\theta}$ is approximately normal with mean $\theta$ and variance $\tau^2(\theta)/(n[g'(\theta)]^2)$. Thus, $$ \hat{\theta}\ \dot{\sim}\ N\left(\theta,\frac{\operatorname{Var}_\theta(h(X))}{n[g'(\theta)]^2}\right). $$ The symbol $\dot{\sim}$ denotes a large-sample approximation, not an exact finite-sample distribution.

The variance is nonnegative even if $g'(\theta)$ is negative, because the derivative is squared. In practice, $\theta$ is unknown, so estimate the asymptotic standard error by substituting $\hat{\theta}$ into the variance expression, provided the relevant functions are defined at that estimate.

Worked Examples

Worked Example: Estimating the Mean of Exponential Losses

Suppose $X_1,\ldots,X_{100}$ are independent hypothetical loss amounts with an exponential distribution having mean $\theta>0$. The observed sample mean is $\bar X=8.4$ dollars. Find the method-of-moments estimate, its estimated asymptotic standard error, and an approximate 95% normal interval.

Method and moment equation. For an exponential distribution parameterized by its mean, $E[X]=\theta$ and $\operatorname{Var}(X)=\theta^2$. Use $h(X)=X$, so $g(\theta)=\theta$, $g'(\theta)=1$, and $\tau^2(\theta)=\theta^2$. The moment equation is $\bar X=\hat{\theta}$, giving $\hat{\theta}=8.4$ dollars.

Asymptotic variance and standard error. Substitute the estimate into the variance formula:

\widehat{\operatorname{Var}}(\hat{\theta}) =\frac{8.4^2}{100} =\frac{70.56}{100} =0.7056\text{ dollars}^2, \qquad \widehat{\operatorname{SE}}(\hat{\theta})=\sqrt{0.7056}=0.84\text{ dollars}.

Approximate interval. Using the standard normal critical value $1.96$, the margin is $1.96(0.84)=1.6464$ dollars. The endpoints are $8.4-1.6464=6.7536$ and $8.4+1.6464=10.0464$. The approximate interval is $[6.75,10.05]$ dollars, rounded to the nearest cent.

Check and trap avoided. The standard error is in dollars and the variance is in squared dollars. The calculation uses the variance of an individual exponential loss, $\theta^2$, divided by the sample size; it does not use $\theta^2$ as the variance of the estimator without dividing by 100.

Worked Example: A Poisson Frequency Estimate

A hypothetical portfolio has independent claim counts $X_1,\ldots,X_{64}$, each with a Poisson distribution of unknown mean $\lambda$ claims per policy-period. The observed mean count is $\bar X=2.25$. Find the moment estimate and its estimated asymptotic standard error, and give an approximate 95% interval for $\lambda$.

Method and moment equation. A Poisson random variable has $E[X]=\lambda$ and $\operatorname{Var}(X)=\lambda$. Taking $h(X)=X$ gives $g(\lambda)=\lambda$ and $g'(\lambda)=1$. Therefore the method-of-moments estimate is $\hat{\lambda}=\bar X=2.25$.

Estimated uncertainty. The asymptotic variance estimate is $2.25/64=0.03515625$ claims squared per policy-period squared. Its square root is $0.1875$ claims per policy-period, since $0.1875^2=0.03515625$.

Approximate interval. The normal-approximation margin is $1.96(0.1875)=0.3675$. Thus the endpoints are $2.25-0.3675=1.8825$ and $2.25+0.3675=2.6175$. The approximate 95% interval is $[1.8825,2.6175]$ claims per policy-period.

Check and trap avoided. The result is an approximation for the estimator, not a claim that individual counts are normally distributed. The variance used is the Poisson variance $\lambda$, not $\lambda^2$; substituting the observed mean and dividing by 64 gives the estimated variance of the sample mean.

Worked Example: A Nonunit Derivative for a Uniform Endpoint

Suppose $X_1,\ldots,X_{36}$ are independent hypothetical measurements from a uniform distribution on $(0,\theta)$, where $\theta>0$ is unknown. The observed sample mean is $7.5$ units. Find the moment estimate and its estimated asymptotic standard error.

Method and moment equation. For this uniform model, $E[X]=\theta/2$ and $\operatorname{Var}(X)=\theta^2/12$. The moment equation $\bar X=\hat{\theta}/2$ gives $\hat{\theta}=2\bar X=2(7.5)=15$ units.

Linearization and variance. Here $g(\theta)=\theta/2$, so $g'(\theta)=1/2$. The derivative contributes a factor $1/[g'(\theta)]^2=4$; omitting it would understate the estimator variance. Therefore,

\operatorname{Var}(\hat{\theta}) \ \approx\ \frac{\theta^2/12}{36(1/2)^2} =\frac{\theta^2}{108} =\frac{\theta^2}{3(36)}.

Substitute $\hat{\theta}=15$: the estimated variance is $15^2/108=225/108=25/12$, or approximately $2.0833$ squared units. The estimated standard error is $\sqrt{25/12}=5/\sqrt{12}=1.443375\ldots$ units. To four decimal places, it is $1.4434$ units.

Check and trap avoided. Directly using $\theta^2/(12n)$ as the estimator variance would give $225/432$, which is too small by a factor of four. The estimator is twice the sample mean, so its variance is four times the variance of the sample mean. This is a large-sample approximation; it is not an exact normal distribution for a finite sample.

Using and Interpreting the Approximation

The asymptotic standard error describes the typical sampling fluctuation of the estimator, not the variability of individual observations. It is often used to compare the precision of estimates from different sample sizes or to construct a normal-approximation interval. In a Wald-style interval, use $\hat{\theta}$ as the center and the estimated asymptotic standard error as the scale:

\hat{\theta}\ \pm\ z_{1-\alpha/2}\, \widehat{\operatorname{SE}}(\hat{\theta}).

Here $z_{1-\alpha/2}$ is the standard normal quantile with probability $1-\alpha/2$ to its left. This interval is approximate and may be unreliable when the sample is small, the distribution of the moment is strongly skewed, or the estimator lies near a parameter boundary. It can also extend outside the parameter space. Such a result is a warning about the interval approximation, not a reason to treat an invalid parameter value as possible.

The derivative explains an important aspect of precision. A small $|g'(\theta)|$ makes the asymptotic variance large because the population moment changes little as $\theta$ changes. In that situation, different parameter values produce similar moment values, so random variation in the sample moment makes the parameter harder to estimate. If $g'(\theta)=0$, the displayed formula does not apply: the first-order linearization has no parameter term to invert, and a different analysis is needed.

Common Exam Traps

Using the variance of an observation as the variance of its sample mean. The sample moment variance includes division by $n$. For example, $\operatorname{Var}(\bar X)=\operatorname{Var}(X)/n$.
Forgetting the derivative. The moment estimator’s variance is the moment variance divided by $n[g'(\theta)]^2$. A nonunit derivative changes the answer, as in the uniform example.
Using a variance formula for the wrong model. A Poisson variable has variance $\lambda$; an exponential variable parameterized by its mean $\theta$ has variance $\theta^2$. State the model and parameterization before substituting.
Calling the approximation exact. The normal result is a large-sample statement. A sample mean of discrete or skewed observations need not itself be normal for finite $n$.
Rounding too early. Keep unrounded standard errors and margins through interval calculations, then round the final endpoints to the requested precision.

Key takeaway: For a scalar moment equation $\bar h=g(\hat{\theta})$, the sample moment has asymptotic variance $\operatorname{Var}_\theta(h(X))/n$. Linearizing the equation divides that variance by $[g'(\theta)]^2$. Estimate the resulting standard error by substituting the moment estimate, and check the model, derivative, sample-size factor, and large-sample conditions.

Check Your Understanding

For each problem, identify the moment equation before applying the asymptotic variance formula.

Independent hypothetical Bernoulli observations have unknown success probability $p$. For $n=100$, the sample proportion is $0.30$. Give the moment estimate and plug-in asymptotic standard error. (Answer: $\hat p=0.30$; $\widehat{\operatorname{SE}}=\sqrt{0.30(0.70)/100}\approx0.0458$.)
Independent hypothetical exponential losses have unknown mean $\theta$. For $n=25$, the sample mean is $4.8$ dollars. Give the moment estimate and plug-in asymptotic standard error. (Answer: $\hat\theta=4.8$ dollars; $\widehat{\operatorname{SE}}=4.8/\sqrt{25}=0.96$ dollars.)
Independent observations are uniform on $(0,\theta)$. For $n=75$, the sample mean is $6$ units. Find the moment estimate and its plug-in asymptotic standard error. (Answer: $\hat\theta=12$ units; $\widehat{\operatorname{SE}}=12/\sqrt{3(75)}=0.80$ units.)
Independent hypothetical Poisson counts have unknown mean $\lambda$. For $n=100$, the sample mean is $1.44$. Give the plug-in asymptotic variance and standard error of the moment estimate. (Answer: $\widehat{\operatorname{Var}}(\hat\lambda)=1.44/100=0.0144$; $\widehat{\operatorname{SE}}=0.12$.)
Independent hypothetical exponential observations have unknown rate $\lambda>0$, so $E[X]=1/\lambda$ and $\operatorname{Var}(X)=1/\lambda^2$. For $n=80$, the sample mean is $0.5$. Use the moment equation and linearization to find the estimate and plug-in standard error. (Answer: $\hat\lambda=1/0.5=2$; the asymptotic variance is $\lambda^2/n$, so the plug-in standard error is $\sqrt{4/80}\approx0.2236$.)

What You'll Learn