Method Of Moments Versus Likelihood Efficiency

One Model, More Than One Moment

For a single-parameter model, the method of moments and maximum likelihood can lead to the same estimator—or to estimators with different precision. The exponential distribution makes this comparison especially clear. Its usual first-moment estimator of the rate is exactly the maximum-likelihood estimator. But an estimator built from the second raw moment has a larger asymptotic variance. The difference is not that moments are inherently inefficient; it is that some moment choices retain less information about the parameter than others.

We use the rate parameterization throughout: \(X_1,\ldots,X_n\) are independent and identically distributed, with \(X_i\sim\operatorname{Exp}(\lambda)\) and \(\lambda>0\). Thus \(f(x;\lambda)=\lambda e^{-\lambda x}\) for \(x>0\), and \(E[X_i]=1/\lambda\). The sample is assumed to come from this model.

The Usual Moment Estimator and the MLE

The first population moment is \(E[X_i]=1/\lambda\). Matching it to the sample mean gives \(1/\hat{\lambda}_{\mathrm{MM}}=\bar X\), so \(\hat{\lambda}_{\mathrm{MM}}=1/\bar X\). The likelihood gives the same answer: its logarithm is \(n\log\lambda-\lambda\sum_{i=1}^n X_i\), whose maximum occurs at \(n/\sum_i X_i\).

Asymptotic Variance of the Shared Estimator

For one observation, the score is \(s_\lambda(X)=\partial\log f(X;\lambda)/\partial\lambda=1/\lambda-X\). Since \(E[X]=1/\lambda\) and \(\operatorname{Var}(X)=1/\lambda^2\), the Fisher information per observation is \(I(\lambda)=E[s_\lambda(X)^2]=1/\lambda^2\). The corresponding asymptotic information bound for a regular estimator of \(\lambda\) is \(1/I(\lambda)=\lambda^2\).

The shared estimator is a smooth function of the sample mean. Because \(E[X]=1/\lambda\) and \(\operatorname{Var}(X)=1/\lambda^2\), the central limit theorem gives \(\sqrt n(\bar X-1/\lambda)\xrightarrow{d}N(0,1/\lambda^2)\). Applying the delta method to \(g(x)=1/x\), whose derivative at \(1/\lambda\) is \(-\lambda^2\), yields the following result.

The derivation requires a finite variance for \(X_i\), a nonzero population mean, and differentiability of \(g(x)=1/x\) near \(1/\lambda>0\). These conditions hold in the stated model. There is no asymptotic efficiency loss for the usual moment estimator because it is not a different estimator from the MLE.

What If We Match the Second Raw Moment?

A different moment equation gives a different estimator. For an exponential random variable, \(E[X^2]=2/\lambda^2\). Equating this to the sample second raw moment \(M_2=n^{-1}\sum_iX_i^2\) gives \(\hat{\lambda}_2=\sqrt{2/M_2}\). This estimator is consistent because \(M_2\xrightarrow{p}2/\lambda^2\), and the function \(m\mapsto\sqrt{2/m}\) is continuous at that positive limit.

Why the Second-Moment Choice Is Less Efficient

The difference can be seen by comparing how the estimators respond to individual observations. For an estimator formed by matching a moment \(h(X)\), let \(m(\lambda)=E_\lambda[h(X)]\) and suppose \(m\) is differentiable with \(m'(\lambda)\ne0\). The estimating equation is \(n^{-1}\sum_i h(X_i)=m(\lambda)\). A first-order Taylor expansion around the true \(\lambda\) gives the asymptotic variance

Indeed, writing \(\hat\lambda\) for the solution, the equation and Taylor expansion imply

The central limit theorem applied to \(h(X_i)\) then gives the stated variance, provided \(h(X)\) has finite variance and the solution is consistent. For \(h(X)=X\), this formula gives \((1/\lambda^2)/(-1/\lambda^2)^2=\lambda^2\). For \(h(X)=X^2\), it gives \((20/\lambda^4)/(-4/\lambda^3)^2=5\lambda^2/4\).

For a further explanation, the first-order influence function associated with this moment equation is \(\psi(X)=\{h(X)-m(\lambda)\}/m'(\lambda)\). In the regular exponential model, the score has variance \(1/\lambda^2\). Differentiating the moment under the integral gives \(E_\lambda[\{h(X)-m(\lambda)\}s_\lambda(X)]=m'(\lambda)\), so \(E_\lambda[\psi(X)s_\lambda(X)]=1\). By the Cauchy–Schwarz inequality,

Thus the moment estimator’s asymptotic variance cannot be below \(1/I(\lambda)=\lambda^2\). Equality requires \(\psi(X)\) to be proportional to the score, apart from events of probability zero. With \(h(X)=X\), the influence function is \(-\lambda^2(X-1/\lambda)=\lambda(1/\lambda-X)\), proportional to the score. With \(h(X)=X^2\), it is \(-\lambda^3(X^2-2/\lambda^2)/4\), a quadratic function of \(X\), not proportional to the score \(1/\lambda-X\). Since the exponential distribution has a continuous positive density on \(x>0\), those two functions cannot be proportional almost surely. The inequality is strict, as the variance calculation confirms. \(\square\)

Common Mistake and Practical Meaning

A common mistake is to say that “the method-of-moments estimator” for an exponential rate is necessarily less efficient than the MLE. For one unknown rate, the usual method of moments matches the first raw moment and gives \(1/\bar X\), exactly the MLE. A different result arises only after specifying a different moment equation, such as matching the second raw moment alone. The correct comparison is between particular estimators, not between the names of two broad estimation methods.

This distinction matters whenever a model offers several possible moment conditions. A moment may be easy to calculate and still produce a less precise estimator. For the exponential rate, the first moment captures the score’s linear dependence on \(X\), while the second-moment estimator responds quadratically. The calculation shows how that choice affects precision: the second-moment estimator’s asymptotic variance is \(5/4\) times the information bound.