From a Model to Moment Estimates
The method of moments turns a specified probability model into equations that can be solved using sample data. The central task is not merely to recall a formula: it is to identify the model’s unknown parameters, choose enough sample moments to determine them, solve the resulting system, and check that the solution is admissible.
Suppose \(X_1,\ldots,X_N\) are independent observations from the same distribution, whose parameter vector is \(\boldsymbol{\theta}=(\theta_1,\ldots,\theta_k)\). The method of moments matches sample moments to the corresponding theoretical moments. For \(k\) unknown parameters, it typically uses \(k\) moment equations. The equations need to determine the parameters uniquely, and the resulting values must satisfy the model’s parameter restrictions.
For the first two raw moments, the empirical variance using divisor \(N\) is \(m'_2-(m'_1)^2\). This is the variance of the empirical distribution that places equal probability on each observed value. It is not generally the usual unbiased sample variance, which uses divisor \(N-1\). Method-of-moments calculations should use the moments specified or calculated from the data, not silently switch variance conventions.
A Reliable Solving Routine
A method-of-moments solution is a small system-solving problem. The same routine applies whether the observations are counts, measurements, or proportions.
Specify the distribution, its parameter meanings, and the assumptions about how the observations were collected.
Choose at least as many moment equations as there are unknown parameters, using moments that can identify them.
Use the observed data or supplied summaries to obtain the needed empirical raw moments or the equivalent mean and variance.
Solve the equations, test the parameter restrictions, and substitute the estimates back into the model moments.
The check is substantive, not cosmetic. A numerical solution may be negative when a parameter must be positive, or may fail to reproduce the sample moments because of an algebra error. Even when the equations match exactly, that does not establish that the model itself is appropriate; the model is taken as specified for this estimation exercise.
Why Moment Matching Is Reasonable
Moment matching uses the fact that sample averages tend to reflect population averages when observations are independent and identically distributed and the relevant moments exist. For example, \(m'_1\) averages the observed values, while \(E[X]\) is the population average. The law of large numbers explains why \(m'_1\) is a natural stand-in for \(E[X]\) in a sufficiently large sample.
For several parameters, the selected population moments are functions of those parameters. Equating them to the observed sample moments gives equations to invert. This reasoning motivates the method; it does not guarantee that the equations have a unique solution or that a finite-sample estimate will be plausible. Those questions must be checked in each model.
Worked Example: Recovering Binomial Parameters From Two Moments
Assume \(X_1,\ldots,X_N\) are independent observations from a common binomial distribution with an unknown number \(M\) of trials and an unknown success probability \(p\). The recorded summaries are \(m'_1=6\) and \(m'_2=39\). Find the method-of-moments estimates of \(M\) and \(p\).
Method. For \(X\sim\operatorname{Binomial}(M,p)\), the model has mean \(E[X]=Mp\) and variance \(\operatorname{Var}(X)=Mp(1-p)\). There are two unknowns, so use the first two raw moments, or equivalently the mean and the variance derived from them.
First calculate the empirical variance:
Set the model mean equal to 6 and the model variance equal to 3:
Divide the variance equation by the mean equation. Since the fitted mean is nonzero, this gives \(1-p=3/6=0.5\), so \(\hat p=0.5\). Then \(M\hat p=6\), which gives \(\hat M=6/0.5=12\).
Verification. The estimates satisfy the restrictions \(0<\hat p<1\) and positive integer \(\hat M\). Their fitted mean is \(12(0.5)=6\), and their fitted variance is \(12(0.5)(0.5)=3\). Thus the fitted second raw moment is \(3+6^2=39\), exactly the supplied value.
Common mistake avoided. Do not set \(Mp(1-p)=m'_2\). The second raw moment is not the variance: here it is 39, whereas the variance derived from the two raw moments is 3.
Solving for Two Shape Parameters
For a model with two parameters that both affect the mean and spread, two moments are often needed. The beta distribution provides a useful example because its parameters jointly determine the location and concentration of observations on the interval from 0 to 1.
Worked Example: Estimating Beta Shape Parameters
Assume \(X_1,\ldots,X_N\) are independent observations from a beta distribution with unknown shape parameters \(\alpha>0\) and \(\beta>0\). The observations are proportions, and their empirical raw moments are \(m'_1=0.4\) and \(m'_2=0.20\). Estimate both shape parameters.
Method. Write \(s=\alpha+\beta\). For this beta model, the mean is \(\alpha/s\), the variance is \(\alpha\beta/[s^2(s+1)]\), and the variance can also be written as \(\mu(1-\mu)/(s+1)\), where \(\mu=\alpha/s\).
The empirical mean is \(\hat\mu=0.4\). First obtain the empirical variance from the raw moments:
Match the model mean and variance. The variance equation gives
Therefore \(s+1=0.24/0.04=6\), so \(s=5\). Since \(\alpha/s=0.4\), \(\hat\alpha=5(0.4)=2\). Then \(\hat\beta=s-\hat\alpha=5-2=3\).
Verification. The estimated mean is \(2/(2+3)=0.4\). The estimated variance is \(2(3)/[5^2(6)]=6/150=0.04\). The second raw moment is consequently \(0.04+(0.4)^2=0.20\). Both supplied moments are reproduced, and both shape estimates are positive.
Common mistake avoided. The variance formula contains \(s+1\), not just \(s\). Omitting the \(+1\) would give a different total shape and would fail the variance check.
Using a Variance-to-Mean Ratio
For some count distributions, ratios of moments simplify the equations. A negative binomial model illustrates this strategy. Here \(X\) is the number of failures before the \(r\)th success, each trial has success probability \(p\), and \(r>0\). Under this convention, \(E[X]=r(1-p)/p\) and \(\operatorname{Var}(X)=r(1-p)/p^2\). A clear statement of the convention matters because other negative binomial definitions count successes or use different parameter labels.
Worked Example: Solving a Negative Binomial Moment System
Assume independent observations follow the negative binomial convention just stated, with unknown \(r\) and \(p\). Their empirical raw moments are \(m'_1=4\) and \(m'_2=24\). Find and verify the method-of-moments estimates.
Method. Convert the raw second moment to a variance:
The theoretical variance divided by the theoretical mean is
Equating this ratio to the empirical variance-to-mean ratio gives \(8/4=1/p\), so \(\hat p=4/8=0.5\). Next, use the mean equation \(r(1-p)/p=4\):
Verification. With \(r=4\) and \(p=0.5\), the mean is \(4(0.5)/0.5=4\), and the variance is \(4(0.5)/(0.5)^2=8\). The fitted second raw moment is \(8+4^2=24\). The estimates also satisfy \(r>0\) and \(0<p<1\).
Common mistake avoided. The ratio \(\operatorname{Var}(X)/E[X]\) equals \(1/p\), not \(p\). Reversing the ratio would give \(p=2\), which is impossible for a success probability. The parameter restriction immediately flags that error.
What the Verification Does—and Does Not—Show
Substitution into the moment equations is an efficient way to catch algebra mistakes. Check every chosen equation, not just the first one: a proposed parameter pair may reproduce the mean but not the variance. Also check the model’s parameter space. For example, probabilities must lie between 0 and 1, scale parameters must be positive, and an unknown binomial trial count must be a positive integer.
A solution that violates a restriction is not repaired by rounding it into range. It indicates that the moment equations have produced an inadmissible estimate for the stated model, or that an equation or calculation should be reviewed. Similarly, matching moments does not prove the model fits all features of the data: different distributions can share the same first few moments.
Keep the distinction between moment estimation and other estimation criteria clear. The method here is defined by matching theoretical moments to empirical moments. It does not choose parameters by maximizing a likelihood, and exact moment matching in a sample does not imply that the estimates are optimal under every statistical criterion.
Check Your Understanding
For each problem, state the model assumptions, form the needed moment equations, and check the fitted moments and parameter restrictions.
- Independent counts follow a binomial distribution with unknown trial count \(M\) and success probability \(p\). The empirical mean is 4 and the empirical variance, calculated with divisor \(N\), is 2. Find the method-of-moments estimates. (Answer: \(\hat p=0.5,\ \hat M=8\).)
- Independent observations follow a gamma distribution parameterized by shape \(k>0\) and scale \(\theta>0\), with mean \(k\theta\) and variance \(k\theta^2\). The empirical mean is 10 and the empirical variance is 20. Estimate both parameters and state the fitted second raw moment. (Answer: \(\hat k=5,\ \hat\theta=2\); fitted second raw moment \(=120\).)
- Independent observations follow a continuous uniform distribution on an unknown interval \([a,b]\). The empirical mean is 7 and the empirical variance is 12. Use the uniform mean and variance equations to estimate both endpoints, then verify the width and midpoint. (Answer: \(\hat a=1,\ \hat b=13\); width \(12\), midpoint \(7\).)
- Independent observations count failures before the \(r\)th success in a negative binomial model. The empirical mean is 6 and empirical variance is 12. Estimate \(p\) and \(r\), then give the fitted second raw moment. (Answer: \(\hat p=0.5,\ \hat r=6\); fitted second raw moment \(=48\).)