From Sample Moments to Estimating Equations
A probability model describes population moments as functions of unknown parameters. A sample provides observed moments. The method of moments connects these two by setting selected sample moments equal to their model-based population counterparts and solving for the unknown parameter. The resulting equations are estimating equations: they are a way to construct an estimate, not identities that must hold exactly in the population.
For example, if a model says the population mean is \(g(\theta)\), where \(\theta\) is the unknown parameter, the first raw moment gives the equation \(g(\theta)=\bar X\). Solving this equation produces a method of moments estimate, provided a solution exists and lies in the allowed parameter space.
For one unknown parameter, the basic procedure uses one selected moment equation. Often that is the first raw moment, but it need not be: if the mean does not depend on the parameter, another moment may be informative. A solution must also satisfy the model’s parameter restrictions. An algebraic root outside the allowed range is not an admissible estimate.
Why Moment Matching Is Reasonable
The method is motivated by the behavior of sample moments. If the observations are iid and \(E_\theta[|X|^r]\) is finite, the law of large numbers gives \(M'_r\) close to \(\mu'_r(\theta)\) for a sufficiently large sample. The method reverses that relationship: it uses the observed sample moment to infer the parameter value whose model moment would match it.
The match is not usually exact for the unknown true parameter in a finite sample. Instead, the equation defines an estimate. A simple justification also shows how the equation is solved when the moment function is invertible.
Calculate \(\mu'_r(\theta)=E_\theta[X^r]\) from the assumed distribution.
Calculate \(M'_r=\frac{1}{n}\sum_{i=1}^n X_i^r\) from the data.
Set \(M'_r=\mu'_r(\theta)\), solve for \(\theta\), and retain only solutions in the parameter space.
If \(\mu'_r(\theta)=g(\theta)\) is one-to-one on the parameter space, this equation gives \(\hat\theta_{\text{MM}}=g^{-1}(M'_r)\), when that inverse is defined at the observed moment. The law of large numbers explains the motivation: the observed moment estimates the model moment. It does not guarantee that every finite sample gives a valid solution, or that a chosen equation has a unique solution.
Worked Method of Moments Equations
Worked Example: Estimating a Parameter in a Bounded Density Model
Assume \(X_1,\ldots,X_n\) are iid with density \(f_\theta(x)=1+\theta(2x-1)\) for \(0\le x\le1\), where \(-1\le\theta\le1\). A hypothetical sample is \(0.2, 0.4, 0.6, 0.8\). Find the first-moment method of moments estimate of \(\theta\).
Method. Match the sample mean to the model mean. The stated range of \(\theta\) ensures the density is nonnegative on the support. It integrates to one because \[ \int_0^1 [1+\theta(2x-1)]\,dx=1+\theta[x^2-x]_0^1=1. \]
The observed sample mean is \[ \bar X=\frac{0.2+0.4+0.6+0.8}{4}=\frac{2.0}{4}=0.5. \] The first-moment equation is \(0.5=\frac12+\frac{\theta}{6}\). Subtracting \(\frac12\) gives \(0=\frac{\theta}{6}\), so \(\hat\theta_{\text{MM}}=0\). This lies in \([-1,1]\), as required.
Check. At \(\theta=0\), the model is uniform on \([0,1]\), whose mean is \(1/2\), matching the sample mean. A common mistake is to equate \(\theta\) directly to \(\bar X\); the equation is between the model mean and the sample mean, not between the parameter and the sample mean.
Worked Example: Using a Second Raw Moment
Suppose iid observations take values \(-1,0,1\), with probabilities \(P_\theta(X=-1)=\theta/2\), \(P_\theta(X=0)=1-\theta\), and \(P_\theta(X=1)=\theta/2\), where \(0\le\theta\le1\). A hypothetical sample is \(-1,0,1,1,0,0\). Find a method of moments estimate using the second raw moment.
Method. First calculate the moment function. The first raw moment cannot identify \(\theta\), because symmetry gives \[ E_\theta[X]=(-1)\frac{\theta}{2}+0(1-\theta)+(1)\frac{\theta}{2}=0 \] for every allowed \(\theta\). Use the second raw moment instead: \[ E_\theta[X^2]=(-1)^2\frac{\theta}{2}+0^2(1-\theta)+(1)^2\frac{\theta}{2}=\theta. \]
The sample second raw moment is \[ M'_2=\frac{(-1)^2+0^2+1^2+1^2+0^2+0^2}{6} =\frac{3}{6}=0.5. \] The method of moments equation is \(M'_2=E_\theta[X^2]\), so \(0.5=\theta\), giving \(\hat\theta_{\text{MM}}=0.5\). This is within the allowed interval.
Check. At \(\theta=0.5\), the model assigns probability \(0.25\) to each of \(-1\) and \(1\), so \(E[X^2]=0.25+0.25=0.5\), as required. The trap is to use the first raw moment just because it is the simplest: here it is always zero and supplies no information about the parameter.
Worked Example: Solving a Nonlinear Moment Equation
Let iid observations have support \(\{0,1,2\}\), with \(P_\theta(X=0)=\theta\), \(P_\theta(X=1)=(1-\theta)^2\), and \(P_\theta(X=2)=\theta-\theta^2\), where \(0\le\theta\le1\). For a hypothetical sample \(0,1,1,1\), find the first-moment method of moments estimate.
Method. The probabilities are nonnegative on the stated parameter interval and sum to one: \[ \theta+(1-\theta)^2+\theta-\theta^2 =\theta+(1-2\theta+\theta^2)+\theta-\theta^2=1. \] Calculate the model mean: \[ E_\theta[X]=0\cdot\theta+1\cdot(1-\theta)^2+2(\theta-\theta^2) =1-2\theta+\theta^2+2\theta-2\theta^2=1-\theta^2. \]
The sample mean is \(\bar X=(0+1+1+1)/4=3/4\). Equating moments gives \(3/4=1-\theta^2\), so \(\theta^2=1/4\). The algebraic roots are \(\theta=1/2\) and \(\theta=-1/2\), but only \(1/2\) belongs to the allowed interval \([0,1]\). Thus \(\hat\theta_{\text{MM}}=1/2\).
Check. At \(\theta=1/2\), the probabilities are \(1/2,1/4,1/4\), which sum to one, and the model mean is \(0(1/2)+1(1/4)+2(1/4)=3/4\). The common mistake is to report both square-root solutions without checking the parameter space.
Conditions, Limits, and Exam Traps
A method of moments equation is only as useful as the model and moment chosen. The model must specify the distribution and parameter space; the required population moment must exist; and the observed sample moment must be calculated using the stated definition. For iid data with a finite relevant moment, the law of large numbers supports the matching idea. Without those conditions, the usual justification may fail.
A solution is not guaranteed. The sample moment may fall outside the range of the model moment function, or inversion may produce roots outside the parameter space. For example, in the preceding nonlinear model, every allowed \(\theta\) gives \(E_\theta[X]=1-\theta^2\le1\). If a sample from the support \(\{0,1,2\}\) has mean greater than one, the first-moment equation has no admissible solution. That does not justify changing the parameter constraint or silently using an invalid root; it means this selected equation does not produce an admissible estimate for that data set.
The method can also have more than one solution if the moment function is not one-to-one. In that case, solve the equation fully and apply the parameter restrictions; if multiple admissible roots remain, the equation alone has not selected a unique estimate. The method of moments is a construction rule, not a guarantee of uniqueness, admissibility, or optimal statistical performance.
- Do not match the parameter directly to a sample statistic. Match the model’s population moment, which is a function of the parameter, to the sample moment.
- Do not assume the first moment is informative. Check whether it actually depends on the unknown parameter.
- Do not confuse raw and central moments. The raw moment of order \(r\) averages \(X_i^r\); a central moment uses deviations from a mean. Use the moment specified by the equation you derive.
- Do not stop at algebra. Check the parameter space, and substitute the estimate back into the moment equation as a verification.
Check Your Understanding
For each problem, form the stated moment equation, solve it, and check whether the estimate is admissible.
- Assume iid observations have density \(f_\theta(x)=1+\theta(2x-1)\) on \(0\le x\le1\), with \(-1\le\theta\le1\). The observations are \(0.25,0.45,0.65,0.85\). Find the first-moment method of moments estimate. (Answer: \(\hat\theta_{\text{MM}}=0.3\))
- For iid observations with \(P_\theta(X=0)=\theta\), \(P_\theta(X=1)=1-2\theta\), and \(P_\theta(X=2)=\theta\), where \(0\le\theta\le0.5\), use the second raw moment. The sample is \(0,1,2,1,2\). Find the estimate. (Answer: \(\hat\theta_{\text{MM}}=0.5\))
- Suppose a one-parameter model has \(E_\theta[X]=\frac{\theta}{1+\theta}\), with \(\theta>0\). The sample mean is \(0.6\). Solve the first-moment equation. (Answer: \(\hat\theta_{\text{MM}}=1.5\))
- In the model \(P_\theta(X=0)=\theta\), \(P_\theta(X=1)=(1-\theta)^2\), \(P_\theta(X=2)=\theta-\theta^2\), \(0\le\theta\le1\), a sample has mean \(1.2\). Does the first-moment equation give an admissible estimate? (Answer: No; \(1.2=1-\theta^2\) has no real solution.)
- For iid observations with \(P_\theta(X=-1)=P_\theta(X=1)=\theta/2\) and \(P_\theta(X=0)=1-\theta\), \(0\le\theta\le1\), the sample is \(-1,1,0,0\). Use the second raw moment and verify the estimate. (Answer: \(\hat\theta_{\text{MM}}=0.5\), since \(M'_2=2/4=0.5=E_\theta[X^2]\).)