When Sample Moments Reach the Edge of a Model
Method of moments estimates parameters by matching theoretical moments to corresponding moments calculated from the sample. Usually this means solving equations such as \(E[X]=\bar X\) and \(E[X^2]=n^{-1}\sum_{i=1}^n x_i^2\). But a sample can have an unusual feature—such as every observation being identical, or one observation lying far from the rest—that pushes the equations to the edge of the parameter space. The resulting estimate may be valid at a boundary, may describe a degenerate model excluded by the assumptions, or may fail to satisfy a model constraint.
The key distinction is between solving the moment equations and obtaining an admissible estimate. A numerical solution is not automatically an allowed parameter value or a model that could have produced the observed data.
Because every squared deviation is nonnegative, \(v_n\geq 0\). Moreover, \(v_n=0\) if and only if every observation is equal to the sample mean. This fact makes constant samples a direct test of whether a model’s moment equations can be solved at a boundary.
Boundary Values, Degeneracy, and Failure
A parameter space may include its endpoints or may require parameters to be strictly inside them. That difference determines whether an endpoint solution is admissible. For example, a Bernoulli probability is often allowed to be \(0\) or \(1\), while a normal variance must be strictly positive for a nondegenerate normal distribution. In the first case an endpoint estimate can be a valid boundary estimate; in the second, a zero variance does not describe a normal distribution under the usual model.
A second issue is support. A fitted model must assign positive density or probability to the observed values, as appropriate. For example, if a model has support \(x\geq\theta\), then an estimate \(\hat\theta\) must not exceed the smallest observed value. Moment equations can produce a numerical \(\hat\theta\) that violates this requirement.
| What the equations produce | What it means | What to check |
|---|---|---|
| Parameter at an included endpoint | A boundary estimate may be admissible. | Does the stated parameter space include the endpoint? |
| Parameter at an excluded endpoint | The equations have a limiting or boundary solution, but not an admissible one. | Does the model require a strict inequality, such as variance greater than zero? |
| Parameter outside its allowed range | The moment solution is infeasible. | Does it satisfy all parameter restrictions and support conditions? |
| No solution to the equations | The selected moments cannot be matched by that model. | Can the equations be solved simultaneously within the parameter space? |
Why Constant Samples Force Zero Empirical Variance
The identity defining \(v_n\) gives an immediate justification. If \(x_i=c\) for every \(i\), then \(m_1'=c\), \(m_2'=c^2\), and \(v_n=c^2-c^2=0\). Conversely, if \(v_n=0\), it is an average of nonnegative terms \((x_i-m_1')^2\). Their sum can be zero only if each term is zero, so \(x_i=m_1'\) for every \(i\). Thus a zero empirical variance is not merely a small variance: it exactly identifies a constant sample.
Worked Example: A Bernoulli Estimate at an Included Boundary
A hypothetical audit records whether each of five independently selected forms contains a specified coding error. Let \(X_i=1\) for an error and \(X_i=0\) otherwise, with \(X_i\sim\operatorname{Bernoulli}(p)\) and \(0\leq p\leq1\). The observations are \(1,1,1,1,1\). Find the method-of-moments estimate of \(p\), and determine whether it is admissible.
Method: match the first moment. For a Bernoulli variable, \(E[X_i]=p\). The empirical first moment is \(m_1'=(1+1+1+1+1)/5=5/5=1\).
Equating the moments gives \(p=m_1'\), so \(\hat p_{\mathrm{MOM}}=1\). The estimate is at the upper endpoint of the specified parameter space, but that endpoint is included. It is therefore an admissible boundary estimate. As a check, the empirical second moment is \(m_2'=1\), and the empirical variance is \(v_n=1-1^2=0\), consistent with all five outcomes being identical.
Common mistake to avoid: Rejecting the estimate solely because it is at a boundary. Here the model permits \(p=1\); a parameter space restricted instead to \(0<p<1\) would exclude it.
Zero Variance Can Collapse a Model
For a nondegenerate normal model \(X\sim N(\mu,\sigma^2)\), the moment equations are \(E[X]=\mu\) and \(\operatorname{Var}(X)=\sigma^2\), with \(\sigma^2>0\). Matching the empirical mean and variance gives \(\hat\mu_{\mathrm{MOM}}=m_1'\) and \(\hat\sigma^2_{\mathrm{MOM}}=v_n\). If all observations are identical, the equations force \(\hat\sigma^2_{\mathrm{MOM}}=0\). This solves the variance equation algebraically, but zero is not an allowed variance for a nondegenerate normal distribution. It describes a point mass, not a normal distribution with positive variance.
The same issue appears in other models whose moment equations require a strictly positive spread. The right conclusion is not that the sample variance should be altered to make it positive. Instead, report that the unrestricted moment equations reach an excluded boundary and hence do not give an admissible estimate under the stated model.
Worked Example: Constant Measurements Under a Normal Model
A hypothetical device produces four readings, in units, recorded as \(6,6,6,6\). Model the readings as independent observations from \(N(\mu,\sigma^2)\), where \(\mu\) is real and \(\sigma^2>0\). Find the moment estimates and assess whether they define a permitted normal model.
Method: match the first two moments. The sample first moment is \(m_1'=(6+6+6+6)/4=24/4=6\). The second raw moment is \(m_2'=(36+36+36+36)/4=144/4=36\). Therefore \(v_n=m_2'-(m_1')^2=36-6^2=36-36=0\).
The first-moment equation gives \(\hat\mu_{\mathrm{MOM}}=6\). The variance equation gives \(\hat\sigma^2_{\mathrm{MOM}}=0\). The latter violates the required condition \(\sigma^2>0\), so the moment equations have no admissible solution in the nondegenerate normal family. In the closure that permits a point mass, the observations correspond to a degenerate distribution concentrated at 6, but that is not the specified normal model.
Check: each observed deviation from 6 is zero, so the empirical variance must be zero. Replacing divisor \(n\) with \(n-1\) would still give zero here and would not repair the boundary problem.
Common mistake to avoid: Calling \(N(6,0)\) an ordinary normal distribution. Under the stated model, the variance must be positive.
Extreme Observations and Support Constraints
A sample need not be constant to cause trouble. In a model with a parameter-dependent support, moment equations may return a parameter value incompatible with one or more observed values. The equations match averages; they do not automatically enforce the support restrictions.
For a shifted exponential model with location \(\theta\) and scale \(\beta>0\), the density is \(f_X(x)=\beta^{-1}\exp(-(x-\theta)/\beta)\) for \(x\geq\theta\), and zero for \(x<\theta\). Its mean is \(\theta+\beta\), and its variance is \(\beta^2\). Matching the first two moments gives \(\hat\beta=\sqrt{v_n}\) and \(\hat\theta=m_1'-\sqrt{v_n}\), taking the positive square root because \(\beta>0\). A valid fitted support must also satisfy \(\hat\theta\leq\min_i x_i\).
Worked Example: A Shift Estimate Above the Smallest Observation
Three hypothetical waiting times, in hours, are recorded as \(0,1,1\). Suppose they are to be modeled as independent observations from a shifted exponential distribution with unrestricted real location \(\theta\), positive scale \(\beta\), and support \(x\geq\theta\). Apply the moment equations and check the support condition.
Method: match the mean and variance. First, \(m_1'=(0+1+1)/3=2/3\). Next, \(m_2'=(0^2+1^2+1^2)/3=2/3\). Hence \(v_n=2/3-(2/3)^2=2/3-4/9=6/9-4/9=2/9\).
The scale estimate is \(\hat\beta=\sqrt{2/9}=\sqrt{2}/3\), which is positive. The location estimate is \(\hat\theta=2/3-\sqrt{2}/3=(2-\sqrt{2})/3\), approximately \(0.1953\). But the smallest observation is \(0\), so the fitted support would require every observation to be at least \(0.1953\). The recorded zero violates that condition. Thus the moment equations have a numerical solution, but it is not a feasible fit to these data under the specified shifted exponential model.
Check: \((2-\sqrt{2})/3>0\) because \(2>\sqrt{2}\), confirming that \(\hat\theta\) lies above the observed minimum. The scale remains positive, so the failure is specifically the support constraint, not the scale restriction.
Common mistake to avoid: Stopping after calculating a positive scale and a real location. For a shifted distribution, always compare the fitted location with the smallest observation.
Moment Equations for a Uniform Distribution
A constant sample can also collapse a two-endpoint model. Suppose \(X\) is uniform on \((a,b)\), with \(a<b\). Its mean and variance are \((a+b)/2\) and \((b-a)^2/12\). Given empirical moments \(m_1'\) and \(v_n\), the equations imply \[ a=m_1'-\sqrt{3v_n},\qquad b=m_1'+\sqrt{3v_n}. \] These formulas follow by adding and subtracting the moment equations: \(a+b=2m_1'\), while \((b-a)^2=12v_n\). Since \(b-a>0\), take \(b-a=\sqrt{12v_n}=2\sqrt{3v_n}\); solving the sum and difference yields the displayed endpoints. If \(v_n=0\), both endpoints equal \(m_1'\), contradicting \(a<b\).
Worked Example: A Collapsed Uniform-Endpoint Estimate
A hypothetical sample of three material thicknesses, in millimeters, is \(4,4,4\). Suppose the observations are modeled by a uniform distribution on \((a,b)\), where \(a<b\). Calculate the moment estimates for both endpoints and decide whether they are admissible.
Method: match the uniform mean and variance. The empirical mean is \(m_1'=(4+4+4)/3=4\), and \(m_2'=(16+16+16)/3=16\). Thus \(v_n=16-4^2=0\). The endpoint formulas give \(\hat a=4-\sqrt{3(0)}=4\) and \(\hat b=4+\sqrt{3(0)}=4\).
The equations produce a collapsed interval with \(\hat a=\hat b\), not the required \(a<b\). Therefore there is no admissible method-of-moments estimate in the nondegenerate uniform family. The fact that the sample mean is 4 does not make \((4,4)\) a valid uniform interval.
Common mistake to avoid: Reporting two equal endpoints as a valid uniform estimate. Check the strict endpoint ordering after solving the equations.
A Reliable Boundary-Check Procedure
When moment estimates look extreme, use a systematic check rather than forcing an interior answer. In particular, keep the original empirical moments visible: rounding an estimate or changing the variance convention can conceal the reason an equation reaches a boundary.
Use \(m_1'\) and \(m_2'\), then calculate \(v_n=m_2'-(m_1')^2\) with divisor \(n\).
Show the algebra and identify any square roots, divisions, or other operations that require special conditions.
Check strict positivity, endpoint inclusion, and any ordering or range restrictions.
For a support depending on the parameter, verify that every observation lies in the fitted support.
Distinguish an admissible boundary estimate from an excluded boundary, an infeasible numerical solution, or a failure to solve the equations.
Common Exam Traps
- Treating every boundary as invalid. A Bernoulli estimate of \(p=1\) is allowed if the parameter space includes 1. Check the stated range.
- Treating every algebraic solution as a valid estimate. A zero normal variance or equal uniform endpoints violate the nondegeneracy conditions in those models.
- Checking parameter ranges but not support. A positive scale and real location do not suffice if an observation lies below the fitted location.
- Confusing a small variance with a zero variance. If \(v_n>0\), the sample is not constant, even if the value is close to zero. Apply the equations to the actual calculated moments.
- Using the unbiased sample variance without checking the convention. Standard method-of-moments matching uses the empirical raw moments with divisor \(n\); substituting a divisor-\((n-1)\) variance changes the equations.
Check Your Understanding
For each problem, calculate the relevant moments and decide whether the result is an admissible estimate under the stated model.
- A Bernoulli sample is \(0,0,0,0\), and the parameter space is \(0\leq p\leq1\). Find the method-of-moments estimate and classify it. (Answer: \(\hat p_{\mathrm{MOM}}=0\), an admissible lower-boundary estimate.)
- Four observations from a nondegenerate \(N(\mu,\sigma^2)\) model are \(2,2,2,2\), with \(\sigma^2>0\). Give both moment estimates and assess admissibility. (Answer: \(\hat\mu_{\mathrm{MOM}}=2\), \(\hat\sigma^2_{\mathrm{MOM}}=0\); no admissible nondegenerate normal estimate.)
- For observations \(2,2,8\) modeled as uniform on \((a,b)\), calculate the moment estimates of \(a\) and \(b\). (Answer: \(m_1'=4\), \(v_n=8\), \(\hat a=4-2\sqrt{6}\), \(\hat b=4+2\sqrt{6}\); these satisfy \(\hat a<\hat b\) and contain all three observations.)
- For shifted exponential observations \(1,2,2\), use \(\hat\beta=\sqrt{v_n}\) and \(\hat\theta=m_1'-\sqrt{v_n}\), then check the support. (Answer: \(m_1'=5/3\), \(v_n=2/9\), \(\hat\beta=\sqrt{2}/3\), \(\hat\theta=(5-\sqrt{2})/3\approx1.1953>1\); the observation 1 is outside the fitted support.)
- For a Beta model on \((0,1)\), the usual moment equations use \(m=E[X]\), \(v=\operatorname{Var}(X)\), and \(k=m(1-m)/v-1\), with shape parameters \(\alpha=mk\) and \(\beta=(1-m)k\). The sample is \(0,0,1,1\). Calculate the moment quantities and explain the result. (Answer: \(m=1/2\), \(v=1/4\), \(k=0\), and \(\alpha=\beta=0\), which is inadmissible; moreover, the endpoint observations are outside the standard Beta support \((0,1)\).)