Convergence Of Empirical Raw Moments

Why Empirical Moments Have Limits

A sample average of powers summarizes features of a distribution: the first raw moment is its mean, the second records the average squared value, and higher orders give increasing weight to large observations. In econometrics, these quantities often enter estimators and sample criteria. Before treating a sample moment as a stable estimate of a population quantity, we need to know whether that population moment exists and whether a law of large numbers applies.

The key move is simple: the average of the $k$th powers is an ordinary sample average, but of transformed observations. The weak law therefore gives a direct convergence result, provided the transformed variable has a finite mean. For powers, this condition is a finite absolute moment of the corresponding order.

Definition: For observations $X_1,\ldots,X_n$, the empirical raw moment of order $k$ is $$ m_{k,n}=\frac{1}{n}\sum_{i=1}^{n}X_i^k. $$ If $X$ has a finite expectation of its $k$th power, the corresponding population raw moment is $\mu_k=\mathbb{E}[X^k]$. “Raw” means that the powers are taken from $X_i$, not from deviations around a sample or population mean.

Throughout, take $k$ to be a positive integer and $X_1,X_2,\ldots$ to be independent and identically distributed (iid) copies of $X$. The empirical moment $m_{k,n}$ is measured in the units of $X$ raised to the $k$th power. For example, if $X$ is measured in dollars, then $m_{2,n}$ and $\mu_2$ are measured in dollars squared.

The Weak-Law Result for Powers

The weak law of large numbers for iid variables with finite mean says that the sample average of an integrable variable converges in probability to its expectation. Apply it to $Z_i=X_i^k$. The relevant integrability condition is $\mathbb{E}|Z_i|<\infty$, which is exactly $\mathbb{E}|X|^k<\infty$. This absolute-moment condition is important, particularly for odd powers: positive and negative values can cancel in a formal calculation, but cancellation alone does not make an expectation finite.

Theorem: Suppose $X_1,X_2,\ldots$ are iid and, for a positive integer $k$, $\mathbb{E}|X|^k<\infty$. Then the population raw moment $\mu_k=\mathbb{E}[X^k]$ is finite, and $$ m_{k,n}=\frac{1}{n}\sum_{i=1}^{n}X_i^k \xrightarrow{p} \mu_k. $$ That is, for every $\varepsilon>0$, $P(|m_{k,n}-\mu_k|>\varepsilon)$ tends to zero as $n$ tends to infinity.

Transform each observation.
Set $Z_i=X_i^k$. Because the $X_i$ are iid, the $Z_i$ are also iid.

Check the mean condition.
$\mathbb{E}|Z_i|=\mathbb{E}|X_i^k|=\mathbb{E}|X|^k<\infty$, so $Z_i$ has a finite expectation.

Apply the weak law.
The average $n^{-1}\sum_{i=1}^{n}Z_i$ converges in probability to $\mathbb{E}[Z_i]$. These are $m_{k,n}$ and $\mu_k$, respectively.

This argument uses the weak law with a finite mean, not a finite-variance requirement for $X^k$. A stronger condition, $\mathbb{E}|X|^{2k}<\infty$, is sufficient because it makes $X^k$ have finite variance. In that case, the finite-variance weak law also gives the bound

P\bigl(|m_{k,n}-\mu_k|>\varepsilon\bigr) \leq \frac{\operatorname{Var}(X^k)}{n\varepsilon^2}.

But that bound is not needed for the theorem above. Requiring a finite $2k$th absolute moment would rule out some distributions for which the finite-mean weak law still applies.

Worked Example: A Second Raw Moment With Signed Observations

Let each hypothetical observation, measured in index points, be drawn iid from a distribution with $P(X=-2)=0.2$, $P(X=1)=0.5$, and $P(X=3)=0.3$. Find the population second raw moment and state the convergence result. Suppose a particular sample of five observations is $-2,1,1,3,1$.

Check the condition. The distribution has only three possible values, so $\mathbb{E}|X|^2$ is finite. In fact, $$ \mathbb{E}|X|^2=0.2(4)+0.5(1)+0.3(9)=0.8+0.5+2.7=4.0. $$ The iid assumption also holds by the setup.

Find the target. Since squaring removes the sign, $$ \mu_2=\mathbb{E}[X^2]=0.2(-2)^2+0.5(1)^2+0.3(3)^2=4.0 $$ in squared index points. Thus $m_{2,n}\xrightarrow{p}4.0$ as the sample size increases.

Calculate the sample moment. For the stated hypothetical sample, $$ m_{2,5}=\frac{(-2)^2+1^2+1^2+3^2+1^2}{5} =\frac{16}{5}=3.2 $$ squared index points. A sample value of 3.2 does not contradict convergence to 4.0: convergence in probability is a statement about the behavior of the estimator across repeated samples as $n$ grows, not a claim that every finite-sample moment equals its target.

Moment Existence Determines Which Orders Are Available

The condition depends on the order being studied. A finite first absolute moment supports convergence of the sample mean; a finite second absolute moment supports convergence of the second raw moment; and a finite $k$th absolute moment supports convergence of the $k$th raw moment. Finite variance of $X$ is not enough to guarantee that every higher raw moment exists. A distribution can have a well-defined variance and still have an infinite third or fourth absolute moment.

The following heavy-tailed example makes the order-by-order distinction explicit. For a nonnegative variable, the absolute moment $\mathbb{E}|X|^k$ equals $\mathbb{E}[X^k]$, so no cancellation between signs is involved.

Worked Example: A Pareto Distribution With a Missing Third Moment

Suppose hypothetical iid observations follow a Pareto distribution with scale 1 and tail probability $P(X>x)=x^{-5/2}$ for $x\geq 1$. Its density is $f(x)=(5/2)x^{-7/2}$ for $x\geq1$. Determine which of the first three raw moments the weak-law result covers.

For a positive order $r$, integrate the density: $$ \mathbb{E}[X^r]=\frac{5}{2}\int_1^\infty x^{r-7/2}\,dx. $$ The integral is finite exactly when $r-7/2<-1$, or $r<5/2$. When it is finite, evaluating the integral gives $$ \mathbb{E}[X^r]=\frac{5/2}{5/2-r}. $$

For $r=1$, the mean is $(5/2)/(3/2)=5/3$. For $r=2$, the second raw moment is $(5/2)/(1/2)=5$. Both moments are finite, so the empirical first and second raw moments converge in probability to $5/3$ and $5$, respectively. For $r=3$, the integral is infinite. There is no finite population third raw moment to which the finite-mean weak-law result can be applied.

For illustration, a hypothetical sample $1,1.2,2$ has $$ m_{2,3}=\frac{1^2+1.2^2+2^2}{3} =\frac{6.44}{3}\approx2.147, \qquad m_{3,3}=\frac{1^3+1.2^3+2^3}{3} =\frac{10.728}{3}=3.576. $$ These are well-defined finite-sample calculations. The second-moment theorem concerns how $m_{2,n}$ behaves as $n$ grows; the finite value $m_{3,3}=3.576$ does not create a finite population third moment.

More generally, if $X$ is nonnegative and $\mathbb{E}[X^k]=\infty$, the sample averages of $X_i^k$ do not converge in probability to a finite constant. One way to see the issue is to truncate: for any fixed cutoff $C$, the bounded variables $\min(X_i^k,C)$ have a sample average converging to $\mathbb{E}[\min(X^k,C)]$. As $C$ increases, these expectations increase without bound. Consequently, for any proposed finite bound, one can choose a truncation whose limiting average exceeds it. This is why a finite empirical moment in one sample cannot substitute for checking the corresponding population moment.

Worked Example: A Third Raw Moment and Its Units

Let hypothetical iid measurements $X$, in seconds, take the values $-1,0,2$ with probabilities $1/4,1/2,1/4$. A sample of four measurements is $-1,0,2,2$. Find the population and empirical third raw moments, and state the convergence conclusion.

Check the condition. The distribution is bounded, so $\mathbb{E}|X|^3<\infty$. More explicitly, $$ \mathbb{E}|X|^3=\frac14(1)+\frac12(0)+\frac14(8)=\frac94. $$ The measurements are assumed iid, as required by the result.

Calculate the population raw moment. For the signed third power, $$ \mu_3=\mathbb{E}[X^3] =\frac14(-1)^3+\frac12(0)^3+\frac14(2)^3 =-\frac14+0+2=\frac74=1.75 $$ cubic seconds. The negative observation contributes a negative term to the raw third moment; the absolute-moment check above is what establishes integrability.

Calculate the sample raw moment. For the given sample, $$ m_{3,4}=\frac{(-1)^3+0^3+2^3+2^3}{4} =\frac{15}{4}=3.75 $$ cubic seconds. The theorem says that $m_{3,n}\xrightarrow{p}1.75$ as $n$ increases under the stated iid distribution. It does not say that $m_{3,n}$ is centered around the sample mean; this is a raw moment, formed from powers of the original measurements.

Scope, Interpretation, and a Common Mistake

For each fixed order $k$, the result answers a specific question: does the average of $X_i^k$ approach a finite population quantity? The answer follows from the distribution of $X$, the iid sampling assumption, and the finite absolute $k$th moment. It is not a statement about the speed of convergence or the distribution of the estimation error; those are different questions.

A common mistake is to reason, “The variance of $X$ is finite, so all empirical raw moments converge.” That conclusion is too broad. Finite variance gives a finite second moment, but it does not ensure finite absolute moments of orders three and above. In the Pareto example, the second moment is finite while the third is infinite. The correction is to check $\mathbb{E}|X|^k<\infty$ for the particular order $k$ being averaged.

A second source of confusion is to call a centered quantity a raw moment. The empirical raw moment is $n^{-1}\sum_i X_i^k$; a centered sample quantity instead uses deviations such as $X_i-\bar{x}$. They are different expressions and should not be interchanged. First identify the quantity being averaged, then check its integrability and apply the weak law to that variable.

Key takeaway: Under iid sampling, $\mathbb{E}|X|^k<\infty$ is the finite-mean condition that lets the weak law deliver $m_{k,n}\xrightarrow{p}\mathbb{E}[X^k]$. Check the moment condition at the same order as the empirical power.

Check Your Understanding

Use the order of the moment and the stated distributional conditions to answer each question.

State the empirical raw moment of order $k$ for observations $X_1,\ldots,X_n$.
For iid $X_i$, what moment-existence condition supports convergence of $n^{-1}\sum_i X_i^4$ to a finite population fourth raw moment?
Why is $\mathbb{E}|X|^k<\infty$ the relevant condition for an odd-order moment rather than simply relying on cancellation of positive and negative values?
A variable has a finite second moment but an infinite third absolute moment. Which conclusion about empirical raw moments follows from the stated weak-law result, and which does not?
In what way does a third raw moment differ from a centered third sample quantity?

Convergence Of Empirical Raw Moments

What You'll Learn