The Weak Law Of Large Numbers With Finite Mean

Why Finite Mean Is Enough

The weak law of large numbers says that, under suitable assumptions, the sample average gets close in probability to the population mean as the sample size grows. The finite-variance version establishes this by bounding the sample average's variance. But some distributions have a finite mean and an infinite variance, so that direct variance argument cannot be used. This matters in econometrics when an outcome can occasionally take very large values: the mean may exist even when the variability is too great for a finite-variance calculation.

The key idea is to separate ordinary observations from the extreme tail. For any fixed cutoff, replace values beyond it by the cutoff. The resulting observations are bounded, so the finite-variance weak law applies to them. The expected size of the part removed can be made small when the original variable has a finite absolute mean. Taking the cutoff large and then the sample size large controls both sources of error.

Definition: A real random variable \(X\) has a finite mean here if it is integrable: \(E|X|<\infty\). Its mean \(\mu=E[X]\) is then a finite real number. This assumption does not require \(E[X^2]<\infty\), so the variance may be infinite.

The Weak Law With Finite Mean

Let \(X_1,X_2,\ldots\) be independent and identically distributed (iid) real random variables, each with \(E|X_i|<\infty\), and let \(\mu=E[X_i]\). Define their sample average by \(\bar X_n=n^{-1}\sum_{i=1}^n X_i\). Convergence in probability to \(\mu\) means that, for every fixed \(\varepsilon>0\), the probability that \(\bar X_n\) differs from \(\mu\) by more than \(\varepsilon\) tends to zero.

Theorem: If \(X_1,X_2,\ldots\) are iid and \(E|X_1|<\infty\), then

\bar X_n \xrightarrow{p} \mu=E[X_1].

In other words, for every \(\varepsilon>0\), \(P(|\bar X_n-\mu|>\varepsilon)\to 0\) as \(n\to\infty\). A finite variance is not required.

The assumptions matter. Identical distribution gives a common target mean, while independence permits a variance bound for the average of the truncated observations. Integrability controls the expected magnitude of the discarded tail. The theorem does not assert that the sample average has a finite variance.

Proof by Truncation

For a positive cutoff \(K\), truncate each observation by clipping it to the interval \([-K,K]\):

X_i^{(K)}=\max(-K,\min(X_i,K)).

For fixed \(K\), these truncated variables are iid and bounded, so they have finite variance. Write \(\mu_K=E[X_i^{(K)}]\). The sample average's error relative to \(\mu\) has three parts: the centered sampling error of the truncated observations, the average size of the removed tails, and the difference between the truncated and original means.

|\bar X_n-\mu| \leq \left|\frac{1}{n}\sum_{i=1}^n X_i^{(K)}-\mu_K\right| +\frac{1}{n}\sum_{i=1}^n |X_i-X_i^{(K)}| +|\mu_K-\mu|.

Make the mean bias small.
Since \(|\mu-\mu_K|\leq E|X_1-X_1^{(K)}|\), integrability implies that this difference tends to zero as \(K\) grows. The tail beyond larger cutoffs contributes less and less to \(E|X_1|\).

Control the average removed tail.
The terms \(|X_i-X_i^{(K)}|\) are nonnegative and have common expectation \(E|X_1-X_1^{(K)}|\). Markov's inequality bounds the probability that their average exceeds a fixed threshold.

Control the truncated sampling error.
For fixed \(K\), the truncated variables have finite variance. The finite-variance weak law therefore makes their centered sample average converge in probability to zero as \(n\) grows.

Here is the probability argument that completes the proof. Fix \(\varepsilon>0\), and let \(\eta>0\) be arbitrary. Choose \(K\) large enough that \(E|X_1-X_1^{(K)}|<\min(\varepsilon/3,\eta\varepsilon/3)\). Then the bias is less than \(\varepsilon/3\), and Markov's inequality gives

P\left(\frac{1}{n}\sum_{i=1}^n|X_i-X_i^{(K)}|>\frac{\varepsilon}{3}\right) \leq \frac{3E|X_1-X_1^{(K)}|}{\varepsilon}<\eta.

For this fixed \(K\), the probability that the first term in the decomposition exceeds \(\varepsilon/3\) tends to zero by the finite-variance weak law. If the total error exceeds \(\varepsilon\), then at least one of the first two terms exceeds \(\varepsilon/3\), because the bias is already less than \(\varepsilon/3\). Thus the limit superior of \(P(|\bar X_n-\mu|>\varepsilon)\) is at most \(\eta\). Since \(\eta\) can be any positive number, that limit is zero. The order of choices is essential: choose the cutoff first, then let the sample size grow.

Worked Examples

Worked Example: A Finite-Mean Pareto Distribution

Suppose iid observations have a Pareto distribution with lower bound 1 and tail probability \(P(X>x)=x^{-3/2}\) for \(x\geq1\). This is a hypothetical heavy-tailed model. Does its mean exist, and can the finite-mean weak law apply?

For a nonnegative random variable, the tail-integral formula gives \(E[X]=\int_0^\infty P(X>t)\,dt\). In this case,

E[X]=\int_0^1 1\,dt+\int_1^\infty t^{-3/2}\,dt =1+2=3.

The mean is finite. But the corresponding second-moment integral is

E[X^2]=\int_0^\infty 2tP(X>t)\,dt =1+2\int_1^\infty t^{-1/2}\,dt=\infty.

So this distribution has infinite variance, and a direct application of a finite-variance variance bound to \(X_i\) is unavailable. It does satisfy the assumptions of the finite-mean weak law: the observations are iid and \(E|X|=E[X]=3<\infty\). Therefore \(\bar X_n\xrightarrow{p}3\). The result is convergence in probability, not a guarantee that any particular finite sample average will be close to 3.

Worked Example: A Numerical Truncation Bound

Consider the hypothetical distribution \(P(X=2^k)=3\cdot4^{-k}\) for integers \(k\geq1\). The probabilities sum to \(3\sum_{k=1}^{\infty}4^{-k}=1\), and

E[X]=3\sum_{k=1}^{\infty}2^{-k}=3, \qquad E[X^2]=\sum_{k=1}^{\infty}3\cdot4^{-k}4^k=\infty.

Thus the mean exists although the variance does not. Use cutoff \(K=2^{11}=2048\) and define \(Y=\min(X,K)\). The mean removed by truncation is

E[X-Y] =\sum_{k=12}^{\infty}3\cdot4^{-k}(2^k-2048) =2^{-10} =\frac{1}{1024} \approx 0.0009766.

For \(n\) iid observations, the bias \(|E[Y]-3|\) is \(1/1024\). Also, \(Y^2\leq2048X\), so \(\operatorname{Var}(Y)\leq E[Y^2]\leq2048E[X]=6144\). At \(n=120{,}000{,}000\), Chebyshev's inequality bounds the probability that the centered truncated average differs from \(E[Y]\) by more than \(0.1/3\):

P\left(\left|\frac{1}{n}\sum_{i=1}^nY_i-E[Y]\right|>\frac{0.1}{3}\right) \leq\frac{6144}{120{,}000{,}000(0.1/3)^2} =0.04608.

Markov's inequality bounds the probability that the average removed tail exceeds \(0.1/3\) by \(3(1/1024)/0.1\approx0.02930\). The bias is less than \(0.1/3\); consequently, an error \(|\bar X_n-3|>0.1\) requires at least one of those two random errors to exceed \(0.1/3\). Adding the bounds gives \(P(|\bar X_n-3|>0.1)\leq0.04608+0.02930=0.07538\). This is a conservative bound, not an estimate of the actual probability. It illustrates how truncation gives a valid argument even when the original variance is infinite.

Worked Example: Choosing a Cutoff for the Pareto Model

Return to the Pareto model in the first example. Let \(Y=\min(X,K)\) with \(K=1{,}000{,}000\). For this distribution, \(E[X-Y]=2/\sqrt K=0.002\), so \(E[Y]=2.998\). To obtain a simple variance bound, use the tail-integral formula for the second moment of \(Y\):

E[Y^2]=\int_0^K2tP(X>t)\,dt =1+2\int_1^K t^{-1/2}\,dt =4\sqrt K-3=3997.

It follows that \(\operatorname{Var}(Y)\leq3997\). For \(n=1{,}000{,}000\), Chebyshev's inequality gives

P\left(\left|\frac{1}{n}\sum_{i=1}^nY_i-E[Y]\right|>\frac{1}{3}\right) \leq\frac{3997}{1{,}000{,}000(1/3)^2} \approx0.03597.

The expected removed tail is \(0.002\), so Markov's inequality bounds the probability that its sample average exceeds \(1/3\) by \(3(0.002)=0.006\). The bias from replacing \(E[X]=3\) by \(E[Y]=2.998\) is \(0.002<1/3\). Hence \(P(|\bar X_n-3|>1)\leq0.03597+0.006=0.04197\). This deliberately simple bound uses a large cutoff to control tail error and then a large sample to control truncated sampling error. The theorem itself is asymptotic; it does not require this particular cutoff or numerical bound.

What the Result Does—and Does Not—Say

The finite-mean weak law is useful because it separates consistency of the sample average from the stronger moment condition needed for a finite-variance argument. Under iid sampling and a finite absolute mean, the sample average converges in probability to the population mean even when the variance is infinite. The proof does not claim a particular rate of convergence, a normal approximation, or a useful finite-sample bound without additional information about the distribution's tail.

A common mistake is to say, “The variance is infinite, so the law of large numbers cannot apply.” That conclusion confuses the assumptions of the finite-variance proof with the assumptions of the finite-mean theorem. The correction is to check \(E|X|<\infty\): if it holds, truncation supplies bounded variables for the sampling-error step and integrability controls the discarded tail. Conversely, merely having a finite-looking or informally calculated mean is not enough; the required condition is integrability.

Independence also cannot be dropped just because the marginal mean is finite. For example, let \(X_i=Z\) for every \(i\), where \(Z\) is a nondegenerate random variable with a finite mean. Each observation has the same finite-mean distribution, but \(\bar X_n=Z\) for every \(n\), so the sample average does not approach the constant \(E[Z]\) in probability unless \(Z\) is itself constant. The iid condition rules out this complete dependence.

Key takeaway: For iid observations, \(E|X_1|<\infty\) is enough for \(\bar X_n\xrightarrow{p}E[X_1]\). Truncate first, control the bounded sample average, then make the tail and mean bias small.

Check Your Understanding

Try these questions using the theorem and proof above.

Which assumption permits the mean \(\mu=E[X_1]\) to be treated as a finite real number in the theorem?
Why does a fixed truncation cutoff give the truncated observations finite variance?
In the proof, what controls the average size of the removed tails, and what controls the centered truncated sample average?
Can the theorem apply when \(E[X_1^2]=\infty\)? State the other relevant assumptions.
If \(X_i=Z\) for every \(i\), with \(Z\) nondegenerate and integrable, which iid assumption fails, and what is \(\bar X_n\)?