Truncation Proof of the Weak Law With Finite First Moments

Why Truncation Extends the Weak Law

Chebyshev’s weak law gives a direct proof of convergence for IID observations with finite variance. But a finite mean does not guarantee a finite variance: a distribution can have rare, very large observations whose squares are not integrable. The weak law still holds under the weaker assumption of a finite absolute first moment. Truncation proves this by controlling ordinary-sized observations with variance bounds and handling the rare large observations separately.

Let $X_1,X_2,\ldots$ be IID real-valued random variables, and write $\bar X_n=n^{-1}\sum_{i=1}^n X_i$. The relevant mean assumption is $E[|X_1|]<\infty$, which ensures that $\mu=E[X_1]$ exists and is finite. The goal is to prove $\bar X_n\xrightarrow{p}\mu$.

Definition: For a cutoff $K>0$, define the truncated observation $Y_i^{(K)}=X_i\mathbf{1}\{|X_i|\le K\}$ and the discarded part $R_i^{(K)}=X_i\mathbf{1}\{|X_i|>K\}$. Then $X_i=Y_i^{(K)}+R_i^{(K)}$, and $Y_i^{(K)}$ is bounded in absolute value by $K$. Define the tail first moment $t_K=E[|X_1|\mathbf{1}\{|X_1|>K\}]$.

Because $E[|X_1|]<\infty$, the dominated convergence theorem gives $t_K\to0$ as $K\to\infty$. For each fixed $K$, the truncated variables are IID, bounded, and therefore have finite variance. The proof uses that finite variance for the truncated average, not for the original observations.

A Bound for a Fixed Cutoff

The order of the argument matters: first hold the cutoff fixed while the sample size grows, and then increase the cutoff to make the discarded tails negligible. The following bound makes that separation explicit. Write $Y^{(K)}=X_1\mathbf{1}\{|X_1|\le K\}$, so its variance is finite.

Truncation bound: If $t_K<\varepsilon/4$, then for every $n\ge1$, $$ P\bigl(|\bar X_n-\mu|>\varepsilon\bigr) \le \frac{4\operatorname{Var}(Y^{(K)})}{n\varepsilon^2} +\frac{4t_K}{\varepsilon}. $$

To prove the bound, use the decompositions $\bar X_n=\bar Y_n^{(K)}+\bar R_n^{(K)}$ and $\mu=E[Y^{(K)}]+E[R_1^{(K)}]$. The triangle inequality gives $$ |\bar X_n-\mu| \le |\bar Y_n^{(K)}-E[Y^{(K)}]| +|\bar R_n^{(K)}-E[R_1^{(K)}]|. $$

If the left side exceeds $\varepsilon$, at least one term on the right exceeds $\varepsilon/2$. Since the $Y_i^{(K)}$ are IID, $\operatorname{Var}(\bar Y_n^{(K)})=\operatorname{Var}(Y^{(K)})/n$. Chebyshev’s inequality therefore bounds the probability that the first term exceeds $\varepsilon/2$ by $4\operatorname{Var}(Y^{(K)})/(n\varepsilon^2)$.

For the second term, the triangle inequality and $|E[R_1^{(K)}]|\le E[|R_1^{(K)}|]=t_K$ imply $$ |\bar R_n^{(K)}-E[R_1^{(K)}]| \le \frac{1}{n}\sum_{i=1}^n|R_i^{(K)}|+t_K. $$ If this quantity exceeds $\varepsilon/2$, then the average of the absolute discarded parts exceeds $\varepsilon/2-t_K>\varepsilon/4$. Markov’s inequality, and $E[|R_i^{(K)}|]=t_K$, bound the probability of that event by $4t_K/\varepsilon$. Adding the two probability bounds proves the stated inequality. $\square$

The IID Weak Law with a Finite First Moment

Theorem: If $X_1,X_2,\ldots$ are IID and $E[|X_1|]<\infty$, then $$ \bar X_n\xrightarrow{p}E[X_1]. $$

Proof. Fix $\varepsilon>0$. For every sufficiently large fixed $K$, $t_K<\varepsilon/4$, so the truncation bound applies. Keep this $K$ fixed and let $n\to\infty$. Its variance term tends to zero, giving $$ \limsup_{n\to\infty}P\bigl(|\bar X_n-\mu|>\varepsilon\bigr) \le \frac{4t_K}{\varepsilon}. $$ This inequality holds for every sufficiently large $K$. Letting $K\to\infty$ makes the right side tend to zero because $E[|X_1|]<\infty$ implies $t_K\to0$. The probabilities are nonnegative, so their limit is zero. This is convergence in probability to $\mu$. $\square$

Fix a cutoff.
For fixed $K$, use Chebyshev’s inequality on the average of the bounded $Y_i^{(K)}$.

Bound the discarded part.
Markov’s inequality controls its average absolute size by the tail first moment $t_K$.

Take the limits in order.
First let $n\to\infty$ with $K$ fixed; then let $K\to\infty$ to remove the tail bound.

Worked Applications

Worked Example: A Pareto Mean with Infinite Variance

Let $X_1,X_2,\ldots$ be IID with density $f(x)=\tfrac32x^{-5/2}$ for $x\ge1$, and zero otherwise. This is a Pareto distribution with finite mean but infinite variance. Calculate the mean and a truncation bound for $\varepsilon=1$ and $K=4096$.

The density integrates to one because $\int_1^\infty \tfrac32x^{-5/2}\,dx=1$. Its mean is $$ E[X_1]=\frac32\int_1^\infty x^{-3/2}\,dx=3. $$ Its second moment is infinite, since $\tfrac32\int_1^\infty x^{-1/2}\,dx=\infty$; thus a direct finite-variance argument is unavailable.

For $K\ge1$, the discarded first moment and the truncated second moment are $$ t_K=\frac32\int_K^\infty x^{-3/2}\,dx=3K^{-1/2}, \qquad E[(Y^{(K)})^2]=\frac32\int_1^K x^{-1/2}\,dx=3(\sqrt K-1). $$ Since variance is no greater than the second moment, at $K=4096$ we have $t_K=3/64=0.046875<1/4$ and $\operatorname{Var}(Y^{(K)})\le 3(64-1)=189$. The truncation bound gives $$ P(|\bar X_n-3|>1)\le \frac{756}{n}+0.1875. $$ This particular fixed-cutoff bound is not meant to be sharp. The theorem follows by making the tail term arbitrarily small with a larger cutoff, then taking $n$ large for that fixed cutoff.

Worked Example: A Symmetric Distribution with Infinite Variance

For each integer $k\ge1$, assign probability $c/(2k^{5/2})$ to each of $k$ and $-k$, where $c=1/\zeta(5/2)$ and $\zeta(s)=\sum_{k=1}^{\infty}k^{-s}$. The probabilities sum to $c\zeta(5/2)=1$. Verify that the weak law applies even though the variance is infinite.

By symmetry and absolute integrability, $E[X_1]=0$. The absolute first moment is $$ E[|X_1|] =\sum_{k=1}^{\infty} k\frac{c}{k^{5/2}} =c\sum_{k=1}^{\infty}k^{-3/2} =c\zeta(3/2)<\infty. $$ But $E[X_1^2]=c\sum_{k=1}^{\infty}k^{-1/2}=\infty$.

For an integer cutoff $K\ge1$, the discarded first moment satisfies $$ t_K=c\sum_{k>K}k^{-3/2} \le c\int_K^\infty x^{-3/2}\,dx =\frac{2c}{\sqrt K}. $$ The truncated mean is zero by symmetry, and $$ \operatorname{Var}(Y^{(K)})=E[(Y^{(K)})^2] =c\sum_{k=1}^K k^{-1/2} \le c\left(1+\int_1^K x^{-1/2}\,dx\right) =c(2\sqrt K-1). $$ For example, $K=10{,}000$ gives $t_K\le c/50<1/4$, since $c<1$. For $\varepsilon=1$, the truncation bound is therefore at most $4c(199)/n+4c/50$, which tends to $0.08c$ as $n\to\infty$. Increasing $K$ makes this limiting tail bound arbitrarily small. The theorem consequently gives $\bar X_n\xrightarrow{p}0$ despite the infinite variance.

Worked Example: An Exponential Distribution

Suppose $X_i$ are IID exponential random variables with rate $2$, so $f(x)=2e^{-2x}$ for $x\ge0$. Their mean is $1/2$. Calculate the bound for $\varepsilon=1$, $K=2$, and $n=100$.

The discarded first moment is $$ t_2=E[X_1\mathbf{1}\{X_1>2\}] =\int_2^\infty 2xe^{-2x}\,dx =\frac52e^{-4}\approx0.04579. $$ The truncated second moment is $$ E[(Y^{(2)})^2] =\int_0^2 2x^2e^{-2x}\,dx =\frac12-\frac{13}{2}e^{-4} \approx0.38095. $$ Thus $\operatorname{Var}(Y^{(2)})\le0.38095$, and $t_2<1/4$ as required. The bound is $$ P(|\bar X_{100}-1/2|>1) \le \frac{4(0.38095)}{100}+4(0.04579) \approx0.1984. $$ The calculation illustrates how the bounded-part variance and the discarded-tail expectation enter separately.

Common Pitfall and Why the Argument Matters

A common mistake is to apply Chebyshev’s inequality directly to $\bar X_n$ under only $E[|X_1|]<\infty$. That step requires $\operatorname{Var}(X_1)<\infty$, which is not guaranteed. The correction is to apply Chebyshev only to the bounded truncated observations, and to control the remainder using its expected absolute size.

Another mistake is to claim that the tail term is negligible for a fixed cutoff simply because $n$ grows. The bound $4t_K/\varepsilon$ does not depend on $n$; it is made small by increasing $K$. The variance term, in turn, tends to zero as $n$ grows with $K$ fixed. Keeping this order of limits explicit is the key to the proof.

The result shows why finite variance is sufficient for the IID weak law but not necessary. Rare extreme values may prevent variance-based arguments from working, while their contribution to the sample average can still be controlled when the absolute first moment is finite.

Check Your Understanding

Use the truncation argument to answer these questions.

Why is $Y_i^{(K)}$ guaranteed to have finite variance for a fixed finite $K$?
Which inequality controls the probability that the average absolute discarded part is large?
Why must the cutoff-tail term be made small by increasing $K$, rather than by increasing $n$?
For the Pareto density in the first worked example, what is the tail first moment $t_K$ as a function of $K$?
Can the theorem apply when the observations have infinite variance? Give the feature of the symmetric discrete example that permits it.