Finite-Mean Strong Laws by Truncation

Why Truncation Gives a Strong Law

A finite variance makes many strong-law proofs relatively direct, but it is stronger than necessary for convergence of sample averages. The challenge is that a random variable can have a finite mean and an infinite variance, so a variance-based argument cannot be applied to the original observations. Truncation resolves this by replacing the $n$th observation with a bounded version whose bound grows with $n$.

The growing bound matters. A fixed truncation would permanently discard part of the distribution, while a level that increases with the sample index can eventually retain almost every observed value. The proof must show both that discarded observations occur only finitely often and that the centered truncated observations still have enough variance control for their averages to converge.

Theorem (IID strong law with a finite first absolute moment): Let $X_1,X_2,\ldots$ be independent and identically distributed random variables with $E[|X_1|]<\infty$. Then $E[X_1]$ is finite and $$ \frac{1}{n}\sum_{j=1}^{n}X_j\xrightarrow{a.s.}E[X_1]. $$

The assumption concerns the absolute first moment, not merely a formal signed expectation: $E[|X_1|]<\infty$ ensures that $E[X_1]$ is well defined and finite. No finite second moment is assumed.

Truncate at the Observation Index

Write $\mu=E[X_1]$, and for each $n\geq 1$ define $$ Y_n=X_n\mathbf{1}\{|X_n|\leq n\}. $$ The truncation is symmetric, so the definition applies whether observations are positive or negative. Each $Y_n$ is bounded in absolute value by $n$, and the $Y_n$ remain independent because each is a measurable function of a different independent $X_n$. They are generally not identically distributed, since the cutoff depends on $n$.

Worked Example: Bernoulli Observations

Suppose $X_1,X_2,\ldots$ are IID Bernoulli$(p)$, where $0\leq p\leq 1$. Each observation takes the values 0 and 1, so $E[|X_1|]=E[X_1]=p<\infty$. For every $n\geq 1$, $|X_n|\leq n$, and therefore $Y_n=X_n$: the truncation makes no change.

The theorem gives $$ \frac{1}{n}\sum_{j=1}^{n}X_j\xrightarrow{a.s.}p. $$ The left-hand side is the proportion of successes in the first $n$ trials. This simple case also illustrates that the theorem concerns averages of observations, whether those observations are themselves indicators or more general real-valued quantities.

Two Bounds That Make the Proof Work

First, the probability that the $n$th observation is changed by truncation is summable. Since the variables are identically distributed, $$ \sum_{n=1}^{\infty}P(|X_n|>n) =\sum_{n=1}^{\infty}P(|X_1|>n) =E\!\left[\sum_{n=1}^{\infty}\mathbf{1}\{|X_1|>n\}\right] \leq E[|X_1|]<\infty. $$ The equality interchanging the sum and expectation is justified by Tonelli’s theorem, because all summands are nonnegative. For any fixed $t\geq0$, the number of positive integers $n$ with $n<t$ is at most $t$, which gives the inequality.

Second, the total variance of the truncated observations, after weighting by $n^{-2}$, is finite. Again using Tonelli’s theorem, $$ \sum_{n=1}^{\infty}\frac{\operatorname{Var}(Y_n)}{n^2} \leq \sum_{n=1}^{\infty}\frac{E[Y_n^2]}{n^2} =E\!\left[|X_1|^2\sum_{n=1}^{\infty} \frac{\mathbf{1}\{|X_1|\leq n\}}{n^2}\right]. $$ To bound the expression inside the expectation, fix $t=|X_1|$. If $0\leq t<1$, then $t^2\sum_{n=1}^{\infty}n^{-2}\leq 2t^2\leq 2t$. If $t\geq1$, let $m=\lceil t\rceil$. Then $$ t^2\sum_{n=m}^{\infty}\frac{1}{n^2} \leq t^2\left(\frac{1}{m^2}+\frac{1}{m}\right) \leq \frac{2t^2}{m} \leq 2t. $$ Here the sum is bounded by its first term plus the integral of $x^{-2}$ from $m$ to infinity, and $m\geq t$. Thus in all cases the quantity is at most $2t$, giving $$ \sum_{n=1}^{\infty}\frac{\operatorname{Var}(Y_n)}{n^2} \leq 2E[|X_1|]<\infty. $$ This is where the first absolute moment is enough: the truncation converts the weighted second moments into a bound proportional to $|X_1|$.

Worked Example: Exponential Observations

Let $X_1,X_2,\ldots$ be IID exponential random variables with rate $\lambda>0$, so their density is $f(x)=\lambda e^{-\lambda x}$ for $x\geq0$. Direct integration gives $$ E[|X_1|]=E[X_1]=\int_0^\infty x\lambda e^{-\lambda x}\,dx =\frac{1}{\lambda}. $$ The finite-mean strong law therefore implies that the sample average converges almost surely to $1/\lambda$.

For this distribution, the truncation-error probabilities can also be checked directly: $$ \sum_{n=1}^{\infty}P(|X_n|>n) =\sum_{n=1}^{\infty}e^{-\lambda n} =\frac{e^{-\lambda}}{1-e^{-\lambda}}<\infty. $$ The truncation level $n$ thus produces only finitely many changes almost surely, in accordance with the general argument.

From the Bounds to Almost-Sure Convergence

We use the following consequence of Kolmogorov’s maximal inequality and Kronecker’s lemma. State it with its hypotheses, since independence and centering are essential to this step.

Summable-variance criterion: If $Z_1,Z_2,\ldots$ are independent, mean-zero random variables with finite variances and $$ \sum_{n=1}^{\infty}\frac{\operatorname{Var}(Z_n)}{n^2}<\infty, $$ then $\frac{1}{n}\sum_{j=1}^{n}Z_j\xrightarrow{a.s.}0$.

For completeness, let $W_n=Z_n/n$. These variables are independent and centered, and their variances have finite sum. Kolmogorov’s maximal inequality says that for independent, mean-zero, finite-variance variables $W_j$, and integers $b\geq a$, $$ P\!\left(\max_{a\leq k\leq b}\left|\sum_{j=a}^{k}W_j\right|>\varepsilon\right) \leq \frac{1}{\varepsilon^2}\sum_{j=a}^{b}\operatorname{Var}(W_j). $$ Choose increasing integers $a_r$ such that $\sum_{j=a_r}^{\infty}\operatorname{Var}(W_j)\leq 2^{-4r}$. Taking $b$ to infinity in the inequality, by continuity of probability for increasing events, gives $$ P\!\left(\sup_{k\geq a_r}\left|\sum_{j=a_r}^{k}W_j\right|>2^{-r}\right) \leq 2^{2r}2^{-4r}=2^{-2r}. $$ The right-hand sides are summable in $r$. By the Borel–Cantelli lemma, almost surely only finitely many of these events occur. For all sufficiently large $r$, every partial sum beginning at $a_r$ therefore has absolute value at most $2^{-r}$. If $q\geq p\geq a_r$, then $$ \left|\sum_{j=p}^{q}W_j\right| \leq \left|\sum_{j=a_r}^{q}W_j\right| +\left|\sum_{j=a_r}^{p-1}W_j\right| \leq 2^{1-r}. $$ Consequently, the series $\sum_{n=1}^{\infty}W_n=\sum_{n=1}^{\infty}Z_n/n$ is almost surely Cauchy and hence converges almost surely. Kronecker’s lemma then gives $n^{-1}\sum_{j=1}^{n}Z_j\to0$ on that event. This proves the criterion. $\square$

Apply the criterion with $Z_n=Y_n-E[Y_n]$. The variables $Z_n$ are independent and centered, and their variances equal $\operatorname{Var}(Y_n)$. The weighted variance bound above verifies the criterion’s summability assumption, so $$ \frac{1}{n}\sum_{j=1}^{n}\bigl(Y_j-E[Y_j]\bigr)\xrightarrow{a.s.}0. $$ Also, $E[Y_n]=E[X_1\mathbf{1}\{|X_1|\leq n\}]\to E[X_1]=\mu$ by dominated convergence: the truncated terms converge pointwise to $X_1$ and are bounded in absolute value by the integrable variable $|X_1|$. The Cesàro means of a convergent numerical sequence have the same limit, so $n^{-1}\sum_{j=1}^{n}E[Y_j]\to\mu$.

Finally, the first bound in this section and the first Borel–Cantelli lemma show that $P(|X_n|>n)$ occurs only finitely often almost surely. Thus, with probability one, there is a finite random index after which $X_n=Y_n$ for every $n$. On each such outcome, $\sum_{j=1}^{n}(X_j-Y_j)$ is eventually a fixed finite sum, and dividing it by $n$ makes it tend to zero. Combining this fact with the convergence of the truncated averages proves the theorem.

Worked Example: A Pareto Distribution With Infinite Variance

Let $X_1,X_2,\ldots$ be IID with density $f(x)=\frac{3}{2}x^{-5/2}$ for $x\geq1$, and density zero otherwise. Its first moment is $$ E[X_1]=\frac{3}{2}\int_1^\infty x^{-3/2}\,dx=3, $$ whereas $$ E[X_1^2]=\frac{3}{2}\int_1^\infty x^{-1/2}\,dx=\infty. $$ Thus the observations have finite mean but infinite variance.

For $n\geq1$, the truncated variable $Y_n=X_n\mathbf{1}\{X_n\leq n\}$ has $$ E[Y_n^2]=\frac{3}{2}\int_1^n x^{-1/2}\,dx=3(\sqrt{n}-1), $$ with $E[Y_1^2]=0$, as this formula also gives. Therefore $$ \sum_{n=1}^{\infty}\frac{\operatorname{Var}(Y_n)}{n^2} \leq 3\sum_{n=1}^{\infty}\frac{\sqrt{n}-1}{n^2} \leq 3\sum_{n=1}^{\infty}\frac{1}{n^{3/2}}<\infty. $$ Moreover, $P(X_n>n)=n^{-3/2}$ for $n\geq1$, so the truncation-error probabilities are summable as well. The theorem gives $\frac{1}{n}\sum_{j=1}^{n}X_j\xrightarrow{a.s.}3$, despite the infinite variance.

Common Pitfall and Why the Argument Matters

A tempting but invalid shortcut is to write $\sum_n\operatorname{Var}(X_n)/n^2<\infty$ and apply a variance-based strong-law criterion. Under the theorem’s assumptions, the variances $\operatorname{Var}(X_n)$ need not even be finite; in the Pareto example they are infinite. The correction is not to assume a second moment silently. Instead, truncate each observation, prove the weighted variance bound for those bounded variables using $E[|X_1|]<\infty$, and separately show that truncation changes only finitely many observations almost surely.

The proof’s two controls serve distinct purposes. Tail summability prevents infinitely many observations from being altered, while weighted variance summability controls the cumulative fluctuations of the centered truncated variables. Together they extend the IID strong law from finite-variance samples to the full finite-absolute-mean setting.

Check Your Understanding

Check that you can identify the role of each assumption and each step in the proof.

Why is $E[|X_1|]<\infty$ sufficient to make $E[X_1]$ finite, including when $X_1$ can take negative values?
For $Y_n=X_n\mathbf{1}\{|X_n|\leq n\}$, explain why the $Y_n$ are independent but need not be identically distributed.
Which theorem turns $\sum_n P(|X_n|>n)<\infty$ into the assertion that only finitely many truncation errors occur almost surely?
Why does the weighted truncated-variance bound require only a first absolute moment rather than a finite second moment?
In the Pareto example, verify from the density that the mean is $3$ and the second moment is infinite.