Fourth-Moment Proof of the Bounded IID Strong Law

Why a Fourth Moment Gives Almost-Sure Convergence

A law of large numbers for sample averages can assert convergence in probability or, more strongly, convergence almost surely. The distinction matters: almost-sure convergence says that, outside an event of probability zero, the averages along a single infinite sequence of observations eventually stay close to their target. For bounded IID observations, a direct fourth-moment calculation gives a concise proof of this stronger conclusion.

The key is the rate of the tail bound. A second-moment argument generally gives a deviation probability of order $1/n$, whose sum over $n$ need not be finite. The fourth-moment calculation below gives a bound of order $1/n^2$, which is summable. The first Borel–Cantelli lemma then controls the deviations that occur infinitely often.

Theorem: Let $X_1,X_2,\ldots$ be IID random variables with $P(|X_1|\le M)=1$ for some finite constant $M$. Write $\mu=E[X_1]$. Then $$ \frac{1}{n}\sum_{i=1}^n X_i \xrightarrow{a.s.} \mu. $$

Here, “bounded” means bounded almost surely by a fixed finite constant, and the same bound applies to every observation because the variables are identically distributed. The theorem does not require the observations to have a density or a particular parametric distribution.

The Fourth Moment of a Centered Sum

Center the observations by defining $Y_i=X_i-\mu$, and let $S_n=\sum_{i=1}^nY_i$. The variables $Y_i$ are independent, have mean zero, and satisfy $|Y_i|\le 2M$ almost surely. The next identity isolates why the fourth moment is useful.

Fourth-moment identity: If $Y_1,\ldots,Y_n$ are independent, mean-zero random variables with finite fourth moments, then $$ E\left[\left(\sum_{i=1}^n Y_i\right)^4\right] = \sum_{i=1}^n E[Y_i^4] + 6\sum_{1\le i<j\le n} E[Y_i^2]E[Y_j^2]. $$ In the IID case, this becomes $$ E[S_n^4]=nE[Y_1^4]+3n(n-1)\bigl(E[Y_1^2]\bigr)^2. $$

To verify the identity, expand the fourth power as a sum of terms $Y_iY_jY_kY_\ell$ over all ordered quadruples of indices. If an index appears exactly once, independence factors out $E[Y_r]=0$, so that term has expectation zero. The only surviving patterns are four equal indices and two pairs of equal indices.

The four-equal pattern contributes $\sum_i E[Y_i^4]$. For each unordered pair $i<j$, the product $Y_i^2Y_j^2$ occurs in $\binom{4}{2}=6$ orders, each with expectation $E[Y_i^2]E[Y_j^2]$ by independence. This proves the identity. In the IID case, there are $\binom{n}{2}$ pairs, so their contribution is $6\binom{n}{2}(E[Y_1^2])^2=3n(n-1)(E[Y_1^2])^2$.

For the bounded IID variables in the theorem, put $B=2M$. Then $|Y_i|\le B$ almost surely, so $E[Y_i^4]\le B^4$ and $E[Y_i^2]\le B^2$. The identity gives

E[S_n^4] \le nB^4+3n(n-1)B^4 =(3n^2-2n)B^4 \le 3n^2B^4.

Proof of the Strong Law

We use Markov’s inequality in the form $P(Z\ge a)\le E[Z]/a$ for a nonnegative random variable $Z$ and $a>0$. We also use the first Borel–Cantelli lemma: if a sequence of events $A_n$ satisfies $\sum_{n=1}^{\infty}P(A_n)<\infty$, then $P(A_n\ \text{infinitely often})=0$. The lemma requires no independence among the events.

Bound a fixed-size deviation.
For $\varepsilon>0$, apply Markov’s inequality to $S_n^4$, using the fourth-moment bound.

Sum the probabilities.
The resulting upper bound is a constant times $n^{-2}$, so the probabilities are summable.

Exclude infinitely many deviations.
Apply the first Borel–Cantelli lemma for each positive rational tolerance, then take a countable intersection.

For fixed $\varepsilon>0$, let $A_n(\varepsilon)=\{|S_n|>\varepsilon n\}$. Since $S_n^4$ is nonnegative, Markov’s inequality and the moment bound yield

P(A_n(\varepsilon)) =P(S_n^4>\varepsilon^4 n^4) \le \frac{E[S_n^4]}{\varepsilon^4n^4} \le \frac{3B^4}{\varepsilon^4n^2}.

The series $\sum_{n=1}^{\infty}n^{-2}$ is finite. Therefore $\sum_nP(A_n(\varepsilon))<\infty$, and the first Borel–Cantelli lemma implies that, with probability one, only finitely many events $A_n(\varepsilon)$ occur. Equivalently, for this fixed $\varepsilon$, almost surely $|S_n|/n\le\varepsilon$ for all sufficiently large $n$.

To obtain convergence to zero rather than a statement for just one tolerance, apply this result to $\varepsilon=1/k$ for each positive integer $k$. Each corresponding probability-one event has a probability-one intersection, because the intersection is countable. On that intersection, for every $k$, eventually $|S_n|/n\le1/k$. Given any $\delta>0$, choose $k$ with $1/k<\delta$; then eventually $|S_n|/n<\delta$. Thus $S_n/n\to0$ almost surely. Since

\frac{1}{n}\sum_{i=1}^n X_i-\mu=\frac{S_n}{n},

the sample average converges almost surely to $\mu$. $\square$

Worked Applications

Worked Example: Bernoulli Observations

Let $X_i$ be IID Bernoulli$(1/3)$: $P(X_i=1)=1/3$ and $P(X_i=0)=2/3$. The mean is $\mu=1/3$. For $Y_i=X_i-\mu$, the two possible values are $2/3$ and $-1/3$, so $|Y_i|\le1$.

Writing $p=1/3$ and $q=2/3$, direct calculation gives

E[Y_i^2]=pq=\frac{2}{9}, \qquad E[Y_i^4] =pq^4+qp^4 =pq(p^3+q^3) =\frac{2}{27}.

Here $p^3+q^3=1-3pq=1/3$. The fourth-moment identity therefore gives

E[S_n^4] =\frac{2n}{27} +3n(n-1)\frac{4}{81} =\frac{4n^2-2n}{27}.

For any $\varepsilon>0$, Markov’s inequality gives $P(|\bar X_n-1/3|>\varepsilon)\le(4n^2-2n)/(27\varepsilon^4n^4)\le4/(27\varepsilon^4n^2)$. These bounds are summable in $n$, so the proof establishes $\bar X_n\to1/3$ almost surely.

Worked Example: A Uniform Distribution on a Bounded Interval

Let $X_i$ be IID Uniform$(2,4)$. The mean is $\mu=3$, and $Y_i=X_i-3$ is Uniform$(-1,1)$. Thus $|Y_i|\le1$. Integrating its density, which is $1/2$ on $(-1,1)$, gives

E[Y_i^2]=\frac12\int_{-1}^1 y^2\,dy=\frac13, \qquad E[Y_i^4]=\frac12\int_{-1}^1 y^4\,dy=\frac15.

Consequently,

E[S_n^4] =\frac{n}{5}+3n(n-1)\frac{1}{9} =\frac{n}{5}+\frac{n(n-1)}{3} \le \frac{n^2}{3}.

The last inequality follows because $n/5+n(n-1)/3\le n^2/3$ is equivalent to $n/5\le n/3$. Therefore $P(|\bar X_n-3|>\varepsilon)\le1/(3\varepsilon^4n^2)$. Summability and Borel–Cantelli give $\bar X_n\to3$ almost surely.

Worked Example: Symmetric Two-Point Observations

Suppose $X_i$ takes the values $-2$ and $2$, each with probability $1/2$. Then $\mu=0$, $Y_i=X_i$, $E[Y_i^2]=4$, and $E[Y_i^4]=16$. Applying the identity,

E[S_n^4] =16n+3n(n-1)(4)^2 =48n^2-32n \le48n^2.

It follows that $P(|\bar X_n|>\varepsilon)\le48/(\varepsilon^4n^2)$. This upper bound may be larger than one for small $n$, but that does not invalidate it; its role is to give a summable bound as $n$ grows. The theorem concludes that $\bar X_n\to0$ almost surely.

Why the Fourth-Moment Step Matters

If one instead applies Markov’s inequality to $S_n^2$, then independence and centering give $E[S_n^2]=n\operatorname{Var}(X_1)$. For fixed $\varepsilon>0$, this only yields

P(|S_n|>\varepsilon n) \le\frac{\operatorname{Var}(X_1)}{\varepsilon^2n}.

The right-hand side is generally not summable over $n$. Borel–Cantelli cannot be applied using that bound to rule out infinitely many deviations. This does not show that the strong law is false; it shows that this particular second-moment argument does not provide the needed summability. The fourth-moment bound improves the rate to $n^{-2}$, which is sufficient.

Common pitfall: A sequence of deviation probabilities tending to zero does not by itself imply that deviations occur only finitely often almost surely. The first Borel–Cantelli lemma needs the probabilities to have a finite sum. Check summability, not merely convergence of the individual probabilities to zero.

Key takeaway: For bounded IID observations, centering and expanding the fourth power leaves only the all-equal and paired-index terms. This gives $E[S_n^4]=O(n^2)$, hence deviation probabilities of order $n^{-2}$; Borel–Cantelli then proves almost-sure convergence of the sample average.

Check Your Understanding

Use the moment identity and the summability argument to answer these questions.

Why do terms with an index appearing exactly once vanish in the fourth-moment expansion?
For IID centered variables with $|Y_i|\le B$, what upper bound does the identity give for $E[S_n^4]$?
Why does a deviation bound proportional to $1/n^2$ allow the first Borel–Cantelli lemma to be used?
Why is it necessary to apply the fixed-tolerance conclusion to a countable sequence of tolerances?
What does a second-moment bound proportional to $1/n$ fail to establish in this proof?