Why the Normalization Depends on the Moment
The finite-mean strong law says that IID observations with a finite first absolute moment have sample averages converging almost surely to their mean. A stronger conclusion is available when observations have a finite pth absolute moment for some \(p>1\): after centering, their partial sums are small relative to \(n^{1/p}\). The Marcinkiewicz–Zygmund strong law gives this conclusion, including the endpoint \(p=1\), where it reduces to the finite-mean strong law.
The normalization matters. For \(p>1\), dividing the centered sum by \(n\) gives the usual sample-average convergence, but the theorem also controls the sum on the larger scale \(n^{1/p}\). Since \(1/p<1\), this is a more informative rate statement. The assumption \(p<2\) allows the proof to control a series of truncated variances even when the original observations need not have finite variance.
The centering condition is meaningful because \(E[|X_1|]<\infty\): for \(p\ge1\), finite pth absolute moment implies finite first absolute moment. If \(Y_k\) are IID with \(E[|Y_1|^p]<\infty\) and \(\mu=E[Y_1]\), apply the theorem to \(X_k=Y_k-\mu\). In particular, $$ n^{1-1/p}\left(\frac{1}{n}\sum_{k=1}^n Y_k-\mu\right)\xrightarrow{a.s.}0. $$ For \(p=1\), this is the ordinary almost-sure convergence of the sample average. For \(p>1\), it gives a finer normalization of its error.
Worked Example: A Centered Two-Point Distribution
Suppose \(X_k\) are IID with \(P(X_k=2/3)=1/3\) and \(P(X_k=-1/3)=2/3\). First verify that the observations are centered: $$ E[X_k]=\frac13\cdot\frac23+\frac23\cdot\left(-\frac13\right)=\frac29-\frac29=0. $$
For \(p=3/2\), the absolute moment is $$ E[|X_k|^{3/2}] =\frac13\left(\frac23\right)^{3/2} +\frac23\left(\frac13\right)^{3/2} =\frac{2\sqrt{2}}{9\sqrt{3}}+\frac{2}{9\sqrt{3}} =\frac{2(\sqrt{2}+1)}{9\sqrt{3}} \approx 0.3097. $$ The moment is finite, and \(1\le 3/2<2\), so the theorem applies: $$ n^{-2/3}\sum_{k=1}^n X_k\xrightarrow{a.s.}0. $$ Thus the centered sum is negligible compared with \(n^{2/3}\), almost surely. The conclusion does not say that the sum itself converges to zero.
Proof by Truncation
The original observations may have infinite variance, so a direct variance argument for their partial sums is not available. Instead, truncate the \(k\)th observation at a level that grows with \(k\). The truncation is chosen to make the observations agree with the originals eventually, while the variances of the centered, rescaled truncated observations have a finite sum.
We use two standard facts. The Kolmogorov convergence criterion states that if independent, mean-zero random variables \(Z_k\) satisfy \(\sum_k\operatorname{Var}(Z_k)<\infty\), then the series \(\sum_k Z_k\) converges almost surely. Kronecker’s lemma states that if \(b_k\) is positive, nondecreasing, and tends to infinity, and \(\sum_k u_k/b_k\) converges, then \(b_n^{-1}\sum_{k=1}^n u_k\to0\). Both will be applied with \(b_k=k^{1/p}\).
Set \(X_k'=X_k\mathbf{1}_{\{|X_k|\le k^{1/p}\}}\). The events where truncation changes an observation occur only finitely often almost surely.
Show that the variances of \((X_k'-E[X_k'])/k^{1/p}\) have a finite sum. The Kolmogorov convergence criterion and Kronecker’s lemma then control their partial sums.
Show that \(n^{-1/p}\sum_{k=1}^n E[X_k']\to0\), and combine this with the eventual agreement of truncated and original observations.
For the first step, the number of positive integers \(k\) for which \(|x|>k^{1/p}\) is at most \(|x|^p\). Consequently, by Tonelli’s theorem, $$ \sum_{k=1}^{\infty}P(|X_k|>k^{1/p}) =\sum_{k=1}^{\infty}P(|X_1|>k^{1/p}) =E\left[\sum_{k=1}^{\infty}\mathbf{1}_{\{|X_1|>k^{1/p}\}}\right] \le E[|X_1|^p]<\infty. $$ The first Borel–Cantelli lemma implies \(X_k'=X_k\) for all but finitely many \(k\), almost surely. Therefore \(n^{-1/p}\sum_{k=1}^n(X_k-X_k')\to0\) almost surely: after the last changed observation, the numerator is a fixed finite sum.
For the second step, the variables \(Z_k=(X_k'-E[X_k'])/k^{1/p}\) are independent and mean zero. Their variances satisfy $$ \sum_{k=1}^{\infty}\operatorname{Var}(Z_k) \le \sum_{k=1}^{\infty}\frac{E[X_1^2\mathbf{1}_{\{|X_1|\le k^{1/p}\}}]}{k^{2/p}} =E\left[|X_1|^2\sum_{\substack{k\ge1\\k\ge |X_1|^p}}k^{-2/p}\right]. $$ The equality follows from Tonelli’s theorem, since all terms are nonnegative. Put \(q=2/p>1\). If \(0\le |x|\le1\), then $$ |x|^2\sum_{k=1}^{\infty}k^{-q} \le \left(\sum_{k=1}^{\infty}k^{-q}\right)|x|^p, $$ because \(p\le2\). If \(|x|>1\), the integral bound for the tail of the convergent p-series gives $$ \sum_{k\ge |x|^p}k^{-q}\le C_p(|x|^p)^{1-q} $$ for a finite constant \(C_p\) depending only on \(p\). Multiplying by \(|x|^2\) bounds this expression by \(C_p|x|^p\), since \(2+p(1-q)=p\). Thus the displayed expectation is finite. By the Kolmogorov convergence criterion, \(\sum_k Z_k\) converges almost surely. Kronecker’s lemma, with \(b_k=k^{1/p}\), now yields $$ \frac{1}{n^{1/p}}\sum_{k=1}^n(X_k'-E[X_k'])\xrightarrow{a.s.}0. $$
It remains to show that the changing truncation levels do not introduce a significant deterministic centering term. Since \(E[X_1]=0\), $$ |E[X_k']|=\left|E[X_1\mathbf{1}_{\{|X_1|>k^{1/p}\}}]\right| \le E[|X_1|\mathbf{1}_{\{|X_1|>k^{1/p}\}}]. $$ For each \(n\), define $$ g_n(x)=\frac{|x|}{n^{1/p}}\sum_{k=1}^n\mathbf{1}_{\{|x|>k^{1/p}\}}. $$ For every fixed \(x\), the sum has only finitely many nonzero terms, so \(g_n(x)\to0\). Also, the number of terms is at most \(\min(n,|x|^p)\). If \(|x|^p\le n\), this gives \(g_n(x)\le |x|^{p+1}/n^{1/p}\le |x|^p\). If \(|x|^p>n\), it gives \(g_n(x)\le |x|n^{1-1/p}\le |x|^p\), using \(p\ge1\). The dominated convergence theorem applies with dominating random variable \(|X_1|^p\), and therefore $$ \frac{1}{n^{1/p}}\sum_{k=1}^n|E[X_k']| \le E[g_n(X_1)]\longrightarrow0. $$
Finally, decompose the original sum into the truncation difference, the centered truncated sum, and the centering correction: $$ \frac{1}{n^{1/p}}\sum_{k=1}^n X_k =\frac{1}{n^{1/p}}\sum_{k=1}^n(X_k-X_k') +\frac{1}{n^{1/p}}\sum_{k=1}^n(X_k'-E[X_k']) +\frac{1}{n^{1/p}}\sum_{k=1}^n E[X_k']. $$ Each term tends to zero almost surely, so the claimed strong law follows. \(\square\)
Worked Example: A Symmetric Heavy-Tailed Distribution
Let \(X_k\) be IID with density $$ f(x)=\frac{3}{4}|x|^{-5/2}\mathbf{1}_{\{|x|\ge1\}}. $$ This is a probability density because its integral over the two tails is \(2(3/4)\int_1^\infty x^{-5/2}\,dx=(3/2)(2/3)=1\). Symmetry and integrability of \(|X_k|\) imply \(E[X_k]=0\); integrability holds because the tail exponent \(3/2\) exceeds one.
Take \(p=6/5\). Direct integration gives $$ E[|X_k|^{6/5}] =2\cdot\frac34\int_1^\infty x^{6/5-5/2}\,dx =\frac32\int_1^\infty x^{-13/10}\,dx =\frac32\cdot\frac{10}{3}=5. $$ Thus the moment required by the theorem is finite, and \(1\le6/5<2\). It follows that $$ n^{-5/6}\sum_{k=1}^n X_k\xrightarrow{a.s.}0. $$ This example has no finite second moment: the integral for \(E[X_k^2]\) has an \(x^{-1/2}\) tail and diverges. The theorem still applies because it needs a finite pth moment for the chosen \(p<2\), not a finite variance.
Worked Example: The Endpoint \(p=1\)
Let \(X_k\) take the values \(1\) and \(-1\), each with probability \(1/2\). Then \(E[X_k]=0\), and \(E[|X_k|]=1<\infty\). The theorem at \(p=1\) gives $$ \frac{1}{n}\sum_{k=1}^n X_k\xrightarrow{a.s.}0. $$ For variables \(Y_k=X_k+4\), the mean is \(4\), and the same conclusion, applied to \(Y_k-4\), says \(n^{-1}\sum_{k=1}^nY_k\to4\) almost surely. The endpoint is therefore the ordinary finite-mean strong law, not a claim that the unnormalized sum converges.
Interpretation and Common Pitfalls
For a noncentered IID sample with finite pth absolute moment, the theorem states that the sample-average error is \(o(n^{1/p-1})\) almost surely. This is a rate statement, not a finite-sample error bound: it does not supply a constant or a probability guarantee at a specified \(n\). The proof also shows why the normalization depends on \(p\): the threshold \(k^{1/p}\) makes the truncation events summable, while the corresponding truncated variances divided by \(k^{2/p}\) are summable precisely in the range \(p<2\).
- Using \(n\) instead of \(n^{1/p}\) in the theorem. The conclusion concerns the centered partial sum divided by \(n^{1/p}\). For \(p>1\), dividing by \(n\) gives a weaker consequence, namely the ordinary law for sample averages.
- Assuming finite variance is necessary. It is not. The heavy-tailed example has infinite variance but satisfies the theorem for \(p=6/5\).
- Ignoring the moving truncation’s mean. The variables \(X_k'\) are generally not centered just because \(X_k\) is. The proof must show that \(n^{-1/p}\sum_{k\le n}E[X_k']\to0\); eventual agreement alone does not establish this.
- Applying the result outside its assumptions. The stated theorem requires IID observations, mean zero, a finite pth absolute moment, and \(1\le p<2\). A finite pth moment alone does not replace centering or independence.
Check Your Understanding
Answer each question using the hypotheses and normalization of the theorem.
- For \(p=4/3\), what power of \(n\) normalizes the centered partial sum in the Marcinkiewicz–Zygmund strong law?
- Why does \(E[|X_1|^p]<\infty\) imply that the centering \(E[X_1]\) exists when \(p\ge1\)?
- In the truncation proof, what does the first Borel–Cantelli lemma establish about the observations \(X_k\) and \(X_k'\)?
- Why is it necessary to control \(\sum_{k=1}^n E[X_k']\) in addition to the centered truncated sum?
- Can the theorem be applied to a centered IID distribution with infinite variance if some \(p\) between one and two has a finite absolute pth moment? Explain.