Divergence of Averages With Infinite Mean

Why Infinite Means Change the Strong-Law Conclusion

The strong law for IID observations with a finite mean says that their averages converge almost surely to that mean. When the observations are nonnegative but have infinite expectation, there is no finite number for the averages to converge to. In fact, under these assumptions, the averages tend to infinity almost surely. The key is to apply the finite-mean strong law to bounded versions of the observations and then let the truncation level increase.

Definition: Let $X_1,X_2,\ldots$ be IID, finite-valued random variables with $P(X_1\geq 0)=1$, and suppose $E[X_1]=\infty$. Their sample average is $\bar X_n=n^{-1}\sum_{i=1}^n X_i$. The statement $\bar X_n\to\infty$ almost surely means that, with probability one, for every finite $A$, there is an index $N$ such that $\bar X_n>A$ for all $n\geq N$.

Nonnegativity matters: it lets us compare each observation with a bounded truncation from below. Infinite expectation alone does not supply such a comparison for signed observations. The result is also a statement about almost every infinite sample path, not just about the expectation of $\bar X_n$.

The Divergence Theorem

Theorem: If $X_1,X_2,\ldots$ are IID, finite-valued, nonnegative random variables and $E[X_1]=\infty$, then $$ \frac{1}{n}\sum_{i=1}^{n}X_i\xrightarrow{a.s.}\infty. $$

For a positive integer $m$, define $X_i^{(m)}=\min(X_i,m)$. For fixed $m$, these variables are IID and bounded, so the strong law for bounded IID observations gives $$ \frac{1}{n}\sum_{i=1}^{n}X_i^{(m)}\xrightarrow{a.s.}E[X_1^{(m)}]. $$ The expectations on the right become arbitrarily large as $m$ increases. The proof must handle all truncation levels on one probability-one event; doing so is possible because the positive integers form a countable set.

Apply the bounded strong law at each level.
For every positive integer $m$, the sample average of $X_i^{(m)}$ converges almost surely to $E[X_1^{(m)}]$.

Use one event for all levels.
Intersect the probability-one convergence events over the countably many positive integers $m$. The resulting event still has probability one.

Compare with the original observations.
Since $X_i\geq X_i^{(m)}$, every original sample average is at least its truncated counterpart. Arbitrarily large truncated limits force the original averages to diverge.

Here is the full justification. Let $A_m$ be the event that the truncated sample average converges to $E[X_1^{(m)}]$. The bounded strong law gives $P(A_m)=1$ for every $m$. Therefore the event $\Omega_0=\bigcap_{m=1}^{\infty}A_m$ has probability one, because $$ P(\Omega_0^c)=P\left(\bigcup_{m=1}^{\infty}A_m^c\right) \leq \sum_{m=1}^{\infty}P(A_m^c)=0. $$ For each outcome in $\Omega_0$, the truncated averages converge at every integer level $m$. Also, $X_1^{(m)}$ increases to $X_1$ as $m$ increases. The monotone convergence theorem, applied to this increasing sequence of nonnegative random variables, yields $$ E[X_1^{(m)}]\ \uparrow\ E[X_1]=\infty. $$ Now fix any outcome in $\Omega_0$ and any finite $A$. Choose an integer $m$ such that $E[X_1^{(m)}]>A$. Since the truncated averages converge to this expectation, they are eventually greater than $A$. For every $n$, $$ \bar X_n=\frac{1}{n}\sum_{i=1}^{n}X_i \geq \frac{1}{n}\sum_{i=1}^{n}X_i^{(m)}. $$ Thus $\bar X_n>A$ eventually. This holds for every finite $A$, on the same probability-one event, so $\bar X_n\to\infty$ almost surely. $\square$

A Useful Formula for Truncations

For nonnegative $X$ and $K>0$, truncated expectations can often be evaluated directly from the survival probabilities. The identity below is also useful for checking how quickly the truncation means grow.

Derived identity: For any nonnegative random variable $X$ and any $K>0$, $$ E[\min(X,K)]=\int_0^K P(X>t)\,dt. $$

For each outcome, $\min(X,K)=\int_0^K\mathbf{1}_{\{X>t\}}\,dt$; changing the indicator at the single endpoint $t=X$ does not change the integral. The function $(\omega,t)\mapsto\mathbf{1}_{\{X(\omega)>t\}}$ is measurable and nonnegative. Tonelli's theorem therefore permits exchanging expectation and integration: $$ E[\min(X,K)] =E\left[\int_0^K\mathbf{1}_{\{X>t\}}\,dt\right] =\int_0^K E[\mathbf{1}_{\{X>t\}}]\,dt =\int_0^K P(X>t)\,dt. $$ This formula does not require $E[X]$ to be finite, because the truncation is bounded and the integral is over a finite interval.

Worked Heavy-Tailed Examples

Worked Example: Pareto Tail With Exponent Below One

Suppose $X_1,X_2,\ldots$ are IID Pareto observations with $P(X_i>x)=x^{-3/4}$ for $x\geq1$, and $P(X_i>x)=1$ for $0\leq x<1$. The survival formula gives $$ E[X_i]=\int_0^\infty P(X_i>t)\,dt =1+\int_1^\infty t^{-3/4}\,dt =\infty, $$ because $\int_1^R t^{-3/4}\,dt=4(R^{1/4}-1)$ grows without bound as $R\to\infty$. Thus the theorem applies.

For a positive integer $m$, the truncated mean is $$ E[\min(X_i,m)] =1+\int_1^m t^{-3/4}\,dt =1+4(m^{1/4}-1) =4m^{1/4}-3. $$ These bounded means increase without bound. Consequently, for an IID sample from this distribution, $\bar X_n\to\infty$ almost surely.

Worked Example: The Borderline Pareto Exponent

Let the IID observations have Pareto tail $P(X_i>x)=x^{-1}$ for $x\geq1$, with $P(X_i>x)=1$ for $0\leq x<1$. Then $$ E[X_i]=1+\int_1^\infty \frac{1}{t}\,dt=\infty. $$ The divergence is logarithmic: for each integer $m\geq1$, $$ E[\min(X_i,m)] =1+\int_1^m \frac{1}{t}\,dt =1+\log m. $$ Although this truncated mean grows more slowly than the truncated mean in the preceding example, it still tends to infinity. The theorem therefore gives $\bar X_n\to\infty$ almost surely here as well. No finite-mean strong-law limit applies.

Worked Example: A Discrete Distribution With Infinite Mean

Let $X_i$ take positive integer values with probability mass function $$ P(X_i=k)=\frac{6}{\pi^2 k^2},\qquad k=1,2,\ldots. $$ These probabilities sum to one because $\sum_{k=1}^{\infty}k^{-2}=\pi^2/6$. But the mean is $$ E[X_i]=\sum_{k=1}^{\infty}k\frac{6}{\pi^2 k^2} =\frac{6}{\pi^2}\sum_{k=1}^{\infty}\frac{1}{k} =\infty. $$ For an integer truncation level $m$, the bounded truncated mean is $$ E[\min(X_i,m)] =\frac{6}{\pi^2}\left(\sum_{k=1}^{m}\frac{1}{k} +m\sum_{k=m+1}^{\infty}\frac{1}{k^2}\right). $$ The second term is nonnegative, while the harmonic sum in the first term diverges as $m\to\infty$. Hence these truncated means tend to infinity. For IID observations from this discrete distribution, the theorem again implies $\bar X_n\to\infty$ almost surely.

Interpretation and a Common Pitfall

The theorem says that almost every sufficiently long sample has an average exceeding any fixed finite threshold, and continues to do so. It does not give a rate: the truncation argument establishes divergence without specifying how quickly it occurs. In heavy-tailed models, very large observations can be rare, so the behavior seen in a finite sample need not make the infinite-mean character obvious.

Wrong reasoning: “The ordinary strong law says the average converges to $E[X_1]$, and here $E[X_1]=\infty$, so the conclusion follows immediately.” The usual finite-mean strong law has a finite-expectation hypothesis; it cannot be invoked with an infinite limit.
Correction: Truncate each observation at a fixed finite level. The bounded strong law applies at that level. Then let the integer truncation level increase, using monotone convergence and a countable intersection of probability-one events.
Another mistake: Conclude that $\bar X_n\to\infty$ merely because $E[\bar X_n]=\infty$ for each $n$. An infinite expectation alone does not establish almost-sure behavior. The truncation comparison is what proves the pathwise conclusion here.

Key takeaway: For IID, nonnegative observations with infinite expectation, bounded truncations have strong-law limits that grow without bound. Since every original observation dominates its truncation, the original sample averages diverge to infinity almost surely.

Check Your Understanding

Use the assumptions and truncation argument from this tutorial to answer the following.

Why is it necessary to take an intersection over truncation levels when proving the theorem?
For a Pareto distribution with $P(X>x)=x^{-\alpha}$ for $x\geq1$, which values of $\alpha>0$ give infinite expectation?
Explain why $E[\bar X_n]=\infty$ for every $n$ is not, by itself, a proof of almost-sure divergence.
What role does nonnegativity play in comparing the original averages with truncated averages?
For the discrete distribution in the third example, identify the divergent series that makes $E[X_i]$ infinite.