Almost-Sure Convergence Through Tail Suprema

Why Look at the Whole Tail?

Almost-sure convergence is a statement about what happens along almost every entire sample path: eventually, the errors remain small. A bound on the error at one index does not capture this eventual behavior. A more direct quantity is the largest error still present anywhere in the tail beginning at that index.

For a fixed tolerance, the event that this tail maximum exceeds the tolerance says that at least one sufficiently late error is too large. As the starting index moves forward, these events decrease. Their intersection records whether such errors occur arbitrarily far along the sequence. This interpretation turns almost-sure convergence into a statement about probabilities of decreasing events.

Tail Suprema and Exceedance Events

Let $X,X_1,X_2,\ldots$ be real-valued random variables on the same probability space. For an integer $n\geq 1$, define the tail supremum of the errors by

M_n=\sup_{k\geq n}|X_k-X|.

The supremum may be $+\infty$, so $M_n$ is an extended nonnegative random variable. It is measurable because, for every real $\varepsilon$,

\{M_n>\varepsilon\} = \bigcup_{k\geq n}\{|X_k-X|>\varepsilon\},

a countable union of measurable events. For each fixed $\varepsilon>0$, write

A_n(\varepsilon)=\{M_n>\varepsilon\}.

These events decrease with $n$: if the tail beginning at $n+1$ contains an error larger than $\varepsilon$, then the tail beginning at $n$ does as well. Thus $A_{n+1}(\varepsilon)\subseteq A_n(\varepsilon)$. Their intersection has a useful pathwise meaning:

\bigcap_{n=1}^{\infty} A_n(\varepsilon) = \{|X_k-X|>\varepsilon\text{ for infinitely many }k\}.

Indeed, belonging to every $A_n(\varepsilon)$ means that after every starting index $n$, there is some $k\geq n$ with error greater than $\varepsilon$. Equivalently, such errors occur arbitrarily far along the sequence.

The Tail-Supremum Criterion

Theorem: The sequence $X_n$ converges almost surely to $X$ if and only if, for every $\varepsilon>0$, $$ P\!\left(\sup_{k\geq n}|X_k-X|>\varepsilon\right)\longrightarrow 0 \quad\text{as }n\to\infty. $$

The forward direction follows from continuity of probability for decreasing events. The reverse direction uses rational tolerances to turn separate probability statements into one event of probability one.

Fix a tolerance.
For each $\varepsilon>0$, consider the decreasing events $A_n(\varepsilon)$.

Identify their intersection.
The intersection is the event that errors greater than $\varepsilon$ occur infinitely often.

Use continuity of probability.
For decreasing events, their probabilities converge to the probability of their intersection.

For the converse, use countably many tolerances.
It is enough to rule out infinitely many exceedances at every positive rational tolerance.

Proof of the Criterion

Suppose first that $X_n\to X$ almost surely. Fix $\varepsilon>0$. On every sample path where $X_n\to X$, there is an index after which $|X_k-X|<\varepsilon$ for all $k$. Therefore that path belongs to only finitely many of the events $A_n(\varepsilon)$, and in particular does not belong to their intersection. Hence

P\!\left(\bigcap_{n=1}^{\infty}A_n(\varepsilon)\right)=0.

Because the events decrease and $P(A_1(\varepsilon))\leq 1$, continuity of probability from above gives

\lim_{n\to\infty}P(A_n(\varepsilon)) = P\!\left(\bigcap_{n=1}^{\infty}A_n(\varepsilon)\right) =0.

This proves the required limit for every fixed $\varepsilon>0$.

Conversely, suppose the stated probability limit holds for every $\varepsilon>0$. For each positive rational number $q$, the events $A_n(q)$ decrease, so continuity from above yields

P\!\left(\bigcap_{n=1}^{\infty}A_n(q)\right) = \lim_{n\to\infty}P(A_n(q)) =0.

There are countably many positive rational numbers. Consequently, the event

N=\bigcup_{q\in\mathbb{Q},\,q>0}\ \bigcap_{n=1}^{\infty}A_n(q)

has probability zero. Take any sample point outside $N$. If its errors did not converge to zero, there would be a $\delta>0$ such that $|X_k-X|\geq\delta$ for infinitely many $k$. Choose a rational $q$ with $0<q<\delta$. Then $|X_k-X|>q$ for infinitely many $k$, so the sample point would belong to $\bigcap_n A_n(q)$, and hence to $N$. This contradicts the choice of sample point. Thus $X_n\to X$ outside a probability-zero event, proving almost-sure convergence. $\square$

Worked Examples

Worked Example: A Deterministic Sequence

Let $X=0$ and let $X_k=1/k$ for every sample point. Find the tail supremum and verify the criterion.

The errors decrease as $k$ increases, so the largest error in the tail beginning at $n$ is the first one:

\sup_{k\geq n}|X_k-X| = \sup_{k\geq n}\frac{1}{k} = \frac{1}{n}.

For any $\varepsilon>0$, the event that this tail supremum exceeds $\varepsilon$ is the whole sample space if $1/n>\varepsilon$, and is empty if $1/n\leq\varepsilon$. Since $1/n\leq\varepsilon$ for all sufficiently large $n$, its probability eventually equals zero. The criterion confirms that $X_n\to X$ almost surely.

Worked Example: One Random Scale Across the Sequence

Let $U\sim\operatorname{Uniform}(0,1)$, set $X=0$, and define $X_k=U/k$. Compute the tail-supremum probability for a fixed $\varepsilon>0$.

Since $U\geq0$, the errors decrease with $k$, giving

\sup_{k\geq n}|X_k-X|=\frac{U}{n}.

Therefore

P\!\left(\sup_{k\geq n}|X_k-X|>\varepsilon\right) = P(U>n\varepsilon) = \begin{cases} 1-n\varepsilon,&n\varepsilon<1,\\ 0,&n\varepsilon\geq1. \end{cases}

For fixed $\varepsilon>0$, the second case applies for all sufficiently large $n$, so the probability tends to zero. The criterion gives $X_n\to0$ almost surely. Directly, $0\leq U/n\leq1/n$ for every outcome, which also verifies the pathwise limit.

Worked Example: A Random but Finite Cutoff

Let $N$ take positive integer values with $P(N=m)=2^{-m}$ for $m=1,2,\ldots$. These probabilities sum to one. Set $X=0$ and $X_k=\mathbf{1}_{\{N\geq k\}}$, where $\mathbf{1}$ is the indicator function. Find the tail-supremum probability.

For every outcome, $N$ is finite and $X_k=0$ once $k>N$. In the tail beginning at $n$, an error of one occurs exactly when $N\geq n$. Thus

\sup_{k\geq n}|X_k-X|=\mathbf{1}_{\{N\geq n\}}, \qquad P(N\geq n)=\sum_{m=n}^{\infty}2^{-m}=2^{1-n}.

If $0<\varepsilon<1$, the exceedance probability is $2^{1-n}$, which tends to zero. If $\varepsilon\geq1$, the strict exceedance event is empty. The criterion holds for every positive tolerance, including the boundary case $\varepsilon=1$. The example shows how a random index after which all errors vanish can still produce nonzero tail probabilities at each finite $n$.

Worked Example: Persistent Exceedances

Let $X=0$, and let $X_k$ be independent Bernoulli random variables with $P(X_k=1)=P(X_k=0)=1/2$. For $\varepsilon=1/2$, calculate the probability of a tail exceedance.

The event $A_n(1/2)$ occurs if at least one $X_k$ equals one for some $k\geq n$. For a finite tail ending at $m\geq n$, independence gives

P(X_k=0\text{ for every }n\leq k\leq m) = \left(\frac12\right)^{m-n+1}.

Thus the probability of at least one one between $n$ and $m$ is $1-2^{-(m-n+1)}$. Letting $m\to\infty$, continuity of probability for the increasing finite-tail events gives $P(A_n(1/2))=1$ for every $n$. These probabilities do not tend to zero. In fact, with probability one, ones occur arbitrarily far along the sequence, so $X_k$ does not converge to zero almost surely.

Common Pitfall and Why the Criterion Matters

A common mistake is to check the tail-supremum condition at only one tolerance and conclude almost-sure convergence. That only rules out infinitely many exceedances at that particular size. A sequence might still fail to converge because its errors keep exceeding a smaller positive tolerance. The converse proof avoids this problem by checking every positive rational tolerance; countability lets the corresponding probability-zero events be combined without losing probability one.

It is also important to keep the strict inequality in the event straight. The condition uses $M_n>\varepsilon$, not $M_n\geq\varepsilon$. If the supremum equals $\varepsilon$ but is not attained, there need not be any index with error greater than $\varepsilon$. The union representation of $A_n(\varepsilon)$ is specifically for the strict exceedance event.

The tail supremum packages all errors from index $n$ onward into one quantity. Its exceedance probabilities therefore measure the chance that a fixed-size error remains somewhere in the future tail. The decreasing-event argument connects this finite-$n$ probability calculation to the pathwise statement that, eventually, errors of every positive size disappear.

Check Your Understanding

Use the definitions and the decreasing-event interpretation to answer the following.

For fixed $\varepsilon>0$, describe in words the event $\bigcap_{n=1}^{\infty}A_n(\varepsilon)$.
Why is $A_{n+1}(\varepsilon)\subseteq A_n(\varepsilon)$?
In the random-cutoff example, calculate $P(N\geq n)$ directly from the mass function $P(N=m)=2^{-m}$.
Why does the converse proof use positive rational tolerances rather than attempt to take a union over every positive real tolerance?
For independent Bernoulli$(1/2)$ variables with $X=0$, why is $P(A_n(1/2))=1$ for each fixed $n$?