Almost-Sure Convergence Implies Convergence in Probability

From Pathwise Convergence to Probabilities

Almost-sure convergence is a statement about sample paths: outside an event of probability zero, the sequence eventually stays within any fixed tolerance of its limit. Convergence in probability is instead a statement about the probability of being outside that tolerance at the current index. The implication between them becomes clear when we track not just a failure at one index, but any failure from a given index onward.

Let \(X_1,X_2,\ldots\) and \(X\) be real-valued random variables on the same probability space. For a fixed \(\varepsilon>0\), define the failure event at index \(n\) by

E_n(\varepsilon)=\{|X_n-X|>\varepsilon\}.

The event \(E_n(\varepsilon)\) says that the error exceeds the tolerance at index \(n\). These individual events need not be nested. Instead, for each \(m\), collect all failures at indices \(n\ge m\):

A_m(\varepsilon)=\bigcup_{n=m}^{\infty}E_n(\varepsilon).

Thus \(A_m(\varepsilon)\) is the event that at least one failure occurs at or after index \(m\). The events \(A_m(\varepsilon)\) are decreasing: \(A_{m+1}(\varepsilon)\subseteq A_m(\varepsilon)\). Their intersection is the event of failures at infinitely many indices:

\bigcap_{m=1}^{\infty}A_m(\varepsilon) = \bigcap_{m=1}^{\infty}\bigcup_{n=m}^{\infty}E_n(\varepsilon).

This construction converts a pathwise statement into a sequence of measurable events to which continuity of probability applies.

Continuity of Probability for Decreasing Events

Theorem: If \(A_1\supseteq A_2\supseteq\cdots\) are measurable events and \(P(A_1)<\infty\), then \(\displaystyle \lim_{m\to\infty}P(A_m)=P\!\left(\bigcap_{m=1}^{\infty}A_m\right)\). For probability measures, \(P(A_1)\le 1\), so the finiteness condition always holds.

Here is a proof from countable additivity. Write \(A=\bigcap_{m=1}^{\infty}A_m\), and for each \(j\ge1\) define \(D_j=A_j\setminus A_{j+1}\). The sets \(D_j\) are pairwise disjoint. For every \(m\),

A_m=A\ \cup\ \bigcup_{j=m}^{\infty}D_j,

where the union on the right is disjoint. Indeed, a point in \(A_m\) but not in \(A\) must leave the decreasing sequence at some first later index, and so belongs to exactly one of the sets \(D_j\) with \(j\ge m\). Countable additivity gives

P(A_m)=P(A)+\sum_{j=m}^{\infty}P(D_j).

The series \(\sum_{j=1}^{\infty}P(D_j)\) is finite, since its sum is at most \(P(A_1)\). Therefore its tails tend to zero as \(m\to\infty\), proving the stated limit. \(\square\)

The Implication

Theorem: If \(X_n\xrightarrow{a.s.}X\), then \(X_n\xrightarrow{p}X\). Equivalently, for every fixed \(\varepsilon>0\), \(\displaystyle P(|X_n-X|>\varepsilon)\to0\).

Proof. Fix \(\varepsilon>0\), and use the events \(E_n(\varepsilon)\) and \(A_m(\varepsilon)\) defined above. If \(X_n(\omega)\to X(\omega)\), then there is an index after which \(|X_n(\omega)-X(\omega)|\le\varepsilon\) for every \(n\). Consequently, that sample point belongs to only finitely many of the failure events \(E_n(\varepsilon)\).

Almost-sure convergence says that the set of sample points where \(X_n(\omega)\) does not converge to \(X(\omega)\) has probability zero. It follows that the event of infinitely many failures at this fixed tolerance also has probability zero:

P\!\left(\bigcap_{m=1}^{\infty}A_m(\varepsilon)\right)=0.

The events \(A_m(\varepsilon)\) decrease with \(m\), so continuity of probability for decreasing events yields \(P(A_m(\varepsilon))\to0\). Since \(E_m(\varepsilon)\subseteq A_m(\varepsilon)\),

0\le P(|X_m-X|>\varepsilon) =P(E_m(\varepsilon)) \le P(A_m(\varepsilon)) \longrightarrow 0.

This holds for every fixed \(\varepsilon>0\), which is precisely convergence in probability. \(\square\)

A Tail-Uniform Consequence

The proof gives more than a bound on the failure probability at one index. For a fixed tolerance, it shows that the probability of any failure after index \(m\) tends to zero:

P\!\left(\sup_{n\ge m}|X_n-X|>\varepsilon\right) = P\!\left(\bigcup_{n=m}^{\infty}\{|X_n-X|>\varepsilon\}\right) \longrightarrow 0.

The equality holds because the supremum exceeds \(\varepsilon\) exactly when at least one term exceeds it. The supremum is measurable: its event of exceeding \(\varepsilon\) is a countable union of measurable events. This tail-uniform statement is often a useful way to read the proof: after a sufficiently late index, the chance of even one large error anywhere in the remaining sequence is small.

Worked Example: A Normal Variable Scaled by the Index

Let \(Z\sim N(0,1)\), set \(X=0\), and define \(X_n=Z/n\). All terms depend on the same random variable \(Z\). For \(\varepsilon>0\), the failure event is

E_n(\varepsilon)=\{|Z|>n\varepsilon\}.

As \(n\) increases, the threshold \(n\varepsilon\) increases, so these failure events are decreasing. Their future union is therefore just the first event in that tail:

A_m(\varepsilon)=\bigcup_{n=m}^{\infty}\{|Z|>n\varepsilon\} =\{|Z|>m\varepsilon\}.

If \(\Phi\) denotes the standard normal distribution function, symmetry of the standard normal distribution gives

P(A_m(\varepsilon)) =P(|Z|>m\varepsilon) =2\{1-\Phi(m\varepsilon)\} \longrightarrow 0.

In particular, \(P(|X_n-X|>\varepsilon)=2\{1-\Phi(n\varepsilon)\}\to0\). The same calculation controls all failures from \(m\) onward, not just the failure at \(m\).

Worked Example: Indicators from a Uniform Random Variable

Let \(U\) be uniform on \((0,1)\), let \(X=0\), and put \(X_n=\mathbf{1}\{U\le1/n\}\). For every outcome with \(U>0\), there is an index after which \(1/n<U\), and then \(X_n=0\) for all later indices. Since \(P(U=0)=0\), this proves \(X_n\xrightarrow{a.s.}0\).

For \(0<\varepsilon<1\), \(X_n\) takes only the values zero and one, so exceeding \(\varepsilon\) is exactly the event \(X_n=1\):

E_n(\varepsilon)=\{U\le1/n\}, \qquad P(E_n(\varepsilon))=\frac1n.

The events decrease as \(n\) increases, and hence

A_m(\varepsilon)=\bigcup_{n=m}^{\infty}\{U\le1/n\} =\{U\le1/m\}, \qquad P(A_m(\varepsilon))=\frac1m\longrightarrow0.

For \(\varepsilon\ge1\), all the strict failure events are empty. This example also shows that convergence in probability does not require the failure probabilities to decrease at a particular rate: here they are \(1/n\).

Worked Example: A Geometric Cutoff

Let \(N\) have the geometric distribution on \(\{1,2,\ldots\}\), with \(P(N=k)=(1-r)r^{k-1}\) for a fixed \(0<r<1\). Define \(X=0\) and \(X_n=\mathbf{1}\{N\ge n\}\). The variable \(N\) is finite with probability one, since

P(N\ge n)=\sum_{k=n}^{\infty}(1-r)r^{k-1}=r^{n-1} \longrightarrow0.

For each finite value of \(N\), \(X_n=0\) whenever \(n>N\). Thus \(X_n\to0\) almost surely. When \(0<\varepsilon<1\), failure at index \(n\) means \(N\ge n\), and a failure at some index \(n\ge m\) means \(N\ge m\). Therefore

P(E_n(\varepsilon))=r^{n-1}, \qquad A_m(\varepsilon)=\{N\ge m\}, \qquad P(A_m(\varepsilon))=r^{m-1}\longrightarrow0.

For \(\varepsilon\ge1\), the failure events are empty. The exact tail probability verifies the theorem directly in this discrete model.

Common Pitfall

A tempting but invalid shortcut is to say that because \(X_n\to X\) almost surely, \(P(E_n(\varepsilon))\to0\) simply because each sample path eventually has no failures. The pathwise statement alone does not justify interchanging a pathwise limit with probability. The proof needs an event argument: almost-sure convergence makes the probability of infinitely many fixed-tolerance failures zero; continuity of probability then makes the probability of any failure after \(m\) tend to zero; and the single failure event at \(m\) is contained in that future-failure event.

It is also important not to replace the decreasing events \(A_m(\varepsilon)\) with the individual events \(E_m(\varepsilon)\) in the continuity step. Individual failure events need not be nested. The decreasing structure belongs to the unions over all future indices, which is why those unions are central to the argument.

Key takeaway: Fix a tolerance, collect every failure from index \(m\) onward, and let \(m\) increase. These future-failure events decrease to the event of infinitely many failures. Almost-sure convergence makes that limiting event null, and continuity of probability forces the tail probabilities—and therefore the individual failure probabilities—to vanish.

Check Your Understanding

Try these questions using the fixed-tolerance events and the decreasing-event argument.

For a fixed \(\varepsilon>0\), define \(E_n(\varepsilon)\) and \(A_m(\varepsilon)\). Why are the \(A_m(\varepsilon)\) decreasing even though the individual \(E_n(\varepsilon)\) need not be?
What event is represented by \(\bigcap_{m=1}^{\infty}A_m(\varepsilon)\)?
In the uniform example, why is \(A_m(\varepsilon)=\{U\le1/m\}\) when \(0<\varepsilon<1\)?
Which inclusion allows the probability of a single failure at index \(m\) to be bounded by the probability of a future failure?
Why does almost-sure convergence imply that the probability of infinitely many failures at a fixed tolerance is zero?