Almost-Sure Convergence Through Sample Paths

Introduction

A sequence of random variables is a sequence of functions on a probability space. For a fixed outcome $\omega$, those functions produce an ordinary numerical sequence $X_1(\omega),X_2(\omega),\ldots$, called its sample path. Almost-sure convergence asks whether these numerical sequences converge to the proposed limit for all outcomes except a set of probability zero.

The phrase “except a set of probability zero” matters: convergence is not required at every outcome. The sample-path formulation also makes the quantifiers precise. For almost every outcome, and for every positive tolerance, the path must eventually remain within that tolerance of its limit.

Definition Through Sample Paths

Let $X_1,X_2,\ldots$ and $X$ be real-valued random variables defined on the same probability space $(\Omega,\mathcal F,P)$. At an outcome $\omega\in\Omega$, the sequence $X_n(\omega)$ is the sample path of the sequence at $\omega$.

Definition: The sequence $X_n$ converges almost surely to $X$, written $X_n\xrightarrow{a.s.}X$, if $$ P\bigl(\{\omega\in\Omega:\lim_{n\to\infty}X_n(\omega)=X(\omega)\}\bigr)=1. $$ Equivalently, there is a measurable set $N$ with $P(N)=0$ such that for every $\omega\notin N$, the numerical sequence $X_n(\omega)$ converges to $X(\omega)$.

Convergence of a numerical sequence has an equivalent formulation in terms of error bounds. Thus the sample-path definition can also be written as

P\left(\left\{\omega:\text{for every }\varepsilon>0\text{ there exists }N(\omega,\varepsilon) \text{ such that }n\geq N(\omega,\varepsilon) \Longrightarrow |X_n(\omega)-X(\omega)|<\varepsilon\right\}\right)=1.

The index $N(\omega,\varepsilon)$ may depend on both the outcome and the tolerance. The definition does not require one fixed index to work for every outcome or every tolerance. It does require that, for each outcome outside the exceptional null set, the bound holds for all subsequent indices.

Why the Convergence Event Is Measurable

The definition refers to the event that a sequence of values converges to $X$. To assign that event a probability, it must be measurable. The countability of the positive integers lets us express the event using only countable unions and intersections of measurable events.

Theorem: For real-valued random variables $X_n$ and $X$, the event that $X_n(\omega)$ converges to $X(\omega)$ is measurable, and $$ \{\omega:X_n(\omega)\to X(\omega)\} = \bigcap_{k=1}^{\infty}\ \bigcup_{m=1}^{\infty}\ \bigcap_{n=m}^{\infty} \{\omega:|X_n(\omega)-X(\omega)|<1/k\}. $$

Proof. Fix an outcome $\omega$. If $X_n(\omega)\to X(\omega)$, then for each positive integer $k$, the error is eventually less than $1/k$. Therefore there is some $m$ such that $|X_n(\omega)-X(\omega)|<1/k$ for every $n\geq m$. This puts $\omega$ in the right-hand side.

Conversely, suppose $\omega$ belongs to the right-hand side. For each $k$, there is an $m$ such that all $n\geq m$ satisfy $|X_n(\omega)-X(\omega)|<1/k$. Given any $\varepsilon>0$, choose $k$ large enough that $1/k<\varepsilon$. The same eventual bound then gives $|X_n(\omega)-X(\omega)|<\varepsilon$. This is convergence to $X(\omega)$.

For each $n$ and $k$, the event $\{|X_n-X|<1/k\}$ is measurable because $X_n-X$ is a random variable. Countable intersections and unions of measurable events are measurable, so the displayed convergence event is measurable. $\square$

Worked Sample-Path Examples

Worked Example: A Deterministic Sequence on Every Sample Path

On any probability space, define $X_n(\omega)=1/n$ and $X(\omega)=0$ for every $\omega$. Verify almost-sure convergence directly from the eventual-bound definition.

Fix any outcome $\omega$ and any $\varepsilon>0$. Choose $N=\lfloor 1/\varepsilon\rfloor+1$. For every $n\geq N$, we have $n>1/\varepsilon$, and hence $$ |X_n(\omega)-X(\omega)|=\frac{1}{n}<\varepsilon. $$ This holds at every outcome, so the convergence event is all of $\Omega$, which has probability one. Thus $X_n\xrightarrow{a.s.}0$.

Worked Example: Powers of a Uniform Random Variable

Let $\Omega=[0,1]$ with the uniform probability distribution, let $U(\omega)=\omega$, and set $X_n=U^n$. Determine whether $X_n$ converges almost surely to zero.

For each $\omega\in[0,1)$, the powers $\omega^n$ tend to zero. If $\omega=0$, all the terms are zero. If $0<\omega<1$, then for any $\varepsilon>0$, choose $N$ large enough that $\omega^N<\varepsilon$. Since $\omega^n\leq\omega^N$ for $n\geq N$, the required eventual bound follows.

At $\omega=1$, however, $X_n(1)=1$ for every $n$, so the path does not converge to zero. The exceptional set is exactly $\{1\}$, which has uniform probability zero. Therefore $U^n\xrightarrow{a.s.}0$, even though convergence fails at one outcome.

Worked Example: A Sequence That Does Not Converge Almost Surely

On any probability space, define $X_n(\omega)=(-1)^n$ and propose $X(\omega)=0$. Check whether the sequence converges almost surely to zero.

At every outcome, the sequence alternates between $1$ and $-1$. Taking $\varepsilon=1$, no index $N$ can make $|X_n(\omega)|<1$ for all $n\geq N$, since the absolute error is always $1$. Thus the eventual-bound condition fails at every outcome; the convergence event has probability zero, not one. In fact, this sequence does not converge to any real limit on any sample path.

Consequences of Pathwise Convergence

On the event of probability one where convergence holds, ordinary facts about real sequences apply. One useful consequence is that an almost-sure limit is unique up to equality with probability one.

Theorem: If $X_n\xrightarrow{a.s.}X$ and $X_n\xrightarrow{a.s.}Y$, then $P(X=Y)=1$.

Proof. Let $A$ be the event on which $X_n(\omega)\to X(\omega)$, and $B$ the event on which $X_n(\omega)\to Y(\omega)$. Both events have probability one, so their intersection also has probability one: $$ P(A\cap B)=1-P(A^c\cup B^c)\geq 1-P(A^c)-P(B^c)=1. $$ For every $\omega\in A\cap B$, the same numerical sequence has limits $X(\omega)$ and $Y(\omega)$. A real sequence has at most one limit, so $X(\omega)=Y(\omega)$. Hence $P(X=Y)=1$. $\square$

Pathwise convergence also preserves sums and products. If $X_n\xrightarrow{a.s.}X$ and $Y_n\xrightarrow{a.s.}Y$, then

X_n+Y_n\xrightarrow{a.s.}X+Y \qquad\text{and}\qquad X_nY_n\xrightarrow{a.s.}XY.

To verify this, take the intersection of the two probability-one events on which the respective sequences converge. On that intersection, the ordinary limit rules give convergence of the sums. For the products, fix any outcome in that intersection. Since $X_n(\omega)\to X(\omega)$, the sequence $X_n(\omega)$ is eventually bounded. Then $$ |X_n(\omega)Y_n(\omega)-X(\omega)Y(\omega)| \leq |X_n(\omega)|\,|Y_n(\omega)-Y(\omega)| +|Y(\omega)|\,|X_n(\omega)-X(\omega)|. $$ The first term tends to zero because its first factor is eventually bounded and its second tends to zero. The second tends to zero because $Y(\omega)$ is finite and $X_n(\omega)\to X(\omega)$. Thus the products converge on an event of probability one, as claimed.

Common Pitfall: Requiring Convergence at Every Outcome

The assertion “$X_n$ converges almost surely” does not mean that every sample path converges. In the uniform example, the path at $\omega=1$ does not converge to zero, but the failure occurs only on a probability-zero set. The definition concerns the probability of the entire convergence event, not whether that event equals all of $\Omega$.

A related quantifier error is to choose the eventual index before specifying the outcome and tolerance. The definition allows $N(\omega,\varepsilon)$ to vary. Requiring a single index to work simultaneously for every path is a stronger condition and is not part of almost-sure convergence.

Key takeaway: Almost-sure convergence is ordinary numerical convergence along almost every sample path. Equivalently, outside a probability-zero exceptional set, every positive error tolerance is eventually satisfied and remains satisfied.

Check Your Understanding

Use the sample-path definition and the probability of the convergence event in your answers.

What are the roles of $\omega$ and $\varepsilon$ in the eventual-bound formulation?
For $X_n(\omega)=1/n$ and $X(\omega)=0$, give an index that works for a specified $\varepsilon>0$.
Why does $U^n\xrightarrow{a.s.}0$ for a uniform $U$ on $[0,1]$, even though the path at $U=1$ does not converge to zero?
For $X_n(\omega)=(-1)^n$, identify a tolerance that shows the sequence cannot converge to zero.
Why must two almost-sure limits of the same sequence agree with probability one?