Convergence Along Sample Paths
Convergence in probability asks whether large errors become unlikely. Almost-sure convergence asks a different question: for almost every outcome, does the realized sequence of values actually settle toward its proposed limit? The distinction is visible when we fix an outcome \(\omega\) and inspect the ordinary numerical sequence \(X_1(\omega),X_2(\omega),\ldots\). Almost-sure convergence requires that this sequence converge for every outcome outside one event of probability zero.
Let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space. Consider real-valued random variables \((X_n)_{n\ge1}\) and \(X\), all defined on this same space. The sequence is indexed by discrete time \(n\ge1\), and its state space is \(\mathbb{R}\). The shared probability space matters: the definition compares \(X_n(\omega)\) with \(X(\omega)\) for the same outcome \(\omega\).
The exceptional event \(N\) is allowed to contain outcomes where convergence fails. It must, however, have probability zero; failure on a positive-probability set cannot be dismissed. The definition does not require convergence on every outcome. It also does not say that convergence occurs after a fixed deterministic time: the point at which a path becomes close to its limit may depend on \(\omega\).
The event in the definition is measurable. In particular, for real-valued random variables, the event that \(X_n\) converges to \(X\) can be written
Each set inside the braces is measurable because \(X_n\) and \(X\) are random variables. Countable unions and intersections of measurable sets are measurable, so the displayed event belongs to \(\mathcal{F}\). The expression says that for every accuracy \(1/m\), the path stays within that accuracy of its limit from some index onward.
A Pathwise Test for Indicators
Suppose \(I_n\) is an indicator sequence: \(I_n=\mathbf{1}_{A_n}\), where each \(A_n\in\mathcal{F}\). The process \((I_n)_{n\ge1}\) has discrete time and state space \(\{0,1\}\). Its values along any fixed outcome can only be zero or one, a restriction that makes convergence especially easy to test.
Proof. First suppose a sequence \(a_n\in\{0,1\}\) converges to a real number \(a\). Convergence makes it a Cauchy sequence: there is an \(N\) such that \(|a_r-a_s|<1/2\) whenever \(r,s\ge N\). But two elements of \(\{0,1\}\) have difference either \(0\) or \(1\). Thus \(|a_r-a_s|<1/2\) forces \(a_r=a_s\), so all terms from \(N\) onward are equal. The sequence is eventually constant, and its limit is that eventual value, either \(0\) or \(1\). Conversely, if all terms from some index onward equal \(c\in\{0,1\}\), then \(a_n\to c\) by the definition of a real limit. Applying these two implications separately to each outcome shows that the indicator path converges exactly when it is eventually constant. It therefore converges almost surely exactly when the event of eventual constancy has probability one. \(\square\)
There is a useful special case when the events are nested. If \(A_{n+1}\subseteq A_n\) for every \(n\), then once an outcome leaves the events it never returns. In that case, the indicator sequence has a pathwise limit that can be identified directly.
Proof. Fix \(\omega\in\Omega\). If \(\omega\in\cap_{n\ge1}A_n\), then \(\omega\in A_n\) for every \(n\), so every indicator equals \(1\) and the limit is \(1\). If \(\omega\notin\cap_{n\ge1}A_n\), there is an index \(k\) with \(\omega\notin A_k\). For every \(n\ge k\), nesting gives \(A_n\subseteq A_k\), so \(\omega\notin A_n\). All indicators from index \(k\) onward equal \(0\), and the limit is \(0\). These cases exhaust the outcomes and give the claimed limit. \(\square\)
Worked Example: Indicators of Shrinking Events
Take \(\Omega=[0,1]\), \(\mathcal{F}\) the Borel sets, and \(\mathbb{P}\) the uniform probability measure. Let \(U(\omega)=\omega\), and define the discrete-time indicator process \(I_n=\mathbf{1}_{\{U\le 1/n\}}\), with state space \(\{0,1\}\). Determine whether \(I_n\) converges almost surely and find its limit.
The events \(A_n=\{U\le1/n\}\) are nested decreasing, since \(1/(n+1)<1/n\). Their intersection is \(\{U=0\}\): if \(U=0\), then \(U\le1/n\) for every \(n\); if \(U=u>0\), choose an integer \(n>1/u\), so \(1/n<u\) and \(u\notin A_n\). The nested-event result therefore gives the pathwise limit \(I_n\to\mathbf{1}_{\{U=0\}}\).
Under the uniform distribution, \(\mathbb{P}(U=0)=0\). Thus the limit equals \(0\) outside a probability-zero event, and \(I_n\to0\) almost surely. At the exceptional outcome \(U=0\), every term equals \(1\), so the path still converges there, but to \(1\), not \(0\). This does not affect convergence to \(0\) almost surely because that outcome has probability zero.
Check: For every \(u>0\), all integers \(n>1/u\) have \(I_n(u)=0\). At \(u=0\), \(I_n(0)=1\) for every \(n\). These pathwise values agree with the identified limits.
Random Walks and Path Increments
For a convergent numerical sequence, consecutive differences must tend to zero. This simple fact gives a direct test for many stochastic processes without requiring a formula for the distribution of their proposed limit.
Proof. The triangle inequality gives \(|x_{n+1}-x_n|\le |x_{n+1}-x|+|x_n-x|\). If \(x_n\to x\), each term on the right tends to zero as \(n\to\infty\). Their sum therefore tends to zero, proving \(x_{n+1}-x_n\to0\). \(\square\)
Worked Example: A Simple Symmetric Random Walk
Let \(\Omega=\{-1,1\}^{\mathbb{N}}\) with the product probability measure under which each coordinate is \(1\) or \(-1\) with probability \(1/2\). For \(\omega=(\eta_1(\omega),\eta_2(\omega),\ldots)\), define the discrete-time random walk on the state space \(\mathbb{Z}\) by \(S_0=0\) and \(S_n=\sum_{k=1}^{n}\eta_k\) for \(n\ge1\). Does \(S_n\) converge almost surely?
In this canonical construction, every outcome has \(\eta_k(\omega)\in\{-1,1\}\) for every \(k\). For every \(\omega\) and every \(n\ge0\), the increment is
If the path \(S_n(\omega)\) converged, the path-increment proposition would imply \(|S_{n+1}(\omega)-S_n(\omega)|\to0\). The increments instead have absolute value \(1\) at every index, so no path converges. Thus the event of convergence is empty and has probability \(0\).
Check: The conclusion follows for every outcome in this sample space, not merely with probability one. If instead increments were specified only to satisfy \(\mathbb{P}(|\eta_k|=1)=1\) separately, the same conclusion would hold almost surely: the event that any increment fails to have absolute value \(1\) is a countable union of probability-zero events, hence has probability zero. This distinction avoids asserting a pathwise identity on outcomes where the assumptions only guarantee it almost surely.
Worked Example: Alternating Indicator Paths
Let \(A\in\mathcal{F}\) have probability \(1/2\) on any probability space. Define \(I_n=\mathbf{1}_A\) when \(n\) is odd and \(I_n=\mathbf{1}_{A^c}\) when \(n\) is even. This is a discrete-time process with state space \(\{0,1\}\). Determine its pathwise and almost-sure convergence.
If \(\omega\in A\), then \(I_n(\omega)\) alternates between \(1\) at odd indices and \(0\) at even indices. If \(\omega\in A^c\), it alternates between \(0\) at odd indices and \(1\) at even indices. In either case, the path is not eventually constant. By the indicator theorem, neither path converges. Since these two cases cover \(\Omega\), the convergence event is empty, and the process does not converge almost surely.
Check: Every path has infinitely many zeros and infinitely many ones, so no tail of any path is constant. The test uses the realized indicator sequence, not only the probability of either value at an individual index.
A Common Error: Replacing Paths by Expectations
Expectation describes an average over outcomes; almost-sure convergence describes what happens along nearly every individual outcome. These are different statements, and a calculation of expectations alone does not settle pathwise convergence.
Common mistake. For the symmetric random walk above, \(\mathbb{E}[\eta_k]=\tfrac12(1)+\tfrac12(-1)=0\), so linearity gives \(\mathbb{E}[S_n]=\sum_{k=1}^n\mathbb{E}[\eta_k]=0\). It is incorrect to conclude from this calculation that \(S_n\to0\) almost surely. The expectation is zero at each fixed time, whereas every path has consecutive differences of absolute value \(1\). The increment test proves that no path converges at all.
Likewise, seeing an indicator equal to zero at many indices does not show that its path eventually stays at zero. For indicators, the precise question is whether the sequence is eventually constant, as proved above. A path that continues to alternate between zero and one does not converge, regardless of how its individual values are summarized.
Almost-sure convergence is useful when the eventual behavior of a realized sequence matters, rather than only its distribution at a fixed time. The pathwise tests in this tutorial are deliberately direct: for indicators, inspect eventual constancy; for partial sums, compare consecutive increments with the necessary condition that they approach zero. These tests can rule convergence in or out without confusing a statement about almost every outcome with a statement about averages.
Check Your Understanding
For each question, distinguish a statement about individual paths from one about probabilities or expectations.
- State the definition of \(X_n\to X\) almost surely using a probability-zero exceptional event.
- Prove that a convergent sequence with values in \(\{0,1\}\) must be eventually constant. Why must its limit be either \(0\) or \(1\)?
- For \(I_n=\mathbf{1}_{\{U\le1/n\}}\) with \(U\) uniform on \([0,1]\), find the limit at \(U=0\) and at each \(U>0\). What is the almost-sure limit?
- If a real sequence \(x_n\) converges, what must happen to \(x_{n+1}-x_n\)? Give the inequality that proves this.
- For the simple symmetric random walk, compute \(|S_{n+1}-S_n|\) and explain why the paths cannot converge.
- Why does \(\mathbb{E}[S_n]=0\) for each \(n\) not establish almost-sure convergence of the random walk?