Two Ways to Describe a Random Sequence’s Limit
A sequence of random variables can get close to a target in more than one sense. One question concerns individual realizations: for almost every outcome, do the sequence’s values approach the target as the index grows? Another concerns average error: does the expected size of the discrepancy become small? These questions lead to almost-sure convergence and \(L^p\) convergence. They describe different features of a sequence, so it is useful to keep their definitions separate.
Throughout, let \(X_1,X_2,\ldots\) and \(X\) be random variables on the same probability space \((\Omega,\mathcal{F},P)\). Using one probability space lets us compare \(X_n(\omega)\) with \(X(\omega)\) for each outcome \(\omega\). The target \(X\) may itself be random; it need not be a constant.
The phrase “almost surely” allows an exceptional set of outcomes with probability zero. On all other outcomes, the sequence converges in the ordinary numerical sense. It does not mean that convergence occurs for every logically possible outcome.
The \(p\)th absolute moment responds both to how often errors occur and to how large they are when they occur. For \(p=1\), it is the expected absolute error. For \(p=2\), it is the expected squared error; taking the square root gives the root-mean-square error. A small expected error is not a claim that the error is small for every outcome.
A Sequence That Converges in Both Modes
Begin with a simple case in which both definitions can be checked directly. It shows how a sample-path statement and an expected-error calculation can describe the same sequence without being the same statement.
Worked Example: A Bernoulli Variable Divided by the Index
Let \(Z\) be a Bernoulli random variable with \(P(Z=1)=P(Z=0)=1/2\). Define \(X_n=Z/n\) and \(X=0\). Verify almost-sure and \(L^p\) convergence for any fixed \(p>0\).
Almost surely. If \(Z(\omega)=0\), then \(X_n(\omega)=0\) for every \(n\). If \(Z(\omega)=1\), then \(X_n(\omega)=1/n\to0\). Thus convergence to zero occurs for both possible values of \(Z\), so \(P(\lim_n X_n=X)=1\).
In \(L^p\). The error is \(|X_n-X|=Z/n\), and \(Z^p=Z\) because \(Z\) is either zero or one. Therefore $$ E[|X_n-X|^p]=E[Z/n^p]=\frac{E[Z]}{n^p} =\frac{1}{2n^p}\longrightarrow 0. $$ Thus \(X_n\) converges to zero in \(L^p\) for every fixed \(p>0\).
Here the sample-path values approach zero, and the expected \(p\)th-power error also vanishes. The next examples show why checking one of these features does not, by itself, settle the other.
Almost-Sure Convergence Without \(L^p\) Convergence
Almost-sure convergence says that, outside a probability-zero set, the errors eventually become smaller than any fixed positive tolerance. It does not place a limit on the expected size of an error at each index. A rare event can produce a very large error, even though each sample path experiences such an event at most finitely often.
Worked Example: Rare, Increasingly Large Errors
Take \(\Omega=[0,1]\) with the uniform probability distribution. Partition the interval into disjoint sets \(A_n\) with probabilities \(P(A_n)=1/[n(n+1)]\), for \(n=1,2,\ldots\). These probabilities sum to one because $$ \sum_{n=1}^{m}\frac{1}{n(n+1)} =\sum_{n=1}^{m}\left(\frac{1}{n}-\frac{1}{n+1}\right) =1-\frac{1}{m+1}\longrightarrow 1. $$ For a fixed \(p>0\), let \(X=0\) and \(X_n=n^{2/p}\mathbf{1}_{A_n}\), where \(\mathbf{1}_{A_n}\) is one on \(A_n\) and zero outside it.
Almost surely. The sets \(A_n\) are disjoint, so each outcome belongs to at most one of them. For any given outcome, \(X_n\) is therefore zero for all but at most one index. In particular, \(X_n(\omega)\to0\) for every outcome, and hence \(X_n\xrightarrow{\mathrm{a.s.}}0\).
In \(L^p\). Since \(\mathbf{1}_{A_n}^p=\mathbf{1}_{A_n}\), $$ E[|X_n-X|^p] =E[n^2\mathbf{1}_{A_n}] =n^2P(A_n) =\frac{n}{n+1}\longrightarrow 1. $$ The required moment does not tend to zero, so \(X_n\) does not converge to zero in \(L^p\). The rare errors do not occur repeatedly along any sample path, but their increasing size prevents the expected \(p\)th-power error from vanishing.
This construction works for every fixed \(p>0\): the amplitude \(n^{2/p}\) is chosen so that raising it to the power \(p\) gives \(n^2\). For the particular case \(p=1\), the sequence has values \(n^2\) on events of probability \(1/[n(n+1)]\), and its expected absolute error is \(n/(n+1)\).
\(L^p\) Convergence Without Almost-Sure Convergence
The reverse contrast is possible too. Expected error can tend to zero while deviations continue to occur infinitely often on almost every sample path. The following construction uses independent events; its calculation makes the distinction explicit.
Worked Example: Errors That Keep Reappearing
Let \(A_2,A_3,\ldots\) be independent events with \(P(A_n)=1/n\), and set \(X_n=\mathbf{1}_{A_n}\) and \(X=0\). For example, these events can be defined as independent coordinate events on a product probability space.
In \(L^p\). Because \(X_n\) takes only the values zero and one, \(|X_n|^p=\mathbf{1}_{A_n}\) for every \(p>0\). Hence $$ E[|X_n-X|^p]=P(A_n)=\frac{1}{n}\longrightarrow 0. $$ Thus \(X_n\) converges to zero in \(L^p\) for every \(p>0\).
Almost surely. For integers \(M\geq N\geq2\), independence gives $$ P\!\left(\bigcap_{n=N}^{M} A_n^c\right) =\prod_{n=N}^{M}\left(1-\frac{1}{n}\right) =\prod_{n=N}^{M}\frac{n-1}{n} =\frac{N-1}{M}. $$ As \(M\to\infty\), this probability tends to zero. Therefore the probability of avoiding every \(A_n\) from any fixed index \(N\) onward is zero. The event that only finitely many \(A_n\) occur is the union over \(N\geq2\) of these zero-probability events, so it also has probability zero. Consequently, \(A_n\) occurs infinitely often with probability one. On those outcomes \(X_n=1\) infinitely often, so \(X_n\) does not converge to zero almost surely.
The expected \(p\)th-power error gets small because the chance of an error at index \(n\) is \(1/n\). Yet these chances, together with independence, allow errors to recur indefinitely along almost every sample path.
How to Read the Two Definitions
The essential distinction is the object each mode examines. Almost-sure convergence concerns the limiting behavior of the entire sequence for individual outcomes, with probability-one coverage. \(L^p\) convergence concerns a sequence of numerical expectations, one for each index. Neither definition says that every error is small at every finite index.
| Mode | Quantity or event checked | What convergence requires |
|---|---|---|
| Almost sure | The sample-path limit \(X_n(\omega)\) | The paths converge to \(X(\omega)\) for a set of outcomes with probability one |
| \(L^p\) | The expected error \(E[|X_n-X|^p]\) | The expected \(p\)th-power error tends to zero |
A finite moment at each index is not enough for \(L^p\) convergence: the moments must tend to zero. Similarly, seeing errors become small on some outcomes does not establish almost-sure convergence; the definition concerns a set of outcomes whose probability is one. These checks should be performed using the relevant definition rather than inferred from a visual impression of a sequence.
Common Mistake: Treating Expected Error as a Pathwise Guarantee
A common error is to argue, “The probability of an error at index \(n\) goes to zero, so errors eventually stop almost surely.” The recurring-events example disproves this reasoning: \(P(A_n)=1/n\to0\), but \(A_n\) occurs infinitely often with probability one. A small probability at each separate index does not, by itself, say that the total sequence has only finitely many errors.
The correction is to check the requested mode directly. For \(L^p\), compute \(E[|X_n-X|^p]\) and show that it tends to zero. For almost-sure convergence, examine the limiting sample paths or the probability of deviations occurring infinitely often. The rare-large-error example warns against the reverse shortcut: eventual pathwise convergence does not ensure that the expected error tends to zero.
Check Your Understanding
For each question, identify whether it concerns sample-path behavior, expected error, or both.
- If \(X_n(\omega)\to X(\omega)\) for every outcome except those in a probability-zero set, which mode of convergence has been established?
- For \(X_n=Z/n\) in the Bernoulli example, calculate \(E[|X_n|^2]\).
- In the rare-large-error example, why does disjointness of the events imply almost-sure convergence to zero?
- For the independent-event example, why does \(E[|X_n|^p]\to0\) fail to establish almost-sure convergence?
- State the condition on \(E[|X_n-X|^p]\) required for convergence in \(L^p\).