From Small Error Probabilities to Pathwise Convergence
Convergence in probability controls the chance of a large error at one index \(n\). Almost-sure convergence makes a stronger statement about an entire sequence: outside an event of probability zero, the errors eventually become smaller than every fixed positive tolerance. A useful way to guarantee this stronger conclusion is to require that the probabilities of errors are not merely small, but summable over the sequence.
The key mechanism is that events whose probabilities have a finite sum can occur only finitely often with probability one. Applying this fact at a countable collection of tolerances gives the required control over sample paths.
The requirement concerns each fixed \(\varepsilon\) separately. The sum may depend on \(\varepsilon\); it need not be bounded uniformly over all positive tolerances. The random variables also need not be independent.
The First Borel–Cantelli Lemma
For events \(A_1,A_2,\ldots\), the event that infinitely many of them occur is $$ \limsup_{n\to\infty}A_n = \bigcap_{m=1}^{\infty}\bigcup_{n=m}^{\infty}A_n. $$ A point \(\omega\) is in this event exactly when it belongs to \(A_n\) for infinitely many indices \(n\).
Proof. For each \(m\), let \(B_m=\bigcup_{n=m}^{\infty}A_n\). The union bound gives $$ P(B_m)\leq \sum_{n=m}^{\infty}P(A_n). $$ Because the series of event probabilities converges, its tails tend to zero, so \(P(B_m)\to 0\) as \(m\to\infty\). The events \(B_m\) decrease as \(m\) increases, and \(\limsup_{n\to\infty}A_n=\bigcap_{m=1}^{\infty}B_m\). By continuity of probability for a decreasing sequence of events, $$ P\left(\limsup_{n\to\infty}A_n\right) = \lim_{m\to\infty}P(B_m) = 0. $$ Thus, with probability one, only finitely many of the events \(A_n\) occur. \(\square\)
Summable Errors Imply Almost-Sure Convergence
Proof. For each positive integer \(k\), define the error event $$ A_{n,k}=\{|X_n-X|>1/k\}. $$ The hypothesis, applied with \(\varepsilon=1/k\), gives \(\sum_{n=1}^{\infty}P(A_{n,k})<\infty\). The first Borel–Cantelli lemma therefore implies $$ P(A_{n,k}\text{ occurs for infinitely many }n)=0 $$ for each fixed \(k\).
Let \(N_k\) be the event that \(A_{n,k}\) occurs infinitely often, and let \(N=\bigcup_{k=1}^{\infty}N_k\). Since this is a countable union of probability-zero events, \(P(N)=0\). If \(\omega\notin N\), then for every \(k\) there is an index \(n_k(\omega)\) such that $$ |X_n(\omega)-X(\omega)|\leq 1/k \quad\text{for every }n\geq n_k(\omega). $$ Now let \(\eta>0\). Choose a positive integer \(k\) with \(1/k<\eta\). For all \(n\geq n_k(\omega)\), the displayed bound implies \(|X_n(\omega)-X(\omega)|<\eta\). This holds for every \(\eta>0\) whenever \(\omega\notin N\), which is precisely \(X_n\xrightarrow{a.s.}X\). \(\square\)
The use of countably many tolerances is essential to this argument. Borel–Cantelli gives a probability-one statement separately for each chosen tolerance; taking a countable union of the exceptional events still leaves a probability-zero set. The tolerances \(1,1/2,1/3,\ldots\) are enough, because for any desired \(\eta>0\), some \(1/k\) is smaller than \(\eta\).
Worked Example: Rare Bernoulli Errors
Let \(B_n\) be Bernoulli random variables with \(P(B_n=1)=1/n^2\), and set \(X_n=B_n\) and \(X=0\). The variables may, for example, be defined on a product probability space; their independence is not needed for the argument.
If \(0<\varepsilon\leq 1\), then \(|X_n-X|>\varepsilon\) exactly when \(B_n=1\), so $$ P(|X_n-X|>\varepsilon)=\frac{1}{n^2}, \qquad \sum_{n=1}^{\infty}P(|X_n-X|>\varepsilon) = \sum_{n=1}^{\infty}\frac{1}{n^2} <\infty. $$ If \(\varepsilon>1\), the error event is empty, so its probabilities also sum to a finite value.
The theorem gives \(B_n\xrightarrow{a.s.}0\). In particular, with probability one, only finitely many \(B_n\) equal 1. A quantitative bound on late errors follows directly: for \(N\geq2\), $$ P(\text{some }B_n=1\text{ for }n\geq N) \leq\sum_{n=N}^{\infty}\frac{1}{n^2} \leq\int_{N-1}^{\infty}\frac{1}{x^2}\,dx =\frac{1}{N-1}. $$
Worked Example: Gaussian Errors Shrinking at Rate \(1/n\)
Let \(Z_n\sim N(0,1)\) for each \(n\), and define \(X_n=X+Z_n/n\) for a real-valued random variable \(X\) on the same probability space. No assumption about dependence between \(X\) and the \(Z_n\), or among the \(Z_n\), is needed. For a fixed \(\varepsilon>0\), $$ P(|X_n-X|>\varepsilon)=P(|Z_n|>n\varepsilon). $$
For \(Z\sim N(0,1)\), its moment generating function is \(E[e^{tZ}]=e^{t^2/2}\). For \(a>0\) and \(t>0\), Markov's inequality gives $$ P(Z>a)=P(e^{tZ}>e^{ta}) \leq e^{-ta}E[e^{tZ}] =e^{-ta+t^2/2}. $$ Taking \(t=a\) yields \(P(Z>a)\leq e^{-a^2/2}\). By symmetry, \(P(Z<-a)\leq e^{-a^2/2}\), and hence \(P(|Z|>a)\leq2e^{-a^2/2}\). Therefore $$ \sum_{n=1}^{\infty}P(|X_n-X|>\varepsilon) \leq 2\sum_{n=1}^{\infty}e^{-n^2\varepsilon^2/2} \leq 2\sum_{n=1}^{\infty}e^{-n\varepsilon^2/2} = \frac{2e^{-\varepsilon^2/2}}{1-e^{-\varepsilon^2/2}} <\infty. $$ The second inequality uses \(n^2\geq n\), and the final series is geometric with ratio \(e^{-\varepsilon^2/2}<1\). The theorem now gives \(X_n\xrightarrow{a.s.}X\).
Worked Example: A Shared Uniform Error
Let \(U\sim\operatorname{Uniform}(0,1)\), and define \(X_n=X+U/n\). For \(\varepsilon>0\), $$ P(|X_n-X|>\varepsilon)=P(U>n\varepsilon). $$ If \(n\varepsilon\geq1\), this probability is zero. If \(n\varepsilon<1\), the uniform distribution gives \(P(U>n\varepsilon)=1-n\varepsilon\). Thus only finitely many indices can have nonzero error probability for this fixed \(\varepsilon\), and $$ \sum_{n=1}^{\infty}P(|X_n-X|>\varepsilon) = \sum_{1\leq n<1/\varepsilon}(1-n\varepsilon) <\infty. $$ The result implies \(X_n\xrightarrow{a.s.}X\). Indeed, directly, \(0\leq U/n\leq1/n\), so \(U/n\to0\) for every outcome. All errors here use the same random variable \(U\); the conclusion does not depend on independence across \(n\).
Why Summability Matters
A convergent series of error probabilities controls the chance that any error above a fixed tolerance occurs at or after a late index. In fact, for every \(\varepsilon>0\) and \(N\geq1\), the union bound gives $$ P\left(\bigcup_{n=N}^{\infty}\{|X_n-X|>\varepsilon\}\right) \leq \sum_{n=N}^{\infty}P(|X_n-X|>\varepsilon). $$ When the series is finite, the right-hand side tends to zero as \(N\to\infty\). The first Borel–Cantelli lemma sharpens this tail control into the statement that, almost surely, errors above that tolerance occur only finitely many times. Countable tolerances then give almost-sure convergence.
Summability also implies that the individual error probabilities tend to zero: the terms of any convergent series of nonnegative numbers tend to zero. Thus the condition ensures convergence in probability as well, but the main conclusion here is stronger and concerns almost every sample path.
Common Mistakes
- Replacing summability with probabilities tending to zero. The fact that \(P(|X_n-X|>\varepsilon)\to0\) does not itself establish that the errors occur only finitely often. The Borel–Cantelli argument here uses the finite sum.
- Assuming independence is required. Independence is not a hypothesis of the first Borel–Cantelli lemma. The union bound and continuity of probability suffice.
- Checking only one tolerance. Showing eventual control at, say, \(\varepsilon=1\) does not prove convergence. Use a countable sequence tending to zero, such as \(1/k\).
- Taking an uncountable union of exceptional events. Probability-zero events can have an uncountable union with positive probability. The proof avoids this problem by using only the countably many tolerances \(1/k\).
Check Your Understanding
Try these questions using the definitions and arguments above.
- State the first Borel–Cantelli lemma and identify which part of its proof does not require independence.
- Why is it enough to apply the summability condition at tolerances \(1/k\), rather than at every positive real number separately in the proof?
- If \(P(|X_n-X|>\varepsilon)=3^{-n}\) for a fixed \(\varepsilon>0\), what does the first Borel–Cantelli lemma say about errors exceeding that tolerance?
- In the Gaussian example, why is \(\sum_{n=1}^{\infty}e^{-n\varepsilon^2/2}\) finite for each fixed \(\varepsilon>0\)?
- Which step would fail if one knew only that each error probability tends to zero, without knowing that their sum is finite?