The Cauchy Criterion for Convergence in Probability

Why a Cauchy Criterion Matters

A convergence statement in probability names a random variable that serves as the limit. The Cauchy criterion asks a different question: can we tell whether a sequence settles down by comparing its terms with one another, without knowing the limit in advance? For real-valued random variables, the answer is yes. A sequence is Cauchy in probability exactly when it converges in probability to a finite random variable on the same probability space.

The key step is to extract a subsequence whose successive terms are close except on events with summable probabilities. Outside a null event, that subsequence is an ordinary Cauchy sequence of real numbers and therefore has a finite limit. The Cauchy condition then brings the full sequence close to that limit in probability.

Definition and Theorem

Definition: A sequence of real-valued random variables $X_1,X_2,\ldots$ is Cauchy in probability if, for every $\varepsilon>0$ and every $\eta>0$, there is an integer $N$ such that $$ P(|X_n-X_m|>\varepsilon)<\eta $$ whenever $n,m\geq N$. Equivalently, for every $\varepsilon>0$, $P(|X_n-X_m|>\varepsilon)$ tends to zero as both indices $n,m$ tend to infinity.

The two indices matter: once $N$ is chosen, the bound must hold for every pair of terms with indices at least $N$, not just for neighboring terms or for a particular pair.

Theorem (Cauchy criterion for convergence in probability): Let $X_1,X_2,\ldots$ be real-valued random variables on a probability space. The sequence is Cauchy in probability if and only if there is a finite real-valued random variable $X$ on that space such that $X_n\xrightarrow{p}X$.

Proof of the Cauchy Criterion

First suppose $X_n\xrightarrow{p}X$, where $X$ is finite almost surely. For any $\varepsilon>0$, the triangle inequality gives $$ \{|X_n-X_m|>\varepsilon\} \subseteq \{|X_n-X|>\varepsilon/2\}\cup\{|X_m-X|>\varepsilon/2\}. $$ Consequently, $$ P(|X_n-X_m|>\varepsilon) \leq P(|X_n-X|>\varepsilon/2)+P(|X_m-X|>\varepsilon/2). $$ Given $\eta>0$, convergence in probability provides an $N$ such that each term on the right is less than $\eta/2$ whenever its index is at least $N$. Thus the left side is less than $\eta$ for every $n,m\geq N$, which is the Cauchy condition.

For the converse, assume $(X_n)$ is Cauchy in probability. We construct a subsequence, obtain its almost-sure limit, and then show that the whole sequence converges in probability to that limit.

Choose a rapidly controlled subsequence.
For each integer $k\geq1$, the Cauchy condition gives an integer $N_k$ such that $P(|X_n-X_m|>2^{-k})<2^{-k}$ whenever $n,m\geq N_k$. Choose strictly increasing indices $n_k$ with $n_k\geq N_k$. Then both $n_k$ and $n_{k+1}$ are at least $N_k$, so $P(|X_{n_{k+1}}-X_{n_k}|>2^{-k})<2^{-k}$.

Show that the subsequence is almost surely Cauchy.
Define $D_k=\{|X_{n_{k+1}}-X_{n_k}|>2^{-k}\}$. Since $\sum_{k=1}^{\infty}P(D_k)\leq\sum_{k=1}^{\infty}2^{-k}<\infty$, the first Borel–Cantelli lemma implies that, with probability one, only finitely many $D_k$ occur. For every outcome with this property, there is a $K$ such that $|X_{n_{k+1}}-X_{n_k}|\leq2^{-k}$ for all $k\geq K$. If $p>q\geq K$, then $$ |X_{n_p}-X_{n_q}| \leq\sum_{k=q}^{p-1}|X_{n_{k+1}}-X_{n_k}| \leq\sum_{k=q}^{p-1}2^{-k} \leq 2^{1-q}. $$ The last bound tends to zero as $q$ increases, so the subsequence is Cauchy in $\mathbb R$ at that outcome.

Define a finite, measurable limit.
Completeness of $\mathbb R$ gives a finite limit for the subsequence on the probability-one event where it is Cauchy. The event on which a sequence of real-valued measurable functions converges is measurable; define $X$ to be that limit there, and set $X=0$ on its complement. This defines a measurable, finite random variable. In particular, $X_{n_k}\to X$ almost surely.

Upgrade from the subsequence to the full sequence.
Almost-sure convergence implies convergence in probability, so $X_{n_k}\xrightarrow{p}X$. To check this implication, fix $\delta>0$. The events $\bigcup_{k\geq j}\{|X_{n_k}-X|>\delta\}$ decrease, as $j$ increases, to a null event. Continuity of probability from above shows their probabilities tend to zero, and each event $\{|X_{n_j}-X|>\delta\}$ is contained in the corresponding union.

It remains to verify convergence in probability for every $X_n$, not merely the selected subsequence. Fix $\varepsilon>0$ and $\eta>0$. By the Cauchy condition, there is an $N$ such that $$ P(|X_n-X_m|>\varepsilon/2)<\eta/2 $$ for all $n,m\geq N$. Choose $k$ large enough that $n_k\geq N$ and $P(|X_{n_k}-X|>\varepsilon/2)<\eta/2$; this is possible because $X_{n_k}\xrightarrow{p}X$. For every $n\geq N$, the triangle inequality and the union bound now give $$ P(|X_n-X|>\varepsilon) \leq P(|X_n-X_{n_k}|>\varepsilon/2)+P(|X_{n_k}-X|>\varepsilon/2) <\eta. $$ This is precisely $X_n\xrightarrow{p}X$, completing the converse and the proof. $\square$

Worked Examples

Worked Example: A Sequence with a Nonconstant Limit

Let $Z$ be Bernoulli with $P(Z=1)=1/3$, and let $W$ be an independent random variable with $P(W=-2)=P(W=1)=1/2$. Set $X_n=Z+W/n$. Show directly that the sequence is Cauchy in probability and identify its limit.

Since $|W|\leq2$, for every outcome and every $n,m$, $$ |X_n-X_m| =|W|\left|\frac{1}{n}-\frac{1}{m}\right| \leq 2\left|\frac{1}{n}-\frac{1}{m}\right| \leq \frac{2}{\min(n,m)}. $$ If $n,m\geq N$, this difference is at most $2/N$. Given $\varepsilon>0$, choose $N>2/\varepsilon$; then $|X_n-X_m|<\varepsilon$ for all outcomes and all $n,m\geq N$. Thus the probability of a difference greater than $\varepsilon$ is zero for those indices, so the sequence is Cauchy in probability.

Also, $|X_n-Z|=|W|/n\leq2/n$ for every outcome. For any $\varepsilon>0$, choosing $n>2/\varepsilon$ makes $|X_n-Z|<\varepsilon$ everywhere. Hence $X_n\xrightarrow{p}Z$, a finite, nonconstant limit.

Worked Example: Rare Spikes Are Cauchy in Probability

Let $U_2,U_3,\ldots$ be independent uniform random variables on $(0,1)$. Define $A_n=\{U_n\leq1/n\}$ and $X_n=n\mathbf{1}_{A_n}$ for $n\geq2$, where $\mathbf{1}_{A_n}$ is one on $A_n$ and zero otherwise. Show that $(X_n)$ is Cauchy in probability, even though its values do not settle to a finite limit almost surely.

For fixed $\varepsilon>0$, whenever $n>\varepsilon$, the spike height $n$ exceeds $\varepsilon$. Therefore $\{|X_n|>\varepsilon\}=A_n$, and $$ P(|X_n|>\varepsilon)=P(A_n)=\frac{1}{n}\longrightarrow0. $$ This proves $X_n\xrightarrow{p}0$, and the forward part of the criterion already implies that the sequence is Cauchy in probability.

There is also a direct pairwise check. If neither $A_n$ nor $A_m$ occurs, then $X_n=X_m=0$. Thus, if $|X_n-X_m|>\varepsilon$, at least one of $A_n,A_m$ must occur. For $n,m\geq2$, $$ P(|X_n-X_m|>\varepsilon) \leq P(A_n\cup A_m) \leq \frac{1}{n}+\frac{1}{m}. $$ For fixed $\varepsilon$, this bound tends to zero as both indices grow, verifying the Cauchy condition. It is only an upper bound: when both spikes occur, their difference is $|n-m|$, not necessarily either spike height.

On the other hand, the events $A_n$ are independent and $\sum_{n=2}^{\infty}P(A_n)=\sum_{n=2}^{\infty}1/n=\infty$. The second Borel–Cantelli lemma gives $P(A_n\text{ infinitely often})=1$. Along those infinitely many indices, $X_n=n$, so the sequence is unbounded almost surely and cannot converge almost surely to a finite value. This does not conflict with convergence in probability: rare spikes can have vanishing probability while still occurring infinitely often almost surely.

Worked Example: Adjacent Differences Do Not Establish the Criterion

Let $X_n$ be the constant random variable $H_n=\sum_{j=1}^{n}1/j$, the $n$th harmonic sum. Its adjacent differences satisfy $$ |X_{n+1}-X_n|=\frac{1}{n+1}\longrightarrow0. $$ Nevertheless, the sequence is not Cauchy in probability.

For any positive integer $m$, $$ |X_{2m}-X_m| =H_{2m}-H_m =\sum_{j=m+1}^{2m}\frac{1}{j} \geq \sum_{j=m+1}^{2m}\frac{1}{2m} =\frac{1}{2}. $$ The inequality holds because there are $m$ terms and each denominator is at most $2m$. Therefore $P(|X_{2m}-X_m|>1/3)=1$ for every $m$. Both indices can be arbitrarily large, so the Cauchy condition fails. Small differences between consecutive terms alone do not control the distance between all sufficiently late pairs.

Common Mistake and Practical Meaning

A common mistake is to replace the two-index Cauchy condition with the claim that $|X_{n+1}-X_n|\xrightarrow{p}0$. The harmonic-sum example shows why that is insufficient: many small consecutive changes can accumulate into a substantial distance. The criterion requires every late pair $X_n,X_m$ to be close with high probability.

The theorem is useful when a natural candidate limit is hard to identify, or when the sequence is built by approximation. Instead of guessing a limit, one may first bound $P(|X_n-X_m|>\varepsilon)$ and establish the Cauchy property. The theorem then guarantees a finite probability limit on the same probability space. Its proof also explains the role of the subsequence: the Cauchy condition controls pairs of terms, while a carefully selected subsequence turns those controls into summable error probabilities and an almost-surely convergent sequence.

Key takeaway: For real-valued random variables, convergence in probability is complete: being Cauchy in probability is exactly what guarantees a finite limit in probability. The Cauchy condition controls all sufficiently late pairs, not only consecutive terms.

Check Your Understanding

Use the quantifiers in the definition and the proof strategy to answer these questions.

State the Cauchy-in-probability condition, including the roles of $\varepsilon$, $\eta$, and $N$.
Why does $X_n\xrightarrow{p}X$ imply that $(X_n)$ is Cauchy in probability?
In the converse proof, why are the probabilities of the selected subsequence’s successive failures required to be summable?
For the rare-spike sequence, why does $|X_n-X_m|>\varepsilon$ imply $A_n\cup A_m$, even when both spikes can occur?
Why does $H_{n+1}-H_n\to0$ fail to show that the harmonic sums are Cauchy?