The Epsilon-Event Definition of Convergence in Probability

Convergence as a Probability of Large Error

A sequence of random variables can fail to get numerically closer to its target on every outcome and still become close with high probability. The precise question is not whether each realized sequence settles down, but whether the chance of an error larger than any fixed tolerance eventually becomes small. Convergence in probability formalizes that statement.

Throughout, $X_1,X_2,\ldots$ and $X$ are real-valued random variables defined on the same probability space $(\Omega,\mathcal{F},P)$. Sharing a probability space makes the difference $X_n-X$ and its deviation event well-defined.

Definition: The sequence $X_n$ converges in probability to $X$, written $X_n\xrightarrow{p}X$, if for every $\varepsilon>0$, $$ \lim_{n\to\infty}P\bigl(|X_n-X|>\varepsilon\bigr)=0. $$ Equivalently, for every $\varepsilon>0$ and every $\eta>0$, there is an integer $N$ such that for all $n\geq N$, $$ P\bigl(|X_n-X|>\varepsilon\bigr)<\eta. $$

The event $\{|X_n-X|>\varepsilon\}$ records that the error at index $n$ exceeds the chosen tolerance. The quantifiers have distinct roles: first choose any positive tolerance $\varepsilon$; then, for that tolerance, the probability of exceeding it must tend to zero as $n$ increases. In the equivalent formulation, $\eta$ is an arbitrarily small upper bound on that probability, and $N$ may depend on both $\varepsilon$ and $\eta$.

The definition does not require a single $N$ to work for every tolerance. For example, it may take longer for the probability of an error exceeding $0.001$ to become small than for the probability of an error exceeding $1$. Nor does the definition say that $X_n=X$ with high probability exactly; it controls the probability of being farther than a positive distance from $X$.

Working Directly With the Definition

A reliable verification begins by writing the deviation event for a fixed $\varepsilon>0$, simplifying it, and then taking the probability limit. The order matters: the tolerance is fixed while $n$ varies. Here are two elementary calculations.

Worked Example: A Deterministic Sequence of Constants

Let $a_n=2+1/n$, viewed as a constant random variable on any probability space, and let $X=2$. Verify that $a_n\xrightarrow{p}X$.

Fix an arbitrary $\varepsilon>0$. Since $a_n$ is constant, $$ P(|a_n-X|>\varepsilon) = \begin{cases} 1, & 1/n>\varepsilon,\\ 0, & 1/n\leq\varepsilon. \end{cases} $$ For every $n\geq 1/\varepsilon$, the second case applies, so the probability is zero. Thus its limit is zero for every fixed $\varepsilon>0$, which is precisely convergence in probability.

More generally, for deterministic constants $a_n$ and $a$, the same calculation shows that $a_n\xrightarrow{p}a$ exactly when the ordinary numerical sequence $a_n$ tends to $a$. If the numerical distance eventually lies below $\varepsilon$, the deviation event is empty; if the distance does not tend to zero, some positive tolerance is exceeded infinitely often.

Worked Example: An Indicator of a Rare Event

Let $A_n$ be events satisfying $P(A_n)=1/n$, and define $X_n=\mathbf{1}_{A_n}$, where an indicator is one on its event and zero otherwise. Show that $X_n\xrightarrow{p}0$.

Fix $\varepsilon>0$. If $0<\varepsilon<1$, then $|X_n-0|>\varepsilon$ occurs exactly when $X_n=1$, so $$ P(|X_n|>\varepsilon)=P(A_n)=\frac{1}{n}\longrightarrow 0. $$ If $\varepsilon\geq 1$, then $|X_n|$ is never greater than $\varepsilon$, and the probability is zero for every $n$. Both cases give the required limit for every positive tolerance.

The indicator result has a useful exact form: for any sequence of events $A_n$, $\mathbf{1}_{A_n}\xrightarrow{p}0$ if and only if $P(A_n)\to0$. Indeed, at any fixed tolerance $0<\varepsilon<1$, the deviation probability equals $P(A_n)$. Conversely, when those event probabilities tend to zero, the deviation probability also tends to zero for tolerances below one and is identically zero for tolerances at least one.

The next example illustrates why the size of an error alone does not determine convergence in probability. Very large errors can be compatible with convergence when their occurrence probabilities vanish.

Worked Example: Growing Errors on Rare Events

Using events with $P(A_n)=1/n$, define $Y_n=n\mathbf{1}_{A_n}$ and take the target to be zero. Verify convergence in probability, even though the nonzero value grows.

Fix $\varepsilon>0$. When $n\leq\varepsilon$, $Y_n\leq\varepsilon$ everywhere, so $P(|Y_n|>\varepsilon)=0$. When $n>\varepsilon$, the inequality $|Y_n|>\varepsilon$ holds exactly on $A_n$, giving $$ P(|Y_n|>\varepsilon)=\frac{1}{n}. $$ For all sufficiently large $n$, therefore, the deviation probability is $1/n$, which tends to zero. Hence $Y_n\xrightarrow{p}0$. The magnitude $n$ of a nonzero value is not bounded, but the chance of seeing that value at index $n$ goes to zero.

Worked Example: A Sequence That Does Not Converge in Probability

Let $B_n$ be events with $P(B_n)=1/2$ for every $n$, and set $Z_n=\mathbf{1}_{B_n}$. Test whether $Z_n$ converges in probability to zero.

Choose the particular tolerance $\varepsilon=1/2$. Because $Z_n$ takes only the values zero and one, $$ P(|Z_n-0|>1/2)=P(B_n)=\frac{1}{2} $$ for every $n$. This probability does not tend to zero, so the definition fails. To disprove convergence, it is enough to find one positive tolerance for which the deviation probabilities do not vanish.

Two Consequences of the Definition

The deviation-event formulation supports useful conclusions without requiring moment calculations or assumptions about independence. First, two limits in probability for the same sequence cannot disagree except on an event of probability zero.

Theorem: If $X_n\xrightarrow{p}X$ and $X_n\xrightarrow{p}Y$, where $X_n,X,Y$ are real-valued random variables on the same probability space, then $P(X=Y)=1$.

Proof. Fix $\varepsilon>0$. By the triangle inequality, if $|X-Y|>2\varepsilon$, then at least one of $|X-X_n|>\varepsilon$ or $|X_n-Y|>\varepsilon$ must hold. Thus $$ \{|X-Y|>2\varepsilon\} \subseteq \{|X-X_n|>\varepsilon\}\cup\{|X_n-Y|>\varepsilon\}. $$ Taking probabilities and using the union bound gives $$ P(|X-Y|>2\varepsilon) \leq P(|X-X_n|>\varepsilon)+P(|X_n-Y|>\varepsilon). $$ Both terms on the right tend to zero by the two assumed convergences. The left side does not depend on $n$, so it must be zero. Taking $\varepsilon=1/(2m)$ for each positive integer $m$ shows $P(|X-Y|>1/m)=0$ for every $m$. Since $$ \{X\ne Y\}=\bigcup_{m=1}^{\infty}\{|X-Y|>1/m\}, $$ countable subadditivity implies $P(X\ne Y)=0$. Therefore $P(X=Y)=1$. $\square$

The conclusion is equality almost surely, not necessarily equality at every point of the sample space. Random variables that differ only on a probability-zero event represent the same limit for this definition.

Theorem: For any sequence of events $A_n$, the indicators $\mathbf{1}_{A_n}$ converge in probability to zero if and only if $P(A_n)\to0$.

Proof. If $P(A_n)\to0$, fix $\varepsilon>0$. For $0<\varepsilon<1$, the event $\{|\mathbf{1}_{A_n}|>\varepsilon\}$ is $A_n$, so its probability tends to zero. For $\varepsilon\geq1$, the event is empty because an indicator is never greater than one. This proves convergence in probability. Conversely, suppose $\mathbf{1}_{A_n}\xrightarrow{p}0$, and choose $\varepsilon=1/2$. Then $$ P(A_n)=P(|\mathbf{1}_{A_n}|>1/2)\longrightarrow0. $$ The two statements are equivalent. $\square$

Interpreting the Quantifiers and Avoiding Pitfalls

Convergence in probability concerns the distribution of the error $X_n-X$ at each index, not the pathwise behavior of every outcome. Each probability is evaluated at one $n$; the definition then asks whether those probabilities shrink as the index increases. It imposes no independence condition on the random variables, and the rare-event calculations above do not depend on how the events at different indices relate to one another.

A frequent mistake is to argue, “The error can be as large as $n$, so $X_n$ cannot converge in probability to zero.” That reasoning ignores how often the large error occurs. For $Y_n=n\mathbf{1}_{A_n}$, the error is indeed $n$ on $A_n$, but for any fixed tolerance its probability is eventually $1/n$, which vanishes. The definition weighs the event probability, not the largest possible error by itself.

The converse mistake is to check only one tolerance. For example, showing that $P(|X_n-X|>1)\to0$ does not by itself establish convergence in probability: the definition requires the same conclusion for every $\varepsilon>0$, including smaller tolerances. To disprove convergence, in contrast, one failing tolerance is sufficient.

Key takeaway: Fix any positive error tolerance, form the event that the error exceeds it, and ask whether that event's probability tends to zero. Large errors on increasingly rare events can still satisfy this definition.

Check Your Understanding

For each question, use the fixed-tolerance event in the definition.

For $X_n=5-2/n$ and $X=5$, calculate the deviation probability for a fixed $\varepsilon>0$ and determine whether convergence in probability holds.
If $P(A_n)=3/(n+3)$, does $\mathbf{1}_{A_n}$ converge in probability to zero? Justify using the event definition.
Let $W_n=n^2\mathbf{1}_{A_n}$, where $P(A_n)=1/n^3$. For a fixed $\varepsilon>0$, find the deviation probability for all sufficiently large $n$, and decide whether $W_n\xrightarrow{p}0$.
Why is it enough to find one positive tolerance that fails the required limit in order to disprove convergence in probability?
If a sequence converges in probability to both $X$ and $Y$, what does the uniqueness theorem conclude, and what does it not require?