Sequences of Random Variables and Three Notions of Limit

One Sequence, Three Questions

A sequence of random variables can approach a limit in more than one meaningful sense. The distinctions matter because a random variable has both a realized value $X_n(\omega)$ at each outcome $\omega$ and a probability distribution across outcomes. A statement about what happens along outcomes is not automatically the same as a statement about probabilities or distributions.

Unless stated otherwise, let $X_1,X_2,\ldots$ and $X$ be real-valued random variables on the same probability space $(\Omega,\mathcal F,P)$. Writing $X_n$ specifies the $n$th random variable; for a fixed outcome $\omega$, the values $X_1(\omega),X_2(\omega),\ldots$ form an ordinary numerical sequence. The three limits below ask different questions about that sequence of random variables.

Definition: The sequence $X_n$ converges almost surely to $X$, written $X_n\xrightarrow{a.s.}X$, if there is a measurable set $A\in\mathcal F$ with $P(A)=1$ such that $X_n(\omega)\to X(\omega)$ as an ordinary real sequence for every $\omega\in A$.

Definition: The sequence $X_n$ converges in probability to $X$, written $X_n\xrightarrow{p}X$, if for every $\varepsilon>0$, $$ P(|X_n-X|>\varepsilon)\longrightarrow 0. $$

Definition: The sequence $X_n$ converges in distribution to $X$, written $X_n\xrightarrow{d}X$, if $F_{X_n}(x)\to F_X(x)$ at every continuity point $x$ of $F_X$, where $F_Y(x)=P(Y\le x)$ is the cumulative distribution function of a real-valued random variable $Y$.

The first two definitions compare $X_n$ and $X$ on the same probability space. Almost-sure convergence asks whether the numerical values eventually approach the target for almost every outcome. Convergence in probability asks whether the probability of a discrepancy larger than any fixed positive tolerance vanishes. Convergence in distribution compares cumulative distribution functions; it concerns the laws of the variables, not their values paired outcome by outcome.

In particular, convergence in distribution is meaningful from marginal distributions alone: its definition uses $F_{X_n}$ and $F_X$, not a joint distribution for $(X_n,X)$. By contrast, the event $\{|X_n-X|>\varepsilon\}$ and the sample-path comparison both depend on how the variables are realized together.

Deterministic Sequences as a Baseline

A deterministic number $c_n$ can be regarded as a constant random variable, equal to $c_n$ at every outcome. In that case, the three notions reduce to ordinary numerical convergence. This baseline makes clear that the modes differ because random variables can vary across outcomes, not because the word “limit” has a different meaning for real numbers.

Theorem: Let $c_n,c\in\mathbb R$, and regard them as constant random variables. Then $c_n\to c$ as real numbers if and only if the corresponding random variables converge to $c$ almost surely, in probability, and in distribution.

Proof. For constant random variables, $P(|c_n-c|>\varepsilon)$ is either zero or one. Thus it tends to zero for every $\varepsilon>0$ exactly when, for each such $\varepsilon$, $|c_n-c|\le\varepsilon$ eventually; this is numerical convergence. The sample-path condition is also exactly numerical convergence, since $c_n$ has the same value at every outcome.

For distributional convergence, the limiting distribution is a point mass at $c$. Its CDF is zero for $x<c$ and one for $x\ge c$; its only discontinuity is at $c$. If $c_n\to c$, then for any continuity point $x<c$, eventually $c_n>x$, so $F_{c_n}(x)=0$. For any continuity point $x>c$, eventually $c_n\le x$, so $F_{c_n}(x)=1$. This proves convergence in distribution.

Conversely, suppose the point-mass distributions at $c_n$ converge in distribution to the point mass at $c$. Fix $\varepsilon>0$. At $x=c-\varepsilon$, the limiting CDF equals zero, so $1\{c_n\le c-\varepsilon\}\to0$; therefore eventually $c_n>c-\varepsilon$. At $x=c+\varepsilon$, the limiting CDF equals one, so $1\{c_n\le c+\varepsilon\}\to1$; therefore eventually $c_n\le c+\varepsilon$. Hence eventually $|c_n-c|\le\varepsilon$. Since this holds for every $\varepsilon>0$, $c_n\to c$. $\square$

Worked Example: A Deterministic Sequence in All Three Modes

Let $c_n=2+(-1)^n/n$, and let $X_n$ be the constant random variable $c_n$. Since $$ |c_n-2|=\frac{1}{n}\longrightarrow 0, $$ the ordinary sequence converges to $2$. For every outcome $\omega$, $X_n(\omega)=c_n\to2$, so $X_n\xrightarrow{a.s.}2$.

For any $\varepsilon>0$, choose an integer $N>1/\varepsilon$. If $n\ge N$, then $|X_n-2|=1/n<\varepsilon$ at every outcome. Consequently $P(|X_n-2|>\varepsilon)=0$ for all $n\ge N$, proving $X_n\xrightarrow{p}2$.

Finally, at every $x<2$, the CDF of the point mass at $2$ is zero; at every $x>2$, it is one. Because $c_n\to2$, for each fixed $x<2$ we eventually have $c_n>x$, and for each fixed $x>2$ we eventually have $c_n\le x$. Thus the CDFs converge at every continuity point of the limiting CDF, proving $X_n\xrightarrow{d}2$.

What Distributional Convergence Uses

The distributional definition has an immediate consequence: only the individual laws of the sequence and the target matter. The random variables need not be placed on the same probability space to compare their distributions. When they are on a common space, their dependence structure still does not enter the definition of convergence in distribution.

Theorem: Suppose $X_n$ and $Y_n$ are real-valued random variables, and $X$ and $Y$ are real-valued random variables, with $F_{X_n}=F_{Y_n}$ for every $n$ and $F_X=F_Y$. Then $X_n\xrightarrow{d}X$ if and only if $Y_n\xrightarrow{d}Y$.

Proof. The equality $F_X=F_Y$ means that the two limiting CDFs have the same continuity points. At each such point $x$, the assumed equality $F_{X_n}(x)=F_{Y_n}(x)$ holds for every $n$. Therefore $F_{X_n}(x)\to F_X(x)$ at every continuity point exactly when $F_{Y_n}(x)\to F_Y(x)$ at every continuity point. These are the two distributional convergence statements. $\square$

Worked Example: Shrinking a Normal Random Variable

Let $Z\sim N(0,1)$, and set $X_n=Z/n$. The same value of $Z$ is used for every $n$, so for each outcome with finite $Z(\omega)$, $$ X_n(\omega)=\frac{Z(\omega)}{n}\longrightarrow 0. $$ A real-valued random variable is finite with probability one, hence $X_n\xrightarrow{a.s.}0$.

For any $\varepsilon>0$, $$ P(|X_n|>\varepsilon)=P(|Z|>n\varepsilon). $$ As $n\to\infty$, the events $\{|Z|>n\varepsilon\}$ decrease to the empty event up to the probability-zero event where $Z$ is not finite. By continuity of probability for decreasing events, their probabilities tend to zero. Thus $X_n\xrightarrow{p}0$.

The CDF of $X_n$ is $F_{X_n}(x)=P(Z\le nx)=\Phi(nx)$, where $\Phi$ is the standard normal CDF. The limiting variable is the constant zero, whose CDF is continuous at every $x\ne0$. If $x<0$, then $nx\to-\infty$, so $\Phi(nx)\to0$; if $x>0$, then $nx\to+\infty$, so $\Phi(nx)\to1$. These are precisely the limiting CDF values at all its continuity points. Therefore $X_n\xrightarrow{d}0$.

Large Values Can Occur on Rare Events

Convergence in probability measures the chance of exceeding a fixed tolerance, not the size of a discrepancy once it occurs. The next example makes that distinction concrete. It also shows why checking a sequence only at its most conspicuous values can be misleading: those values may occur on events whose probabilities vanish.

Worked Example: A Rare Spike

Let $U$ be uniform on $(0,1)$, and define $X_n=n\,1\{U\le1/n\}$. Thus $X_n=n$ on an event of probability $1/n$, and $X_n=0$ otherwise. Take the proposed limit to be the constant zero.

For an outcome with $U=u>0$, choose an integer $N>1/u$. For every $n\ge N$, $1/n<u$, so $X_n(u)=0$. The exceptional outcome $U=0$ has probability zero. Therefore $X_n(\omega)\to0$ almost surely.

Fix $\varepsilon>0$ and take $n>\varepsilon$. On $\{U\le1/n\}$, the absolute error equals $n>\varepsilon$; off that event, it equals zero. It follows that $$ P(|X_n|>\varepsilon)=P(U\le1/n)=\frac{1}{n}\longrightarrow0. $$ Hence $X_n\xrightarrow{p}0$, even though the nonzero value $n$ grows without bound.

For distributional convergence, if $x<0$, then $F_{X_n}(x)=0$. If $x>0$, then for all sufficiently large $n$, $n>x$, and $$ F_{X_n}(x)=P(X_n\le x)=P(U>1/n)=1-\frac1n\longrightarrow1. $$ The point mass at zero has continuity points $x<0$ and $x>0$. At those points the calculated limits equal its CDF, so $X_n\xrightarrow{d}0$ as well.

Common Mistake: Treating the Modes as the Same

A frequent error is to read $X_n\xrightarrow{d}X$ as saying that $X_n(\omega)$ gets close to $X(\omega)$ for most outcomes. That is not what the definition says: it compares marginal CDFs. In the rare-spike example, the distribution becomes concentrated near zero because the probability of the spike vanishes, while the spike's magnitude increases.

A related mistake is to infer sample-path convergence merely from seeing that $P(|X_n-X|>\varepsilon)$ is small for each large $n$. The probability statement concerns one index at a time; the sample-path statement concerns what happens along an entire infinite sequence for a fixed outcome. They ask different questions, so one definition cannot be substituted for another without a separate theorem.

When analyzing a sequence, first identify the proposed limit and then ask which feature is at issue: values along almost every outcome, probabilities of fixed-size errors, or the marginal distributions. State the mode explicitly. The notation $X_n\to X$ without a mode can conceal a materially different claim.

Key takeaway: Almost-sure convergence is eventual numerical convergence along almost every sample path; convergence in probability is vanishing probability of any fixed-size error; convergence in distribution is convergence of CDFs at continuity points of the limiting CDF. They describe different aspects of a sequence.

Check Your Understanding

For each question, state which feature of the sequence is being examined.

In the definition of almost-sure convergence, what is required to hold for every $\omega$ in a set of probability one?
Why does convergence in distribution not require $X_n$ and $X$ to be paired on a common probability space?
For $X_n=Z/n$ with $Z\sim N(0,1)$, write $P(|X_n|>\varepsilon)$ in terms of $Z$.
In the rare-spike example, what is the probability of the event on which $X_n=n$?
Why is the point $x=0$ not required when checking CDF convergence to a point mass at zero?