Why a Small Probability of Disagreement Controls CDFs
Convergence in probability says that, for any fixed tolerance, the chance that \(X_n\) differs from \(X\) by more than that tolerance becomes small. Convergence in distribution asks a different question: do the cumulative distribution functions of \(X_n\) approach the CDF of \(X\) at its continuity points? The connection is that, when \(X_n\) is close to \(X\), the event \(X_n\leq x\) can differ from \(X\leq x\) only when the variables are not close or when \(X\) lies near the threshold \(x\).
The variables \(X_n\) and \(X\) in convergence in probability are defined on the same probability space, so the event \(\{|X_n-X|>\varepsilon\}\) is meaningful. Write \(F_n(x)=P(X_n\leq x)\) and \(F(x)=P(X\leq x)\). Convergence in distribution to the distribution of \(X\) means that \(F_n(x)\to F(x)\) for every continuity point \(x\) of \(F\).
The key is to fix a tolerance \(\varepsilon>0\) and compare events. The tolerance creates a small interval around \(x\), while the probability of exceeding the tolerance accounts for all other discrepancies. First derive those bounds; then convergence in probability removes the error term, and continuity removes the interval around \(x\).
A CDF Bound with a Probability-Error Term
For the upper bound, split according to whether \(X_n\) is within \(\varepsilon\) of \(X\). If \(X_n\leq x\) and \(|X_n-X|\leq\varepsilon\), then \(X\leq x+\varepsilon\). Consequently, $$ \{X_n\leq x\}\subseteq \{X\leq x+\varepsilon\}\cup\{|X_n-X|>\varepsilon\}, $$ which gives \(F_n(x)\leq F(x+\varepsilon)+P(|X_n-X|>\varepsilon)\).
For the lower bound, if \(X\leq x-\varepsilon\) and \(|X_n-X|\leq\varepsilon\), then \(X_n\leq X+\varepsilon\leq x\). Thus $$ \{X\leq x-\varepsilon\}\subseteq\{X_n\leq x\}\cup\{|X_n-X|>\varepsilon\}. $$ Taking probabilities and rearranging gives the claimed lower bound. Both estimates isolate the same error probability; no independence assumption is needed.
Convergence in Probability Implies Convergence in Distribution
Proof. Fix a continuity point \(x\) of \(F\) and fix \(\varepsilon>0\). Apply the CDF bound just proved. Since \(X_n\xrightarrow{p}X\), the error probability tends to zero, so $$ F(x-\varepsilon)\leq \liminf_{n\to\infty}F_n(x) \leq \limsup_{n\to\infty}F_n(x)\leq F(x+\varepsilon). $$ Now let \(\varepsilon\) decrease to zero. The values \(F(x+\varepsilon)\) decrease to \(F(x)\) by right-continuity of a CDF. The values \(F(x-\varepsilon)\) increase to the left limit \(F(x-)\); because \(x\) is a continuity point, \(F(x-)=F(x)\). Therefore the lower and upper limits of \(F_n(x)\) both equal \(F(x)\), and \(F_n(x)\to F(x)\). This holds at every continuity point of \(F\), which is exactly convergence in distribution. \(\square\)
The proof has two distinct ingredients. Convergence in probability eliminates the probability of a discrepancy larger than a fixed tolerance. Continuity of \(F\) then ensures that moving the threshold slightly to either side of \(x\) does not change the limiting CDF value. The continuity condition is why the conclusion is stated at continuity points, not necessarily at every real number.
Choose a continuity point \(x\) of \(F\) and any \(\varepsilon>0\).
Use \(F(x-\varepsilon)-P(|X_n-X|>\varepsilon)\leq F_n(x)\leq F(x+\varepsilon)+P(|X_n-X|>\varepsilon)\).
The error probability tends to zero, leaving \(F(x-\varepsilon)\) and \(F(x+\varepsilon)\) as bounds.
Continuity at \(x\) makes both bounds approach \(F(x)\), proving CDF convergence there.
Worked Applications
Worked Example: A Uniform Random Variable with a Vanishing Shift
Let \(X\sim\operatorname{Uniform}(0,1)\) and define \(X_n=X+1/n\). Show convergence in probability and verify the resulting CDF convergence.
For every outcome, \(|X_n-X|=1/n\). Given any \(\varepsilon>0\), choose an integer \(N>1/\varepsilon\). For every \(n\geq N\), \(1/n<\varepsilon\), so \(P(|X_n-X|>\varepsilon)=0\). Hence \(X_n\xrightarrow{p}X\).
The CDF of \(X\) is \(F(x)=0\) for \(x\leq0\), \(F(x)=x\) for \(0<x<1\), and \(F(x)=1\) for \(x\geq1\). Since \(X_n\) is uniform on \((1/n,1+1/n)\), its CDF is $$ F_n(x)= \begin{cases} 0,&x\leq 1/n,\\ x-1/n,&1/n<x<1+1/n,\\ 1,&x\geq 1+1/n. \end{cases} $$ For each fixed \(x\in(0,1)\), all sufficiently large \(n\) have \(1/n<x<1+1/n\), so \(F_n(x)=x-1/n\to x=F(x)\). For \(x\leq0\), \(F_n(x)=0=F(x)\); for \(x\geq1\), \(F_n(x)\to1=F(x)\) (and at \(x=1\), \(F_n(1)=1-1/n\)). Thus the CDFs converge at every \(x\), consistent with the theorem.
Worked Example: A Rare Perturbation of Fixed Size
Let \(X\sim\operatorname{Uniform}(0,1)\). Independently of \(X\), let \(B_n\) be Bernoulli with \(P(B_n=1)=1/n\), and set \(X_n=X+2B_n\). Verify convergence in probability and calculate \(F_n\).
The difference is \(|X_n-X|=2B_n\). If \(0<\varepsilon<2\), then $$ P(|X_n-X|>\varepsilon)=P(B_n=1)=1/n\longrightarrow0. $$ If \(\varepsilon\geq2\), the probability is zero because the difference never exceeds \(2\). Thus the required limit is zero for every \(\varepsilon>0\), and \(X_n\xrightarrow{p}X\).
Conditioning on \(B_n\), with \(F\) the uniform \((0,1)\) CDF, gives $$ F_n(x)=(1-1/n)F(x)+(1/n)F(x-2). $$ For example, at \(x=3/2\), \(F(3/2)=1\) and \(F(-1/2)=0\), so \(F_n(3/2)=1-1/n\to1=F(3/2)\). More generally, since both \(F(x)\) and \(F(x-2)\) lie between zero and one, $$ |F_n(x)-F(x)|=\frac{1}{n}|F(x-2)-F(x)|\leq\frac{1}{n}. $$ This bound holds for every \(x\), so the CDFs converge to \(F\) at every point.
Worked Example: A Rare Perturbation Whose Size Grows
Again let \(X\sim\operatorname{Uniform}(0,1)\), and let \(B_n\) be Bernoulli with \(P(B_n=1)=1/n^2\), independent of \(X\). Define \(X_n=X+nB_n\). The perturbation can be as large as \(n\); determine whether convergence in probability still holds.
Fix \(\varepsilon>0\), and choose \(N>\varepsilon\). For \(n\geq N\), the difference is zero when \(B_n=0\), and equals \(n>\varepsilon\) when \(B_n=1\). Therefore $$ P(|X_n-X|>\varepsilon)=P(B_n=1)=1/n^2\longrightarrow0. $$ So \(X_n\xrightarrow{p}X\), despite the increasing size of the perturbation when it occurs.
Conditioning on \(B_n\) gives $$ F_n(x)=(1-1/n^2)F(x)+(1/n^2)F(x-n). $$ For every fixed \(x\), eventually \(x-n\leq0\), and hence \(F(x-n)=0\) for the uniform distribution. It follows that \(F_n(x)=(1-1/n^2)F(x)\to F(x)\). The result illustrates that convergence in probability is controlled by the chance of a large discrepancy, not by whether the discrepancy has a fixed upper bound.
Common Mistake and Why the Result Matters
A common mistake is to reason that \(X_n\) must be close to \(X\) on every outcome once \(n\) is large. Convergence in probability does not assert that. In the growing-perturbation example, \(X_n-X=n\) on an event of probability \(1/n^2\). What converges to zero is the probability of exceeding any fixed tolerance, not the maximum possible difference.
Another error is to omit the continuity-point condition and claim CDF convergence at every \(x\). The proof squeezes \(F_n(x)\) between values of \(F\) on either side of \(x\); those bounds meet at \(F(x)\) when \(F\) is continuous there. If they do not meet, this argument does not establish convergence at that point. The continuity-point criterion is therefore built into the conclusion.
The implication is useful because convergence in probability is a statement about the joint behavior of variables on a common probability space, whereas convergence in distribution records only their marginal CDFs. The theorem lets a probabilistic approximation yield a distributional conclusion without calculating the CDF of every \(X_n\). It is one direction of comparison: the proof here does not assert the converse.
Check Your Understanding
Use the CDF bounds and the continuity-point condition to answer each question.
- In the lower CDF bound, which event inclusion lets us compare \(F(x-\varepsilon)\) with \(F_n(x)\)?
- Why is it necessary to choose a fixed \(\varepsilon\) before taking the limit in \(n\)?
- In the vanishing-shift example, calculate \(F_n(1/2)\) for \(n>2\) and find its limit.
- For the growing rare perturbation, what is \(P(|X_n-X|>3)\) when \(n>3\)?
- Where in the proof is continuity of \(F\) at \(x\) used?