Implications And Non-Implications Among Convergence Modes

Why the Convergence Modes Must Be Distinguished

Asymptotic arguments often show that a sequence of random variables \(X_n\) approaches a limit \(X\), but “approaches” can mean several different things. Almost-sure convergence makes a statement about the sample paths, \(L^p\) convergence about expected powered errors, convergence in probability about the chance of a noticeable error, and convergence in distribution about the limiting distributions. These modes are related, but they are not interchangeable.

Unless stated otherwise, take \(X_n\) and \(X\) to be random variables on the same probability space when defining almost-sure, \(L^p\), or probability convergence. Distributional convergence only compares their distributions; the variables need not be defined on the same probability space. For \(p>0\), \(L^p\) convergence requires the relevant \(p\)th absolute moments to be finite.

Definition: As \(n\to\infty\), \(X_n\) converges almost surely to \(X\), written \(X_n\to X\) almost surely, if \(P(\lim_{n\to\infty}X_n=X)=1\). It converges in \(L^p\) to \(X\) if \(E|X_n-X|^p\to0\). It converges in probability to \(X\), written \(X_n\xrightarrow{p}X\), if for every \(\varepsilon>0\), \(P(|X_n-X|>\varepsilon)\to0\). It converges in distribution to \(X\), written \(X_n\xrightarrow{d}X\), if the CDFs of \(X_n\) converge to the CDF of \(X\) at every continuity point of that limiting CDF.

The Valid Implications

The standard implications, for random variables on a common probability space where needed, are summarized below. The \(L^q\)-to-\(L^p\) implication applies when \(q>p>0\).

X_n\to X\ \text{almost surely} \ \Longrightarrow\ X_n\xrightarrow{p}X \ \Longrightarrow\ X_n\xrightarrow{d}X, \qquad X_n\to X\ \text{in }L^p \ \Longrightarrow\ X_n\xrightarrow{p}X, \qquad X_n\to X\ \text{in }L^q \ \Longrightarrow\ X_n\to X\ \text{in }L^p \quad(q>p>0).

The first implication follows by fixing \(\varepsilon>0\). If \(X_n\to X\) almost surely, the indicators \(1\{|X_n-X|>\varepsilon\}\) converge to zero almost surely and are bounded by 1. Bounded convergence therefore gives \(P(|X_n-X|>\varepsilon)=E[1\{|X_n-X|>\varepsilon\}]\to0\).

For the next implication, Markov’s inequality applied to the nonnegative random variable \(|X_n-X|^p\) gives \(P(|X_n-X|>\varepsilon)\le E|X_n-X|^p/\varepsilon^p\). If the \(L^p\) error tends to zero, this bound tends to zero for every fixed \(\varepsilon>0\).

Worked Example: An \(L^2\) Bound Gives Convergence in Probability

Suppose \(X_n\) and \(X\) are defined on a common probability space and \(E[(X_n-X)^2]=4/n\). Show that \(X_n\) converges in probability to \(X\), and bound the probability of an error larger than 0.2.

For any \(\varepsilon>0\), Markov’s inequality gives \(P(|X_n-X|>\varepsilon)\le E[(X_n-X)^2]/\varepsilon^2\). Substituting the stated mean squared error,

P(|X_n-X|>\varepsilon)\le \frac{4}{n\varepsilon^2}.

For each fixed positive \(\varepsilon\), the right-hand side tends to zero, which proves convergence in probability. At \(\varepsilon=0.2\), the bound is \(4/(n(0.2)^2)=100/n\). For example, at \(n=500\), the probability is at most \(0.200\). This is an upper bound, not necessarily the exact probability.

To see why convergence in probability implies convergence in distribution, write \(F_n\) and \(F\) for the CDFs of \(X_n\) and \(X\). For any \(\varepsilon>0\),

F(x-\varepsilon)-P(|X_n-X|>\varepsilon) \ \leq\ F_n(x)\ \leq\ F(x+\varepsilon)+P(|X_n-X|>\varepsilon).

For the upper bound, if \(X_n\le x\) and \(|X_n-X|\le\varepsilon\), then \(X\le x+\varepsilon\). For the lower bound, \(X\le x-\varepsilon\) and \(|X_n-X|\le\varepsilon\) imply \(X_n\le x\). If \(X_n\xrightarrow{p}X\), the error probabilities vanish. At a continuity point \(x\) of \(F\), take limits in \(n\), then let \(\varepsilon\) decrease to zero. The two CDF bounds converge to \(F(x)\), so \(F_n(x)\to F(x)\), as required.

Comparing \(L^p\) Modes

On a probability space, convergence in a higher positive moment implies convergence in every lower positive moment. If \(q>p>0\), Jensen’s inequality for the concave function \(t^{p/q}\) gives

E|X_n-X|^p =E\left[(|X_n-X|^q)^{p/q}\right] \leq \left(E|X_n-X|^q\right)^{p/q}.

Thus, if the \(L^q\) error tends to zero, the displayed upper bound tends to zero as well. The probability-space condition matters: it ensures the total measure is one, which is built into this inequality.

Worked Example: \(L^4\) Convergence Implies \(L^2\) Convergence

Suppose \(E|X_n-X|^4=16/n^2\). Determine whether \(X_n\) converges in \(L^2\), and calculate an upper bound for its squared-error expectation.

Use the inequality with \(q=4\) and \(p=2\):

E|X_n-X|^2 \leq \left(E|X_n-X|^4\right)^{2/4} =\left(\frac{16}{n^2}\right)^{1/2} =\frac{4}{n}.

Since \(4/n\to0\), the squared-error expectation tends to zero, so \(X_n\) converges in \(L^2\) to \(X\). For \(n=100\), this calculation gives \(E|X_n-X|^2\leq0.0400\). It is an upper bound, not an equality unless the moment inequality is tight.

Counterexamples: Why the Converses Fail

The valid arrows cannot generally be reversed. A sequence may approach its limit in probability without approaching it on almost every sample path. It may also converge in probability while retaining large errors on rare events, preventing \(L^p\) convergence.

Worked Example: Convergence in Probability Without Almost-Sure Convergence

Let the probability space be \([0,1)\) with the uniform distribution. For each integer \(k\ge1\), divide this interval into \(2^k\) half-open intervals of length \(2^{-k}\). Enumerate all these intervals in successive blocks, with block \(k\) containing the intervals at level \(k\). For the index \(n\) corresponding to an interval \(I\) in block \(k\), set \(X_n=1\) on \(I\) and \(X_n=0\) elsewhere. Take the proposed limit to be \(X=0\).

Within block \(k\), \(P(|X_n|>1/2)=P(I)=2^{-k}\). As \(n\) increases, its block number \(k\) also increases, so this probability tends to zero. For any other fixed \(\varepsilon>0\), the probability is either \(2^{-k}\) (if \(\varepsilon<1\)) or zero (if \(\varepsilon\ge1\)). Therefore \(X_n\xrightarrow{p}0\).

But every point \(\omega\in[0,1)\) belongs to exactly one interval in each block. Consequently, \(X_n(\omega)=1\) once in every block and equals 0 at other indices. The sequence has no limit at that \(\omega\), so it does not converge almost surely to zero. Small probabilities at individual indices do not rule out infinitely many pathwise departures.

Worked Example: Convergence in Probability Without \(L^p\) Convergence

Fix any \(p>0\). On \((0,1]\) with the uniform distribution, define \(X=0\) and \(X_n=n^{1/p}1\{0<\omega\le1/n\}\). Show that \(X_n\) converges in probability to zero but not in \(L^p\).

For any fixed \(0<\varepsilon<n^{1/p}\), the event \(|X_n|>\varepsilon\) is \((0,1/n]\), whose probability is \(1/n\). For fixed \(\varepsilon\), the inequality \(n^{1/p}>\varepsilon\) holds for all sufficiently large \(n\). Thus \(P(|X_n|>\varepsilon)=1/n\to0\), proving convergence in probability.

However, the \(p\)th absolute moment of the error is

E|X_n-X|^p =E\left[n\,1\{0<\omega\le1/n\}\right] =n\cdot\frac{1}{n} =1.

It does not tend to zero, so convergence in \(L^p\) fails. The errors become increasingly large, even as the events on which they occur become less likely.

Distributional Convergence and Constant Limits

Convergence in distribution generally does not imply convergence in probability: distributional convergence does not require \(X_n\) and \(X\) to be close on the same outcomes. There is an important exception when the limiting variable is a constant \(c\). In that case, \(X_n\xrightarrow{d}c\) does imply \(X_n\xrightarrow{p}c\).

To verify this, fix \(\varepsilon>0\). The CDF of the constant \(c\) is continuous at \(c-\varepsilon\) and \(c+\varepsilon\), and has values 0 and 1 there, respectively. Distributional convergence therefore implies \(P(X_n\le c-\varepsilon)\to0\) and \(P(X_n\le c+\varepsilon)\to1\). Since

P(|X_n-c|>\varepsilon) \leq P(X_n\le c-\varepsilon)+P(X_n>c+\varepsilon),

the probability on the left tends to zero. Conversely, convergence in probability implies convergence in distribution by the CDF argument above. Hence convergence in distribution to a constant and convergence in probability to that constant are equivalent.

Worked Example: A Distributional Limit Without a Probability Limit

Let \(X_1,X_2,\ldots\) be independent Bernoulli random variables with \(P(X_n=1)=P(X_n=0)=1/2\). Let \(X\) have the same Bernoulli distribution. Show that \(X_n\xrightarrow{d}X\), but that \(X_n\) does not converge in probability to any random variable on this probability space.

Every \(X_n\) has exactly the same distribution as \(X\). Its CDF is therefore the limiting CDF for every \(n\), which proves \(X_n\xrightarrow{d}X\).

For distinct \(n,m\), independence gives \(P(|X_n-X_m|>1/2)=P(X_n\ne X_m)=1/2\). If \(X_n\) converged in probability to some random variable \(Y\), then the triangle inequality would give

P(|X_n-X_m|>1/2) \leq P(|X_n-Y|>1/4)+P(|X_m-Y|>1/4).

As \(n,m\to\infty\), the right-hand side would tend to zero, contradicting the left-hand side, which equals \(1/2\). The sequence is thus not Cauchy in probability and cannot converge in probability to any \(Y\). In particular, its distributional convergence to a Bernoulli variable does not establish probability convergence to that variable.

Common Mistake and Practical Summary

A common mistake is to reason that convergence in distribution means the variables themselves get close to their limit. It does not: the definition compares CDFs, not paired outcomes. The independent Bernoulli example has an unchanged distribution at every index and still fails to converge in probability. To infer probability convergence from distributional convergence, check that the limiting distribution is concentrated at a single constant.

When using convergence modes in an asymptotic argument, identify the exact mode established by the available result before drawing a conclusion. In particular, Markov’s inequality can turn vanishing \(L^p\) error into convergence in probability, but it cannot reverse that implication; nor does convergence in probability alone guarantee almost-sure or \(L^p\) convergence.

Key takeaway: Almost-sure and \(L^p\) convergence each imply convergence in probability, which implies convergence in distribution. Higher-order \(L^q\) convergence implies lower-order \(L^p\) convergence on a probability space. The reverse implications fail in general, except that convergence in distribution to a constant is equivalent to convergence in probability to that constant.

Check Your Understanding

For each question, distinguish what the stated convergence mode does and does not establish.

If \(E|X_n-X|^3\to0\), which of the four convergence modes in this tutorial follow, and in what order?
Why does the Markov bound for \(P(|X_n-X|>\varepsilon)\) not imply \(L^p\) convergence from convergence in probability?
In the interval construction, what happens to \(X_n(\omega)\) across successive blocks for a fixed \(\omega\in[0,1)\)?
State the extra condition under which convergence in distribution implies convergence in probability.
If \(X_n\xrightarrow{d}X\) and \(X\) is not constant, can one conclude that \(X_n\xrightarrow{p}X\)? Explain using the meaning of the two convergence modes.