Why Separate Counterexamples Matter
Several notions of convergence describe different features of a sequence of random variables. Convergence in distribution concerns the laws of the variables; convergence in probability concerns their deviations from a particular limit on a shared probability space; almost-sure convergence concerns the behavior of nearly every entire sample path; and \(L^p\) convergence controls an expected \(p\)th-power error. Knowing that one mode holds does not automatically establish another. Explicit examples show which information each mode retains and which it can miss.
Throughout, the random variables are real-valued and indexed by discrete time \(n\ge1\). For convergence in probability and almost-sure convergence, the variables and their proposed limit are defined on the same probability space. We write \(X_n\to X\) in probability when, for every \(\varepsilon>0\), \(\mathbb{P}(|X_n-X|>\varepsilon)\to0\). We write \(X_n\to X\) almost surely when \(\mathbb{P}(\{\omega:X_n(\omega)\to X(\omega)\})=1\). Convergence in distribution, written \(X_n\Rightarrow X\), means that the cumulative distribution functions converge at every continuity point of the distribution function of \(X\). For fixed \(1\le p<\infty\), convergence in \(L^p\) means \(\mathbb{E}|X_n-X|^p\to0\).
The first implication is useful for locating the examples in the hierarchy. Here is a direct verification using distribution functions. It also emphasizes that probability convergence controls deviations from the proposed limit, not merely the separate distributions.
Proof. Assume \(X_n\to X\) in probability, and let \(x\) be a continuity point of the cumulative distribution function \(F_X\). For any \(\delta>0\), on the event \(\{X\le x-\delta,\ |X_n-X|\le\delta\}\) we have \(X_n\le x\). Also, on \(\{X_n\le x,\ |X_n-X|\le\delta\}\) we have \(X\le x+\delta\). Therefore
For fixed \(\delta\), the error probability tends to zero as \(n\to\infty\). Thus the lower and upper limits of \(\mathbb{P}(X_n\le x)\) are bounded below by \(F_X(x-\delta)\) and above by \(F_X(x+\delta)\), respectively. Letting \(\delta\) decrease to zero, continuity at \(x\) makes both bounds tend to \(F_X(x)\). Hence \(\mathbb{P}(X_n\le x)\to F_X(x)\) at every continuity point \(x\), which is convergence in distribution. \(\square\)
Distributional Convergence Does Not Ensure Probability Convergence
Distributional convergence compares one-dimensional laws and does not record how \(X_n\) and \(X\) are paired on their common probability space. The following example keeps the law fixed at every index while maintaining a nonvanishing deviation from the proposed limit.
Worked Example: Equal Laws, Persistent Deviation
Take \(\Omega=(0,1)\), with its Borel sets and uniform probability measure, and let \(U(\omega)=\omega\). The state space is \(\mathbb{R}\), and the time-indexed sequence is \(X_n=U\) for every \(n\ge1\). Set \(X=1-U\). We compare their laws and their deviations on this same probability space.
For \(0\le x\le1\), the distribution function of \(U\) is \(F_U(x)=x\). The variable \(1-U\) is also uniform on \((0,1)\), so \(F_X(x)=x\) on the same interval. Both distribution functions are zero for \(x<0\) and one for \(x\ge1\). Consequently \(F_{X_n}(x)=F_X(x)\) for every real \(x\) and every \(n\), which verifies \(X_n\Rightarrow X\).
But \(|X_n-X|=|2U-1|\) at every index. Choose the fixed tolerance \(\varepsilon=\tfrac12\). The strict deviation event is \(\{|2U-1|>\tfrac12\}=\{U<\tfrac14\}\cup\{U>\tfrac34\}\). These disjoint intervals have probabilities \(\tfrac14\) and \(\tfrac14\), so
Check: Since the deviation probability stays equal to \(\tfrac12\), it does not tend to zero. Thus convergence in probability fails, despite the identical laws. \(\square\)
This example depends on the relationship between the variables, not just their marginal distributions. In fact, \(X_n\) and \(X\) are usually different at each sample point, even though their laws agree. Distributional convergence is not a statement that random variables become close on a particular probability space.
Convergence in Probability Does Not Ensure Almost-Sure Convergence
To separate these modes, it is useful to arrange for the probability of a deviation to shrink while making deviations recur along every sample path. The construction below sweeps through finer and finer partitions of the sample space. It is a direct calculation; it does not require a theorem about sums of event probabilities.
Worked Example: The Typewriter Sequence
Take \(\Omega=[0,1)\), with its Borel sets and uniform probability measure, and state space \(\mathbb{R}\). For each integer \(k\ge0\) and \(j\in\{0,1,\ldots,2^k-1\}\), define the index \(n=2^k+j\) and the random variable
These index blocks fit together without gaps: level \(k\) uses the indices from \(2^k\) through \(2^{k+1}-1\). The displayed intervals partition \([0,1)\) into \(2^k\) half-open intervals, each of probability \(2^{-k}\). Thus, for \(0<\varepsilon<1\), the strict event \(\{|X_n-X|>\varepsilon\}\) is exactly the interval defining \(X_n\). If \(n=2^k+j\), its probability is \(2^{-k}\). As \(n\to\infty\), its level \(k\) tends to infinity, so \(2^{-k}\to0\). For \(\varepsilon\ge1\), the strict event is empty because \(X_n\) takes only the values zero and one. This proves \(X_n\to0\) in probability for every \(\varepsilon>0\).
Now fix any \(\omega\in[0,1)\). At each level \(k\), exactly one of the partition intervals contains \(\omega\); hence exactly one variable in that level equals one at \(\omega\). There is at least one other interval at every level \(k\ge1\), and its indicator equals zero at \(\omega\). As a result, along the indices of increasing levels the sequence has values one infinitely often and zero infinitely often. A real sequence with these two constant subsequences cannot converge. This holds for every \(\omega\), so there is no almost-sure convergence to \(X\), or to any other limit. \(\square\)
The distinction is between fixed-index probabilities and the behavior of an entire path. At a large index, the chance of a deviation is small. Nevertheless, each fixed sample point belongs to one interval at every level, so it encounters infinitely many deviations. Pointwise recurrence of deviations does not contradict their probabilities tending to zero.
Almost-Sure Convergence Does Not Ensure \(L^p\) Convergence
Almost-sure convergence gives eventual control at almost every individual sample point, but it does not by itself bound the expected size of errors uniformly. A sequence can be zero eventually at every sample point while retaining a constant \(p\)th moment at every index: the nonzero values can grow as their supporting events shrink.
Worked Example: Shrinking Events and Constant \(p\)th Moment
Fix \(1\le p<\infty\). Take \(\Omega=(0,1)\) with uniform probability measure, state space \(\mathbb{R}\), and define, for \(n\ge1\),
For each fixed \(\omega\in(0,1)\), once \(n\ge1/\omega\), we have \(\omega\ge1/n\), so \(\omega\notin(0,1/n)\) and \(X_n(\omega)=0\). Thus \(X_n(\omega)\) is eventually zero for every sample point in \(\Omega\), proving almost-sure convergence to zero.
For the \(L^p\) error, raise the definition to the \(p\)th power: \(|X_n-X|^p=n\mathbf{1}_{(0,1/n)}\). The interval \((0,1/n)\) has probability \(1/n\), so
Each \(X_n\) has a finite \(p\)th moment, but these errors do not tend to zero in expectation. Therefore \(X_n\) does not converge to \(X\) in \(L^p\). As an additional check, for any fixed \(\varepsilon>0\), once \(n^{1/p}>\varepsilon\), the event \(\{|X_n-X|>\varepsilon\}\) is exactly \((0,1/n)\), whose probability is \(1/n\to0\). The example is also consistent with convergence in probability. \(\square\)
The last calculation shows why the size of an error and the probability of an error cannot be considered separately. The error is supported on a set of probability \(1/n\), but its \(p\)th power there is \(n\); multiplying these quantities leaves the expectation equal to one.
Common Pitfalls and a Comparison
Common mistake: treating equal distributions as proof of convergence in probability. In the first example, one might incorrectly calculate \(\mathbb{P}(|X_n-X|>\tfrac12)=0\) on the grounds that \(X_n\) and \(X\) have the same uniform distribution. Equal distributions only say that each variable has the same probabilities for events determined by its own value. They do not say that the variables agree, or are close, at the same \(\omega\). The correct calculation uses their joint definition: \(|X_n-X|=|2U-1|\), and the deviation probability is \(\tfrac12\) at every index.
A second common error is to infer pathwise convergence from probabilities that vanish at each index. In the typewriter sequence, the probabilities do vanish, but the intervals move across the sample space as the index increases. To establish almost-sure convergence one must examine the full sequence at each sample point, not just the probability at one index. Finally, moment convergence is a separate requirement: almost-sure convergence in the third example coexists with a constant expected \(p\)th-power error.
| Mode verified | Mode that fails | What the calculation shows |
|---|---|---|
| Convergence in distribution | Convergence in probability | Identical laws do not control the paired deviation. |
| Convergence in probability | Almost-sure convergence | Deviation probabilities can vanish while deviations recur on every path. |
| Almost-sure convergence | \(L^p\) convergence for fixed finite \(p\) | Errors can shrink pointwise while their \(p\)th moments stay constant. |
When assessing a convergence claim, match the test to the mode. For distributional convergence, compare distribution functions at continuity points. For convergence in probability, fix a positive tolerance and calculate the deviation probability. For almost-sure convergence, follow the whole sequence at a sample point. For \(L^p\) convergence, compute or bound the expected \(p\)th power of the error. The three counterexamples show that no one of these calculations can be substituted for another.
Check Your Understanding
For each question, identify the mode being tested and justify your answer using the relevant definition.
- In the equal-law example, what is \(\mathbb{P}(|X_n-X|>\tfrac14)\), and does it depend on \(n\)?
- In the typewriter sequence, why does the half-open convention ensure that each \(\omega\in[0,1)\) belongs to exactly one interval at every level?
- For the typewriter sequence, what is the deviation probability at index \(n=2^k+j\) for a tolerance \(0<\varepsilon<1\)?
- In the shrinking-event example, calculate \(\mathbb{E}|X_n|^p\) directly from the value and probability of the nonzero event.
- Which implication does the proof using distribution functions establish, and which feature of the typewriter example rules out almost-sure convergence?