A Rare Event Carries the Mean
A sequence of random variables can converge almost surely to zero while keeping the same positive expectation at every time. The mechanism is that its nonzero values grow as the events supporting them become rarer. A fair-coin construction makes this effect explicit: surviving \(n\) tosses without a tail has probability \(2^{-n}\), while the value on that event is \(2^n\). Their product remains one.
Let \((\Omega,\mathcal{F},\mathbb{P})\) be the space of infinite sequences of independent fair coin tosses, with \(\mathbb{P}(H)=\mathbb{P}(T)=\tfrac12\). Let \(\mathcal{F}_n\) be the sigma-field generated by the first \(n\) tosses, and let \(\mathcal{F}_0=\{\varnothing,\Omega\}\). For \(n\geq 1\), let \(A_n\) be the event that the first \(n\) tosses are all heads; set \(A_0=\Omega\). Consider the discrete-time process \((X_n)_{n\geq0}\) with state space \(S=\{0\}\cup\{2^n:n\geq0\}\), defined by \(X_n=2^n\mathbf{1}_{A_n}\). Thus \(X_0=1\).
The process is adapted because \(X_n\) is determined by the first \(n\) tosses. Since \(\mathbb{P}(A_n)=2^{-n}\), its expectation is finite and equals \(\mathbb{E}X_n=2^n2^{-n}=1\). The central questions are whether the process is a martingale, what its pathwise limit is, and whether it converges to that limit in \(L^1\).
Martingale Property and Almost-Sure Limit
Proof. We first verify the conditional expectation identity. On the event \(A_n\), the next toss is an independent fair toss. If it is heads, then \(X_{n+1}=2^{n+1}\); if it is tails, then \(X_{n+1}=0\). Therefore, on \(A_n\), the conditional expectation is \(\tfrac12 2^{n+1}+\tfrac12 0=2^n=X_n\). On \(A_n^c\), at least one of the first \(n\) tosses is tails. The process is then zero at time \(n\), and it remains zero at time \(n+1\), so the conditional expectation is also \(0=X_n\). This proves \(\mathbb{E}[X_{n+1}\mid\mathcal{F}_n]=X_n\) almost surely. Integrability follows from \(\mathbb{E}X_n=2^n\mathbb{P}(A_n)=2^n2^{-n}=1\).
Next, the events \(A_n\) decrease with \(n\), and their intersection is the event that every toss is heads. By continuity of probability for decreasing events, \(\mathbb{P}(\bigcap_{n\geq0}A_n)=\lim_{n\to\infty}\mathbb{P}(A_n)=\lim_{n\to\infty}2^{-n}=0\). Outside this probability-zero event, some toss is tails. After that toss, \(X_n=0\) at every subsequent time. Hence \(X_n\to0\) almost surely.
For every \(n\), the limit just found gives \(\mathbb{E}|X_n-0|=\mathbb{E}X_n=1\), not a quantity tending to zero. Thus the process does not converge to zero in \(L^1\). \(\square\)
The result separates three statements that are sometimes mistakenly treated as interchangeable. Almost-sure convergence describes the eventual behavior of almost every individual path. Convergence in probability describes the probabilities of deviations at a fixed positive threshold. Convergence in \(L^1\) controls the expected size of the deviation. This example has the first two kinds of convergence, but not the third.
Worked Example: The Distribution at Time Three
At time \(3\), the process equals \(8\) precisely when the first three tosses are heads. There are \(2^3=8\) equally likely three-toss sequences, and exactly one is \(HHH\). Thus \(\mathbb{P}(X_3=8)=\tfrac18\) and \(\mathbb{P}(X_3=0)=\tfrac78\).
The expectation is \(\mathbb{E}X_3=8\cdot\tfrac18+0\cdot\tfrac78=1\). Also, \(\mathbb{E}[X_3^2]=8^2\cdot\tfrac18+0^2\cdot\tfrac78=64\cdot\tfrac18=8\), so \(\operatorname{Var}(X_3)=8-1^2=7\).
Check: Given that the first two tosses are heads, \(X_2=4\). The third toss gives \(X_3=8\) with probability \(\tfrac12\) and \(X_3=0\) with probability \(\tfrac12\), so the conditional mean is \(8\cdot\tfrac12+0\cdot\tfrac12=4=X_2\). If the first two tosses are not both heads, then both \(X_2\) and \(X_3\) are zero. These cases verify the one-step martingale identity at time \(2\).
Worked Example: The First Tail and Pathwise Convergence
Define \(T=\inf\{k\geq1:\text{the }k\text{th toss is tails}\}\), with \(T=\infty\) if there is no tail. For \(k\geq1\), the event \(T=k\) requires \(k-1\) heads followed by a tail. Independence gives \(\mathbb{P}(T=k)=(\tfrac12)^{k-1}\tfrac12=2^{-k}\). For example, \(\mathbb{P}(T=1)=\tfrac12\), \(\mathbb{P}(T=2)=\tfrac14\), and \(\mathbb{P}(T=3)=\tfrac18\).
The probability that the first tail occurs after toss \(n\), or never occurs, is \(\mathbb{P}(T>n)=2^{-n}\). In particular, \(\mathbb{P}(T=\infty)=\lim_{n\to\infty}2^{-n}=0\). The finite partial sum of the probabilities of a first tail by toss \(m\) is \(\sum_{k=1}^{m}2^{-k}=1-2^{-m}\), which tends to \(1\), consistent with \(\mathbb{P}(T<\infty)=1\).
On the event \(T=k<\infty\), the first \(n\) tosses are all heads for \(n<k\), while for \(n\geq k\) at least one of them is tails. Consequently \(X_n=0\) for every \(n\geq k\), proving the pathwise convergence to zero whenever \(T\) is finite.
Check in probability: Fix \(\varepsilon>0\). If \(2^n\leq\varepsilon\), then \(\mathbb{P}(|X_n|>\varepsilon)=0\). If \(2^n>\varepsilon\), the strict event \(|X_n|>\varepsilon\) occurs exactly on \(A_n\), so its probability is \(2^{-n}\). For all sufficiently large \(n\), the second case applies, and \(2^{-n}\to0\). Therefore \(X_n\to0\) in probability as well.
Why the Limit Does Not Capture the Expectations
The expectation at time \(n\) is carried entirely by \(A_n\). That event has probability \(2^{-n}\), which tends to zero, but the value there is \(2^n\), which grows at the reciprocal rate. Thus the exceptional paths that have not yet seen a tail become rare without their contribution to the mean becoming small.
Worked Example: Tail Expectations at a Fixed Threshold
A family of integrable random variables is uniformly integrable if \(\sup_n\mathbb{E}[|X_n|\mathbf{1}_{\{|X_n|>K\}}]\to0\) as \(K\to\infty\). The event in this definition uses the strict inequality \(>K\), and the same threshold \(K\) must be used for every member of the family.
Take the fixed threshold \(K=5\). At \(n=0,1,2\), the nonzero values are \(1,2,4\), respectively, so none is strictly greater than \(5\), and each tail expectation is zero. At \(n=3\), \(X_3=8\) with probability \(\tfrac18\), so \(\mathbb{E}[|X_3|\mathbf{1}_{\{|X_3|>5\}}]=8\cdot\tfrac18=1\). For every \(n\geq3\), the nonzero value \(2^n\) is strictly greater than \(5\), and the tail expectation is \(2^n\cdot2^{-n}=1\).
More generally, for every finite \(K\geq0\), choose \(n\) with \(2^n>K\). Then \(\mathbb{E}[|X_n|\mathbf{1}_{\{|X_n|>K\}}]=2^n\mathbb{P}(A_n)=1\). No such tail expectation exceeds \(\mathbb{E}|X_n|=1\), so the supremum over \(n\) is exactly \(1\) for every finite \(K\). It does not tend to zero as \(K\) grows, and the martingale is not uniformly integrable.
Check: At the threshold \(K=5\), the first index with a value strictly above the threshold is \(n=3\), since \(2^2=4\leq5\) but \(2^3=8>5\). The tail expectation at that index is exactly \(8\cdot\tfrac18=1\), as computed directly.
A Common Mistake
A tempting but invalid calculation is: “The probability of \(A_n\) tends to zero, so the expectation of \(X_n\) tends to zero.” This drops the size of \(X_n\) on \(A_n\). At \(n=3\), for instance, the event has probability \(\tfrac18\), but its value is \(8\), and their product is \(8\cdot\tfrac18=1\), not zero. For general \(n\), the same calculation gives \(2^n\cdot2^{-n}=1\).
There is a related error in reasoning that \(X_n\to0\) almost surely and therefore \(\mathbb{E}X_n\to0\). Almost-sure convergence alone does not justify passing a limit through an expectation. Here the direct computation gives \(\mathbb{E}X_n=1\) for every \(n\), whereas the almost-sure limit has expectation \(\mathbb{E}0=0\). In particular, if \(X_n\) converged to zero in \(L^1\), then \(|\mathbb{E}X_n-\mathbb{E}0|\leq\mathbb{E}|X_n-0|\) would tend to zero, contradicting \(|1-0|=1\).
This martingale is a useful diagnostic example: bounded first moments, \(\mathbb{E}|X_n|=1\), do not prevent mass from moving to larger values on rarer events. The pathwise limit records what happens eventually on almost every trajectory, while the expectation at each finite time still includes the contribution from trajectories that have so far produced only heads.
Check Your Understanding
Use the definitions and calculations above to answer the following questions.
- For a history in which the first \(n\) tosses are all heads, what are the two possible values of \(X_{n+1}\), and what is their conditional mean?
- Calculate \(\mathbb{P}(T=4)\) and \(\mathbb{P}(T>4)\). Explain how these probabilities relate to \(X_4\).
- For a fixed \(\varepsilon>0\), describe \(\mathbb{P}(|X_n|>\varepsilon)\) in the cases \(2^n\leq\varepsilon\) and \(2^n>\varepsilon\).
- At threshold \(K=9\), find the smallest \(n\) for which \(\mathbb{E}[|X_n|\mathbf{1}_{\{|X_n|>K\}}]\) is nonzero, and calculate that expectation.
- Why does almost-sure convergence to zero not imply that the expectations converge to zero in this example?