From Convergence in Probability to Expectations
Convergence in probability controls the chance of a fixed-size error, but it does not by itself control the contribution of very large values to an expectation. A sequence can be close to its limit with high probability and still carry a fixed amount of expectation in events whose probabilities tend to zero. Uniform integrability rules out that escape into large values. Together, convergence in probability and uniform integrability give convergence in \(L^1\), and therefore convergence of expectations.
Throughout, let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space, so \(\mathbb{P}(\Omega)=1\). Random variables in this tutorial are measurable, real-valued functions. A random variable \(X\) is integrable when \(\mathbb{E}|X|=\int_\Omega |X|\,d\mathbb{P}<\infty\); in that case its expectation \(\mathbb{E}X\) is finite. A sequence \(X_n\) converges in probability to \(X\) if, for every \(\varepsilon>0\), \(\mathbb{P}(|X_n-X|>\varepsilon)\to0\).
The relevant form of Vitali’s theorem applies on any finite measure space. In this probability-space setting, if \((X_n)\) is uniformly integrable and \(X_n\to X\) in probability, then \(X\) is integrable and \(\mathbb{E}|X_n-X|\to0\). This is the Vitali convergence theorem, established earlier in the course by truncation. Its hypotheses are convergence in measure and uniform integrability; on a probability space, convergence in probability is precisely convergence in measure. We now derive the expectation conclusion carefully from this result.
The Expectation Convergence Theorem
Proof. The stated hypotheses are those of Vitali’s theorem on the finite measure space \((\Omega,\mathcal{F},\mathbb{P})\): the sequence is uniformly integrable and converges in measure to \(X\). Vitali’s theorem gives \(X\in L^1(\mathbb{P})\) and \(\mathbb{E}|X_n-X|\to0\). For every \(n\), linearity of the integral for integrable functions and the integral triangle inequality give $$ |\mathbb{E}X_n-\mathbb{E}X| =|\mathbb{E}(X_n-X)| \le \mathbb{E}|X_n-X|. $$ The right side tends to zero, so the left side tends to zero. Therefore \(\mathbb{E}X_n\to\mathbb{E}X\). \(\square\)
The conclusion is stronger than convergence of the numbers \(\mathbb{E}X_n\): it gives convergence in \(L^1\), which controls the expected absolute error. This distinction matters because the signed difference \(\mathbb{E}X_n-\mathbb{E}X\) can be small through cancellation even when \(\mathbb{E}|X_n-X|\) is not.
Worked Example: Powers on the Unit Interval
Take \(\Omega=[0,1]\), \(\mathcal{F}=\mathcal{L}|_{[0,1]}\), and \(\mathbb{P}(A)=m(A)\). Let \(X_n(x)=x^n\), and let \(X(x)=0\) for \(0\le x<1\), with \(X(1)=1\). We verify convergence in probability, uniform integrability, and convergence of expectations.
Fix \(0<\varepsilon<1\). The strict tail event is $$ \{x:|X_n(x)-X(x)|>\varepsilon\}=\{x\in[0,1):x^n>\varepsilon\}. $$ Its measure is \(1-\varepsilon^{1/n}\), which tends to zero as \(n\to\infty\). For \(\varepsilon\ge1\), the event is empty. Thus \(X_n\to X\) in probability. For every \(n\), \(0\le X_n\le1\), so for every \(K\ge1\) the event \(\{|X_n|>K\}\) is empty. The tail expectations are all zero for such \(K\), proving uniform integrability.
The point \(x=1\) has measure zero, so \(\mathbb{E}X=0\). Direct integration gives $$ \mathbb{E}X_n=\int_0^1 x^n\,dx=\frac{1}{n+1}, \qquad \mathbb{E}|X_n-X|=\frac{1}{n+1}. $$ Both quantities tend to zero. Check: substituting \(n=1,2,3\) into \(1/(n+1)\) gives \(1/2,1/3,1/4\), respectively; these agree with the integrals of \(x,x^2,x^3\) on \([0,1]\). The theorem’s conclusion also agrees with the direct \(L^1\) calculation.
Worked Example: Truncations of an Integrable Random Variable
Let \(\Omega=\mathbb{N}=\{1,2,\ldots\}\), let \(\mathcal{F}\) be its power set, and set \(\mathbb{P}(\{k\})=2^{-k}\). This is a probability measure because \(\sum_{k=1}^{\infty}2^{-k}=1\). Define \(X(k)=k\) and \(X_n(k)=\min(k,n)\). For each fixed \(k\), \(X_n(k)=k\) once \(n\ge k\); hence \(X_n\to X\) everywhere, and thus in probability.
For any \(K>0\), the pointwise inequalities \(0\le X_n\le X\) imply $$ X_n\chi_{\{X_n>K\}}\le X\chi_{\{X>K\}}. $$ Indeed, \(X_n>K\) implies \(X>K\), and \(X_n\le X\). If \(q\) is a nonnegative integer, the geometric tail calculation gives $$ \mathbb{E}\bigl[X\chi_{\{X>q\}}\bigr] =\sum_{k=q+1}^{\infty}\frac{k}{2^k} =\frac{q+2}{2^q}. $$ For general \(K\), take \(q=\lfloor K\rfloor\), so \(\{X>K\}=\{X>q\}\). The displayed bound tends to zero as \(K\to\infty\), uniformly in \(n\); therefore \((X_n)\) is uniformly integrable.
For the expectations, \(\mathbb{P}(k\ge n)=\sum_{k=n}^{\infty}2^{-k}=2^{-(n-1)}\). Splitting the sum at \(n\) yields $$ \mathbb{E}X_n =\sum_{k=1}^{n-1}\frac{k}{2^k}+n\,2^{-(n-1)} =2-\frac{1}{2^{n-1}}. $$ Here we used the finite-sum identity \(\sum_{k=1}^{n-1}k2^{-k}=2-(n+1)2^{-(n-1)}\), which follows by multiplying the sum by \(2\) and subtracting the original sum. In particular, for \(n=3\), the direct computation is \(1/2+2/4+3(1/4)=7/4\), matching \(2-1/4=7/4\).
To compute the limit expectation independently, nonnegativity allows the tail-sum formula by Tonelli’s theorem: $$ \mathbb{E}X=\sum_{j=1}^{\infty}\mathbb{P}(X\ge j) =\sum_{j=1}^{\infty}\sum_{k=j}^{\infty}2^{-k} =\sum_{j=1}^{\infty}2^{-(j-1)}=2. $$ Thus \(\mathbb{E}X_n\to2=\mathbb{E}X\), as predicted. The direct expectation formula also checks the limit: \(2-2^{-(n-1)}\to2\).
Why Uniform Integrability Cannot Be Omitted
Worked Example: Rare Spikes with Nonvanishing Expectation
Again let \(\Omega=[0,1]\) with Lebesgue probability measure. Define \(Z_n(x)=n\chi_{(0,1/n)}(x)\). For each fixed \(\varepsilon>0\), when \(n>\varepsilon\) the event \(\{|Z_n|>\varepsilon\}\) is \((0,1/n)\), of measure \(1/n\). Hence \(Z_n\to0\) in probability.
Nevertheless, $$ \mathbb{E}Z_n=n\,m((0,1/n))=n\cdot\frac1n=1 $$ for every \(n\), whereas \(\mathbb{E}0=0\). The tail condition fails: for any fixed \(K>0\), choose an integer \(n>K\). Then \(|Z_n|=n>K\) throughout \((0,1/n)\), and $$ \mathbb{E}\bigl[|Z_n|\chi_{\{|Z_n|>K\}}\bigr] =n\cdot\frac1n=1. $$ Consequently the supremum of the tail expectations over \(n\) is at least \(1\) for every \(K\), so it does not tend to zero. Check: the probability of the spike is \(1/n\), its height is \(n\), and their product is \(1\); the expectation stays fixed even as the spike event becomes rare.
Common mistake. It is incorrect to infer uniform integrability merely from bounded expectations. In the spike example, \(\sup_n\mathbb{E}|Z_n|=1<\infty\), but the tail calculation shows that the family is not uniformly integrable. Bounded expectations control the total mass for each variable; uniform integrability requires the mass beyond the same threshold \(K\) to become small uniformly over all variables. A valid application of the theorem checks both convergence in probability and this uniform tail condition.
In applications, the theorem is useful whenever a probabilistic approximation gives convergence in probability but the quantity of interest is an expectation. One common route to the tail condition is a uniform \(L^p\) bound for some \(p>1\): if \(\sup_n\mathbb{E}|X_n|^p\le C<\infty\), then for \(K>0\), $$ \mathbb{E}\bigl[|X_n|\chi_{\{|X_n|>K\}}\bigr] \le K^{1-p}\mathbb{E}|X_n|^p \le CK^{1-p}. $$ The first inequality follows pointwise on \(\{|X_n|>K\}\) from \(|X_n|\le K^{1-p}|X_n|^p\). Since \(K^{1-p}\to0\), this bound is uniform in \(n\). Thus, when this moment hypothesis is available, convergence in probability implies convergence of expectations by the theorem. The moment condition is sufficient, not necessary; the theorem itself needs uniform integrability.
Check Your Understanding
For each question, identify the probability space and check the convergence and tail hypotheses before drawing a conclusion about expectations.
- On \([0,1]\) with Lebesgue probability measure, let \(Y_n(x)=x^{2n}\). Find its limit in probability and compute \(\mathbb{E}Y_n\).
- For the truncations \(X_n(k)=\min(k,n)\) on the countable probability space above, compute \(\mathbb{E}X_2\) directly from the point probabilities and compare it with the formula \(2-2^{-(n-1)}\).
- For the spike variables \(Z_n=n\chi_{(0,1/n)}\), compute \(\mathbb{E}[|Z_n|\chi_{\{|Z_n|>K\}}]\) when \(n>K\). What does this show about uniform integrability?
- Explain why convergence in probability alone does not justify passing to the limit in expectations, using the spike sequence.
- Suppose \(\sup_n\mathbb{E}|Y_n|^p<\infty\) for some \(p>1\), and \(Y_n\to Y\) in probability. State the tail estimate that verifies uniform integrability and the resulting expectation conclusion.