When Convergence Does Not Preserve Moments

Rare Events Can Carry Significant Moment

Convergence in probability says that large errors become unlikely. It does not say that the errors, when they occur, are small. A variable can therefore be close to its limit on almost every outcome and still have a nonvanishing contribution to its expectation from rare, increasingly large values. This distinction matters whenever a probabilistic approximation is used to infer an average, a variance, or another moment.

Work on a probability space $(\Omega,\mathcal{F},\mathbb{P})$. A sequence $(X_n)_{n\ge0}$ converges in probability to $X$ if, for every $\varepsilon>0$, $\mathbb{P}(|X_n-X|>\varepsilon)\to0$. It converges in $L^1$, or in mean, if $\mathbb{E}|X_n-X|\to0$; this requires the absolute differences to be integrable. Convergence in $L^1$ controls the average absolute error, whereas convergence in probability controls only the chance that the error exceeds a fixed threshold.

Definition: A family $\mathcal{X}$ of integrable random variables is uniformly integrable if $$ \lim_{K\to\infty}\ \sup_{Y\in\mathcal{X}}\mathbb{E}\bigl[|Y|\mathbf{1}_{\{|Y|>K\}}\bigr]=0. $$ The event in this definition is the strict tail event $|Y|>K$. Uniform integrability requires the expected contribution from large values to become small uniformly over the whole family.

The following examples use one common probability space: $\Omega=[0,1]$, with its Borel sets and uniform probability measure, and $U(\omega)=\omega$. Thus $\mathbb{P}(U\le a)=a$ for $0\le a\le1$. The index $n$ is a positive integer.

Worked Example: Rare Values Keep the Mean Away from the Limit

Set $X_n=n\mathbf{1}_{\{U\le1/n\}}$, and take $X=0$. For any fixed $\varepsilon>0$, whenever $n>\varepsilon$, the event $|X_n|>\varepsilon$ is exactly $\{U\le1/n\}$. Hence $$ \mathbb{P}(|X_n-X|>\varepsilon)=\frac1n\longrightarrow0. $$ So $X_n$ converges in probability to zero.

But $X_n=n$ on an event of probability $1/n$, and it is zero otherwise. Therefore $$ \mathbb{E}|X_n-X|=n\cdot\frac1n+0\cdot\left(1-\frac1n\right)=1, \qquad \mathbb{E}X_n=1. $$ The limit has expectation $\mathbb{E}X=0$. Thus neither $L^1$ convergence nor convergence of expectations holds. As a direct check, for every $K>0$, any integer $n>K$ gives $$ \mathbb{E}\bigl[|X_n|\mathbf{1}_{\{|X_n|>K\}}\bigr] =n\cdot\frac1n=1. $$ The family is not uniformly integrable: its worst tail contribution at threshold $K$ does not tend to zero.

This example separates two pieces of information. The probability of the nonzero value vanishes, but its magnitude grows at exactly the compensating rate. To predict expectations from convergence in probability, some additional control on large values is needed.

How the Scale of the Rare Event Changes the Moment

Changing the magnitude or probability of the exceptional value can change the moment conclusion, even while convergence in probability remains true. The next example alters only the rare-event scale.

Worked Example: A Smaller Rare-Event Contribution

On the same probability space, let $Y_n=n\mathbf{1}_{\{U\le1/n^2\}}$, with limit zero. For each $\varepsilon>0$, if $n>\varepsilon$, then $\{|Y_n|>\varepsilon\}=\{U\le1/n^2\}$. Consequently, $$ \mathbb{P}(|Y_n|>\varepsilon)=\frac1{n^2}\longrightarrow0. $$ The sequence converges in probability to zero. Its expectation and $L^1$ error are $$ \mathbb{E}Y_n=\mathbb{E}|Y_n| =n\cdot\frac1{n^2}+0\cdot\left(1-\frac1{n^2}\right) =\frac1n\longrightarrow0. $$ Thus it also converges in $L^1$, and its expectations converge to the expectation of its limit. The calculation identifies the difference from the previous example: the rare event now has probability $1/n^2$, so its contribution to the mean is $1/n$, not a constant.

Check: The probability of a nonzero value is $1/n^2$, and multiplying that probability by the value $n$ gives $1/n$, exactly the expectation computed above.

There is a useful implication in the other direction: convergence in $L^1$ always gives convergence of expectations. It does not require nonnegative variables, but it does require the relevant expectations to exist.

Proposition: If $X_n$ and $X$ are integrable and $\mathbb{E}|X_n-X|\to0$, then $\mathbb{E}X_n\to\mathbb{E}X$.

Proof. Linearity of expectation and the triangle inequality give $|\mathbb{E}X_n-\mathbb{E}X|=|\mathbb{E}(X_n-X)|\le\mathbb{E}|X_n-X|$. The right-hand side tends to zero by the assumed $L^1$ convergence, so the left-hand side tends to zero as well. $\square$

The converse implication fails even when the expectations agree exactly at every index. Equal expectations can result from cancellation between positive and negative rare values, while the absolute error remains large.

Worked Example: Cancellation Hides a Persistent Absolute Error

Define $$ Z_n=n\mathbf{1}_{\{U\le1/(2n)\}}-n\mathbf{1}_{\{1/(2n)<U\le1/n\}}, $$ and let $Z=0$. The two displayed events are disjoint, each has probability $1/(2n)$, and outside their union $Z_n=0$. For any $\varepsilon>0$ and $n>\varepsilon$, $|Z_n|>\varepsilon$ exactly on their union, which has probability $1/n$. Thus $Z_n\to0$ in probability.

The positive and negative contributions to the expectation cancel: $$ \mathbb{E}Z_n=n\cdot\frac1{2n}-n\cdot\frac1{2n}=0. $$ So $\mathbb{E}Z_n=\mathbb{E}Z=0$ for every $n$. However, the absolute value is $n$ on the union of probability $1/n$, so $$ \mathbb{E}|Z_n-Z|=n\cdot\frac1n=1. $$ Therefore expectations converge, but $L^1$ convergence fails. Taking the absolute value removes the cancellation and reveals the persistent error.

Check: The total probability of the two nonzero cases is $1/(2n)+1/(2n)=1/n$; multiplying by the absolute size $n$ gives $1$, as calculated.

Conditions That Prevent the Failure

One direct sufficient condition is a common bound on the variables. In the next result, “bounded” means that there is a finite constant $C$ such that $|X_n|\le C$ almost surely for every $n$. The limit need not be assumed bounded separately.

Theorem: If $X_n\to X$ in probability and $|X_n|\le C$ almost surely for every $n$, for some finite $C$, then $X$ is bounded by $C$ almost surely and $X_n\to X$ in $L^1$.

Proof. Fix $r>0$. On the event $|X|>C+r$, the inequality $|X_n|\le C$ implies $|X-X_n|>r$. Thus $\mathbb{P}(|X|>C+r)\le\mathbb{P}(|X-X_n|>r)$ for each $n$. The right-hand side tends to zero, so $\mathbb{P}(|X|>C+r)=0$. Taking $r=1/m$ for each positive integer $m$, the event $\{|X|>C\}$ is the countable union of the events $\{|X|>C+1/m\}$, each of probability zero. Hence $|X|\le C$ almost surely.

Now fix $\delta>0$, and put $A_n=\{|X_n-X|>\delta\}$. On the complement of $A_n$, the absolute difference is at most $\delta$; on $A_n$, it is at most $2C$ almost surely. Therefore $$ \mathbb{E}|X_n-X|\le\delta+2C\,\mathbb{P}(A_n). $$ For fixed $\delta$, convergence in probability makes the second term tend to zero. Hence $\limsup_{n\to\infty}\mathbb{E}|X_n-X|\le\delta$. Since this holds for every $\delta>0$ and the expectations are nonnegative, they tend to zero. $\square$

Uniform integrability provides a less restrictive condition: values need not share a fixed bound, provided the expected tails are uniformly controlled. The version below assumes integrability of the limit and includes it in the family whose tails are controlled.

Theorem: Suppose $X_n\to X$ in probability, $X$ and all $X_n$ are integrable, and the family $\{X,X_1,X_2,\ldots\}$ is uniformly integrable. Then $X_n\to X$ in $L^1$.

Proof. Let $\eta>0$. By uniform integrability, choose $K>0$ so that, for every $n$, $$ \mathbb{E}\bigl[|X_n|\mathbf{1}_{\{|X_n|>K\}}\bigr]<\eta, \qquad \mathbb{E}\bigl[|X|\mathbf{1}_{\{|X|>K\}}\bigr]<\eta. $$ Fix $\delta>0$, and write $A_n=\{|X_n-X|>\delta\}$. On the complement of $A_n$, $|X_n-X|\le\delta$. On $A_n$, use $|X_n-X|\le|X_n|+|X|$. For either $Y=X_n$ or $Y=X$, $$ \mathbb{E}[|Y|\mathbf{1}_{A_n}] \le \mathbb{E}[|Y|\mathbf{1}_{\{|Y|>K\}}]+K\mathbb{P}(A_n), $$ because on $\{|Y|\le K\}$ the variable $|Y|$ is at most $K$. Adding the two bounds gives $$ \mathbb{E}|X_n-X|\le\delta+2\eta+2K\mathbb{P}(A_n). $$ For fixed $K$ and $\delta$, convergence in probability makes the last term tend to zero. Thus $\limsup_n\mathbb{E}|X_n-X|\le\delta+2\eta$. Since $\delta$ and $\eta$ can be arbitrarily small, $\mathbb{E}|X_n-X|\to0$, as required. $\square$

The proof shows the two roles in the uniform-integrability condition. Away from the event $A_n$, the error is small by choosing $\delta$ small. On $A_n$, convergence in probability makes the event rare, but that alone is not enough: the tail terms must also be small uniformly in $n$. The first rare-event example fails precisely this tail-control test.

Worked Example: A Bounded Rare-Event Sequence

Let $W_n=\mathbf{1}_{\{U\le1/n\}}$ and $W=0$. For $0<\varepsilon<1$, the event $|W_n-W|>\varepsilon$ is $\{U\le1/n\}$, with probability $1/n$; for $\varepsilon\ge1$, it is empty because the error is at most $1$. Hence $W_n\to0$ in probability.

For every $n$, $0\le W_n\le1$, and direct computation gives $$ \mathbb{E}|W_n-W|=1\cdot\frac1n+0\cdot\left(1-\frac1n\right)=\frac1n\longrightarrow0. $$ The expectations converge as well: $\mathbb{E}W_n=1/n\to0=\mathbb{E}W$. This is consistent with the boundedness theorem. Also, for every $K\ge1$, the strict-tail event $\{|W_n|>K\}$ is empty, so the tail expectations are zero. That verifies uniform integrability directly.

Check: The largest possible error is $1$, and it occurs with probability $1/n$; therefore the mean absolute error is exactly $1/n$, not a fixed positive constant.

A Common Mistake and a Practical Test

Common mistake: “$X_n\to0$ in probability, so $\mathbb{E}X_n\to0$.” Applying this incorrectly to $X_n=n\mathbf{1}_{\{U\le1/n\}}$, one might say that $X_n$ is zero with probability $1-1/n$, and therefore its expectation is zero in the limit. The missing calculation is the contribution on the exceptional event: $$ \mathbb{E}X_n=n\cdot\frac1n=1, $$ not zero. The probability of the event is small, but the value on it is large enough to preserve the mean.

When a limit in probability is used to make a claim about expectations, check the absolute error rather than only the probability of a large error. Either prove $L^1$ convergence directly, or establish an adequate tail condition, such as a common bound or uniform integrability. Also distinguish convergence of signed expectations from convergence in $L^1$: the former can conceal cancellation, as the $Z_n$ example demonstrates.

Check Your Understanding

For each question, use the probability of the exceptional event and the size of the variable on that event to check the moment calculation.

For $R_n=n^2\mathbf{1}_{\{U\le1/n^2\}}$, does $R_n$ converge in probability to zero? Compute $\mathbb{E}R_n$ and decide whether it converges to zero.
For $V_n=\sqrt{n}\mathbf{1}_{\{U\le1/n\}}$, compute $\mathbb{P}(|V_n|>\varepsilon)$ for all sufficiently large $n$, and compute $\mathbb{E}|V_n|$.
State why convergence in $L^1$ implies convergence of expectations, including the inequality that proves it.
In the uniform-integrability theorem, identify where convergence in probability is used and where uniform integrability is used.
For the cancellation example $Z_n$, why does $\mathbb{E}Z_n=0$ not imply $\mathbb{E}|Z_n|\to0$?