Conditional Expectation Martingales and Their Limits

Conditional Expectations as Information Accumulates

Suppose an integrable random variable $X$ is observed through a sequence of increasingly informative sigma-fields. At stage $n$, the quantity $\mathbb{E}[X\mid\mathcal{F}_n]$ is the best conditional-mean description of $X$ using the information then available. The central question is whether these stage-by-stage conditional means converge, and what their limit represents. The answer is the conditional expectation given all information in the sequence, with convergence in mean absolute error.

Work on a probability space $(\Omega,\mathcal{F},\mathbb{P})$. Let $(\mathcal{F}_n)_{n\ge0}$ be an increasing sequence of sub-sigma-fields of $\mathcal{F}$, meaning $\mathcal{F}_n\subseteq\mathcal{F}_{n+1}$. Define the limiting sigma-field by $\mathcal{F}_\infty=\sigma(\bigcup_{n\ge0}\mathcal{F}_n)$. For an integrable, real-valued $X$, set $M_n=\mathbb{E}[X\mid\mathcal{F}_n]$. Each $M_n$ is integrable and $\mathcal{F}_n$-measurable, and conditional expectation is understood up to almost-sure equality.

Theorem: For any integrable $X$ and increasing sequence $(\mathcal{F}_n)$, the process $(M_n)_{n\ge0}$ is a martingale with respect to $(\mathcal{F}_n)$, and $$ \lim_{n\to\infty}\mathbb{E}\left[\left|M_n-\mathbb{E}[X\mid\mathcal{F}_\infty]\right|\right]=0. $$ In particular, the limit is the conditional expectation given the sigma-field generated by all the $\mathcal{F}_n$, not necessarily $X$ itself.

Here $L^1$ convergence means that the expected absolute difference tends to zero. The martingale assertion follows from the tower property: conditioning $M_{n+1}$ on $\mathcal{F}_n$ gives $M_n$. The identification and convergence of the limit require an additional approximation argument, which we give in full.

Approximating the Information at Infinity

The key fact is that an integrable random variable measurable with respect to $\mathcal{F}_\infty$ can be approximated in $L^1$ by variables measurable at some finite stage. The proof begins with events. Put $\mathcal{A}=\bigcup_{n\ge0}\mathcal{F}_n$. Since the sigma-fields are increasing, $\mathcal{A}$ is an algebra of sets: any two of its members belong together to some $\mathcal{F}_N$, so their union, intersection, and complements also belong to $\mathcal{A}$.

Approximation fact: If $Y$ is integrable and $\mathcal{F}_\infty$-measurable, then for every $\varepsilon>0$ there are an integer $N$ and an integrable, $\mathcal{F}_N$-measurable $Z$ such that $\mathbb{E}[|Y-Z|]<\varepsilon$.

Proof. First consider events. Let $\mathcal{D}$ be the class of sets $E\in\mathcal{F}_\infty$ such that for every $\varepsilon>0$, some $A\in\mathcal{A}$ satisfies $\mathbb{P}(E\mathbin{\triangle}A)<\varepsilon$, where $\triangle$ denotes symmetric difference. Every set in $\mathcal{A}$ belongs to $\mathcal{D}$. If $E\in\mathcal{D}$, then $\mathbb{P}(E^c\mathbin{\triangle}A^c)=\mathbb{P}(E\mathbin{\triangle}A)$, and $A^c\in\mathcal{A}$; hence $E^c\in\mathcal{D}$. Now take $E_1,E_2,\ldots\in\mathcal{D}$ and $E=\bigcup_{j\ge1}E_j$. By continuity of probability for increasing unions, choose $m$ such that $\mathbb{P}(E\setminus\bigcup_{j=1}^m E_j)<\varepsilon/2$. For each $1\le j\le m$, choose $A_j\in\mathcal{A}$ with $\mathbb{P}(E_j\mathbin{\triangle}A_j)<\varepsilon/(2m)$. The finite union $A=\bigcup_{j=1}^m A_j$ belongs to $\mathcal{A}$, and $$ \mathbb{P}(E\mathbin{\triangle}A) \le \mathbb{P}\left(E\setminus\bigcup_{j=1}^m E_j\right) +\sum_{j=1}^m\mathbb{P}(E_j\mathbin{\triangle}A_j) <\varepsilon. $$ Thus $\mathcal{D}$ is a sigma-field. It contains $\mathcal{A}$, so it contains $\sigma(\mathcal{A})=\mathcal{F}_\infty$. Therefore every event in $\mathcal{F}_\infty$ can be approximated in probability by a set in some finite-stage sigma-field.

To pass from events to $Y$, first approximate $Y$ in $L^1$ by a finite-valued, $\mathcal{F}_\infty$-measurable simple function $V=\sum_{j=1}^r c_j\mathbf{1}_{E_j}$. Such approximation follows by truncating the integrable variable and partitioning its bounded range into intervals of sufficiently small length. For each $E_j$, choose $A_j\in\mathcal{A}$ so that $\mathbb{P}(E_j\mathbin{\triangle}A_j)$ is small. Then $$ \mathbb{E}\left[\left|V-\sum_{j=1}^r c_j\mathbf{1}_{A_j}\right|\right] \le \sum_{j=1}^r |c_j|\mathbb{P}(E_j\mathbin{\triangle}A_j). $$ Choose the event-approximation errors small enough that this bound, together with $\mathbb{E}[|Y-V|]$, is less than $\varepsilon$. There are only finitely many $A_j$, and each belongs to some $\mathcal{F}_{N_j}$; taking $N=\max_j N_j$ makes their linear combination $\mathcal{F}_N$-measurable. It is integrable because it is a finite-valued simple function. This proves the approximation fact. $\square$

Proof of the L1 Limit

We use the contraction property of conditional expectation: for integrable $U$ and a sigma-field $\mathcal{G}$, $$ \mathbb{E}\left[\left|\mathbb{E}[U\mid\mathcal{G}]\right|\right]\le \mathbb{E}[|U|]. $$ Indeed, positivity and the defining properties of conditional expectation give $|\mathbb{E}[U\mid\mathcal{G}]|\le\mathbb{E}[|U|\mid\mathcal{G}]$ almost surely; taking expectations gives the inequality.

Proof of the theorem. The variables $M_n$ are integrable and $\mathcal{F}_n$-measurable by the definition of conditional expectation. For $A\in\mathcal{F}_n$, the tower property, or equivalently the defining integral identity on $A$, gives $$ \mathbb{E}[\mathbf{1}_A M_{n+1}] =\mathbb{E}[\mathbf{1}_A X] =\mathbb{E}[\mathbf{1}_A M_n]. $$ Since $M_n$ is $\mathcal{F}_n$-measurable, this proves $\mathbb{E}[M_{n+1}\mid\mathcal{F}_n]=M_n$ almost surely. Thus $(M_n)$ is a martingale.

Now put $Y=\mathbb{E}[X\mid\mathcal{F}_\infty]$. It is integrable and $\mathcal{F}_\infty$-measurable. Because $\mathcal{F}_n\subseteq\mathcal{F}_\infty$, the tower property gives $$ M_n=\mathbb{E}[X\mid\mathcal{F}_n] =\mathbb{E}[Y\mid\mathcal{F}_n]\quad\text{almost surely}. $$ Fix $\varepsilon>0$. By the approximation fact, choose $N$ and an integrable, $\mathcal{F}_N$-measurable $Z$ with $\mathbb{E}[|Y-Z|]<\varepsilon$. For $n\ge N$, $Z$ is also $\mathcal{F}_n$-measurable, so $\mathbb{E}[Z\mid\mathcal{F}_n]=Z$ almost surely. Therefore, using the contraction property, $$ \begin{aligned} \mathbb{E}[|M_n-Y|] &=\mathbb{E}\left[\left|\mathbb{E}[Y\mid\mathcal{F}_n]-Y\right|\right]\\ &\le \mathbb{E}\left[\left|\mathbb{E}[Y-Z\mid\mathcal{F}_n]\right|\right] +\mathbb{E}\left[\left|\mathbb{E}[Z\mid\mathcal{F}_n]-Y\right|\right]\\ &\le \mathbb{E}[|Y-Z|]+\mathbb{E}[|Z-Y|] <2\varepsilon. \end{aligned} $$ Since this holds for every $\varepsilon>0$ and all $n\ge N$ for the selected approximation, $\mathbb{E}[|M_n-Y|]\to0$. This proves the asserted $L^1$ convergence and identifies its limit. $\square$

There is also an almost-sure conclusion. The standard martingale convergence theorem under uniform integrability, established in the earlier material on that theorem, states that a uniformly integrable martingale converges almost surely and in $L^1$ to an integrable limit. The standard result that conditional expectations of a fixed integrable variable form a uniformly integrable family applies to $\{\mathbb{E}[X\mid\mathcal{F}_n]:n\ge0\}$. Hence $M_n$ converges almost surely as well. Its almost-sure limit agrees with $Y$ almost surely, because the $L^1$ convergence already proved implies convergence in probability, and limits in probability are unique up to almost-sure equality.

Worked Examples

Worked Example: One Reveal and the Unconditional Martingale Check

Let $\Omega=\{\omega_1,\omega_2,\omega_3,\omega_4\}$, with each outcome having probability $\tfrac14$. Let $X$ take values $1,1,5,5$, respectively. Let $\mathcal{F}_0=\{\varnothing,\Omega\}$, and let $\mathcal{F}_n$ for $n\ge1$ reveal the partition $\{\{\omega_1,\omega_2\},\{\omega_3,\omega_4\}\}$. The real-valued process is $M_n=\mathbb{E}[X\mid\mathcal{F}_n]$.

At stage zero, the information is trivial, so $$ M_0=\mathbb{E}[X]=\frac{1+1+5+5}{4}=3. $$ At stage one, the conditional mean on the first block is $(1+1)/2=1$, and on the second block it is $(5+5)/2=5$. Thus $M_1$ has values $1,1,5,5$, and $M_n=M_1$ for $n\ge1$. The martingale check from stage zero is the unconditional average: $$ \mathbb{E}[M_1]=\frac{1+1+5+5}{4}=3=M_0. $$ Since $\mathcal{F}_\infty=\mathcal{F}_1$, the limit is $\mathbb{E}[X\mid\mathcal{F}_\infty]=M_1$, and the $L^1$ error is zero for every $n\ge1$.

Check: the two block averages are $1$ and $5$, neither of which equals $M_0=3$. The martingale condition at time zero compares $M_0$ with the unconditional mean of $M_1$, which is $3$.

Worked Example: Gradually Revealing the Entire Variable

Use the same four equally likely outcomes, with $X(\omega_1)=0$, $X(\omega_2)=2$, $X(\omega_3)=4$, and $X(\omega_4)=6$. Let $\mathcal{F}_0$ be trivial, let $\mathcal{F}_1$ reveal the two blocks $\{\omega_1,\omega_2\}$ and $\{\omega_3,\omega_4\}$, and let $\mathcal{F}_2$ reveal each outcome. For $n\ge2$, keep $\mathcal{F}_n=\mathcal{F}_2$.

The initial mean is $M_0=(0+2+4+6)/4=3$. The two block means at stage one are $(0+2)/2=1$ and $(4+6)/2=5$, so $M_1$ has values $1,1,5,5$. At stage two, $M_2=X$, since the outcome is known. The stage-zero martingale check is $(1+1+5+5)/4=3=M_0$. At stage one, conditional on the first block, the mean of the next values $0,2$ is $1$; conditional on the second block, the mean of $4,6$ is $5$. Hence $\mathbb{E}[M_2\mid\mathcal{F}_1]=M_1$.

Here $\mathcal{F}_\infty=\mathcal{F}_2$, so $\mathbb{E}[X\mid\mathcal{F}_\infty]=X$. The errors are $$ \mathbb{E}[|M_1-X|]=\frac{1+1+1+1}{4}=1,\qquad \mathbb{E}[|M_2-X|]=0. $$ Thus the sequence reaches its limiting conditional expectation at stage two.

Check: the mean of $M_1$ is $(1+1+5+5)/4=3$, the mean of $M_2$ is $(0+2+4+6)/4=3$, and both equal $M_0$, as required by repeated conditioning.

Worked Example: The Limit Retains Only Partial Information

Again take four equally likely outcomes, now with $X$ taking values $0,2,4,8$. Let $\mathcal{F}_n$ for every $n\ge0$ be the sigma-field generated by the two blocks $\{\omega_1,\omega_2\}$ and $\{\omega_3,\omega_4\}$. The filtration never distinguishes outcomes within a block, so $\mathcal{F}_\infty$ is this same two-block sigma-field.

The mean of $X$ is $(0+2+4+8)/4=14/4=7/2$, so $M_0=7/2$ on all outcomes. The conditional mean on the first block is $(0+2)/2=1$; on the second it is $(4+8)/2=6$. Consequently $M_n$ has values $1,1,6,6$ for every $n$, and this is $\mathbb{E}[X\mid\mathcal{F}_\infty]$. The $L^1$ difference from $X$ is $$ \mathbb{E}[|X-M_n|]=\frac{|0-1|+|2-1|+|4-6|+|8-6|}{4} =\frac{1+1+2+2}{4}=\frac32. $$ This error does not decrease with $n$, because no new information is revealed. The limit is not $X$, but its conditional mean given the information that is available.

Check: within the first block the residuals are $-1,1$, with average $0$; within the second they are $-2,2$, also with average $0$. This verifies the defining conditional-mean property on both blocks. Also $\mathbb{E}[M_n]=(1+1+6+6)/4=14/4=7/2=\mathbb{E}[X]$.

Why the Limit Is Useful—and a Common Mistake

The theorem separates two ideas that are easy to conflate. The martingale property says that the current conditional mean is the conditional expectation of the next one given current information. The limit statement says that as the available information grows, these conditional means approach the conditional expectation using the entire generated sigma-field. Only if that sigma-field determines $X$ (up to almost-sure equality) does the limit equal $X$. If information remains hidden, as in the third example, the limit averages over what remains unknown.

A common mistake is to test the martingale condition by comparing each block average of $M_1$ with the value of $M_0$ on that block, even when $\mathcal{F}_0$ is trivial. In the first example, the block averages are $1$ and $5$, while $M_0=3$ everywhere. Neither block average equals $3$, but this does not violate the martingale property: conditioning on the trivial sigma-field takes the unconditional average, $(1+1+5+5)/4=3$. The correct test is $\mathbb{E}[M_1\mid\mathcal{F}_0]=M_0$, not equality of each finer-block mean with $M_0$. For a nontrivial $\mathcal{F}_0$, the corresponding conditional average must be checked separately on each atom of $\mathcal{F}_0$.

Check Your Understanding

For each question, identify the sigma-field that supplies the relevant information and distinguish conditional averages from unconditional ones.

Why is $\bigcup_{n\ge0}\mathcal{F}_n$ an algebra when the sigma-fields are increasing, but not necessarily itself a sigma-field?
In the first worked example, compute $\mathbb{E}[M_1\mid\mathcal{F}_0]$. Why is it not necessary for each block mean of $M_1$ to equal $M_0$?
If $X$ is $\mathcal{F}_N$-measurable, what is $\mathbb{E}[X\mid\mathcal{F}_\infty]$, and what are $M_n$ for $n\ge N$?
In the third worked example, calculate the mean residual $X-M_n$ separately on each of the two blocks.
Which inequality in the $L^1$ convergence proof controls the difference between $\mathbb{E}[Y\mid\mathcal{F}_n]$ and $Y$, and what approximation makes its bound small?