L2 Martingales as Hilbert Space Projections

Conditional Expectation as a Projection

Conditional expectation can be viewed as a geometric operation: it replaces a random variable by the closest square-integrable random variable that can be described using specified information. This perspective makes the behavior of conditional expectation martingales especially transparent. As information increases, the allowable approximations form larger subspaces, and the projections fit together through orthogonality.

Work on a probability space $(\Omega,\mathcal{F},\mathbb{P})$. The space $L^2(\mathcal{F})$ consists of square-integrable, real-valued random variables, identified when they are equal almost surely. Its inner product and norm are $\langle U,V\rangle=\mathbb{E}[UV]$ and $\|U\|_2=(\mathbb{E}[U^2])^{1/2}$. For a sigma-field $\mathcal{G}\subseteq\mathcal{F}$, $L^2(\mathcal{G})$ is the subspace of classes with a $\mathcal{G}$-measurable representative.

Definition: If $X\in L^2(\mathcal{F})$, its conditional expectation $\mathbb{E}[X\mid\mathcal{G}]$ is the $\mathcal{G}$-measurable integrable random variable whose integral over every event in $\mathcal{G}$ agrees with that of $X$. In $L^2$, it is the orthogonal projection of $X$ onto $L^2(\mathcal{G})$: it belongs to that subspace and $X-\mathbb{E}[X\mid\mathcal{G}]$ is orthogonal to every element of it.

The equivalence in this definition is the key fact. If $Y=\mathbb{E}[X\mid\mathcal{G}]$, then for every $A\in\mathcal{G}$, $\mathbb{E}[(X-Y)\mathbf{1}_A]=0$, by the defining property of conditional expectation. It follows first for bounded $\mathcal{G}$-measurable $Z$, by approximation with simple functions, that $\mathbb{E}[(X-Y)Z]=0$. For any $Z\in L^2(\mathcal{G})$, truncate $Z$ to bounded $\mathcal{G}$-measurable variables; these converge to $Z$ in $L^2$, and Cauchy–Schwarz gives the same identity in the limit. Conversely, if $Y\in L^2(\mathcal{G})$ and $\mathbb{E}[(X-Y)Z]=0$ for all $Z\in L^2(\mathcal{G})$, take $Z=\mathbf{1}_A$ for each $A\in\mathcal{G}$. Then $Y$ has the defining integrals of the conditional expectation, so it equals $\mathbb{E}[X\mid\mathcal{G}]$ almost surely.

Here is why the projection is the best approximation. For any $Z\in L^2(\mathcal{G})$, write $X-Z=(X-Y)+(Y-Z)$. The two terms are orthogonal, so expanding the squared norm gives $\|X-Z\|_2^2=\|X-Y\|_2^2+\|Y-Z\|_2^2$. Thus $Y$ minimizes the squared distance to $X$ among all such $Z$, and it is the unique minimizer up to almost-sure equality.

Nested Information and the Pythagorean Identity

Let $(\mathcal{F}_n)_{n\ge0}$ be an increasing filtration on the probability space: $\mathcal{F}_n\subseteq\mathcal{F}_{n+1}\subseteq\mathcal{F}$. For a fixed $X\in L^2(\mathcal{F})$, define $X_n=\mathbb{E}[X\mid\mathcal{F}_n]$ and $H_n=L^2(\mathcal{F}_n)$. Then $H_n\subseteq H_{n+1}$, so these are projections onto nested subspaces.

Theorem (Nested projection identity): For every $m\ge n$, $$ \|X_m-X_n\|_2^2=\|X_m\|_2^2-\|X_n\|_2^2. $$ In particular, $\|X_n\|_2^2$ is nondecreasing and bounded above by $\|X\|_2^2$.

Proof. Since $X_m$ is the projection of $X$ onto $H_m$, $X-X_m$ is orthogonal to $H_m$, and hence to $H_n$. Also, $X_m-X_n\in H_m$. For any $Z\in H_n$, both $\langle X_m,Z\rangle=\langle X,Z\rangle$ and $\langle X_n,Z\rangle=\langle X,Z\rangle$, so $\langle X_m-X_n,Z\rangle=0$. Taking $Z=X_n$ shows $X_m-X_n$ is orthogonal to $X_n$. Since $X_m=X_n+(X_m-X_n)$, expansion of its squared norm gives the stated identity. Finally, $X-X_n$ is orthogonal to $X_n$, so $\|X\|_2^2=\|X_n\|_2^2+\|X-X_n\|_2^2$; this proves the upper bound, and the identity proves monotonicity. $\square$

One consequence is that $(X_n)$ is Cauchy in $L^2$. Indeed, the nondecreasing sequence $\|X_n\|_2^2$ has a finite limit, and the identity makes $\|X_m-X_n\|_2^2$ tend to zero as $m,n$ tend to infinity. Completeness of $L^2$ then gives an $L^2$ limit. More precisely, that limit is the projection onto the information generated by the whole filtration.

Theorem (Limit of conditional expectation projections): Let $\mathcal{F}_\infty=\sigma(\bigcup_{n\ge0}\mathcal{F}_n)$, where the filtration is increasing, and let $X\in L^2(\mathcal{F})$. Then $$ \mathbb{E}[X\mid\mathcal{F}_n]\longrightarrow \mathbb{E}[X\mid\mathcal{F}_\infty] \quad\text{in }L^2. $$

Proof. Write $Y$ for the $L^2$ limit of $X_n$, whose existence follows from the Cauchy argument above. The subspace $L^2(\mathcal{F}_\infty)$ is closed, so $Y$ belongs to it: each $X_n$ is $\mathcal{F}_\infty$-measurable, and an $L^2$-convergent sequence has a subsequence converging almost surely, giving an $\mathcal{F}_\infty$-measurable representative of its limit.

It remains to show that $X-Y$ is orthogonal to all of $L^2(\mathcal{F}_\infty)$. If $Z\in L^2(\mathcal{F}_k)$, then for $n\ge k$, $Z\in H_n$, and the projection property gives $\langle X-X_n,Z\rangle=0$. Passing to the $L^2$ limit yields $\langle X-Y,Z\rangle=0$. To extend this from the union of these spaces to $L^2(\mathcal{F}_\infty)$, let $\mathcal{A}=\bigcup_n\mathcal{F}_n$. It is an algebra of events because the sigma-fields are nested. Every event in $\sigma(\mathcal{A})$ can be approximated in probability by events in $\mathcal{A}$: the class of events with this approximation property contains $\mathcal{A}$, is closed under complements, and is closed under countable unions by first approximating a countable union by a finite union and then approximating each event in that finite union. Consequently, indicators of events in $\mathcal{F}_\infty$ are limits in $L^2$ of indicators from $\mathcal{A}$, since the squared $L^2$ distance between two indicators is the probability of their symmetric difference. Finite linear combinations of such indicators are dense in $L^2(\mathcal{F}_\infty)$, by approximation with simple functions and truncation. Thus the union of the $H_n$ is dense in $L^2(\mathcal{F}_\infty)$. The orthogonality of $X-Y$ extends by continuity to that whole space. By the projection characterization, $Y=\mathbb{E}[X\mid\mathcal{F}_\infty]$, proving the result. $\square$

The sequence $X_n$ is also a martingale with respect to $(\mathcal{F}_n)$: it is integrable and $\mathcal{F}_n$-measurable, and for $m\ge n$, $\mathbb{E}[X_m\mid\mathcal{F}_n]=X_n$. This follows from the tower property of conditional expectation for nested sigma-fields. The projection argument explains its L2 convergence directly; the limit theorem above requires the fixed variable $X$ to be square-integrable and the sigma-fields to increase. L2 convergence also implies convergence in probability, since $\mathbb{P}(|X_n-Y|>\varepsilon)\le \|X_n-Y\|_2^2/\varepsilon^2$ for every $\varepsilon>0$.

Worked Example: Projection onto a Two-Set Partition

Let $\Omega=\{1,2,3\}$, give each outcome probability $1/3$, and let $\mathcal{G}$ distinguish the set $\{1,2\}$ from $\{3\}$. Suppose $X$ takes values $0,3,6$ at outcomes $1,2,3$, respectively. Find its projection onto $L^2(\mathcal{G})$ and check orthogonality.

A $\mathcal{G}$-measurable variable must have a common value on outcomes 1 and 2. The conditional average there is $(0+3)/2=3/2$; on outcome 3 it is $6$. Thus $Y=\mathbb{E}[X\mid\mathcal{G}]$ takes values $3/2,3/2,6$. The residual $X-Y$ takes values $-3/2,3/2,0$.

Any $Z\in L^2(\mathcal{G})$ has values $a,a,b$. Therefore $\langle X-Y,Z\rangle=\frac13((-3/2)a+(3/2)a+0b)=0$, which checks the projection condition. The norm check is $\|X\|_2^2=(0^2+3^2+6^2)/3=15$, $\|Y\|_2^2=((3/2)^2+(3/2)^2+6^2)/3=27/2$, and $\|X-Y\|_2^2=((3/2)^2+(3/2)^2)/3=3/2$. Hence $27/2+3/2=15$, as required by orthogonality.

Worked Example: Two Successive Projections on Four Outcomes

Let $\Omega=\{1,2,3,4\}$ have equal probabilities. Take $\mathcal{F}_0$ to be the trivial sigma-field, let $\mathcal{F}_1$ distinguish $\{1,2\}$ from $\{3,4\}$, and let $\mathcal{F}_2$ be the full sigma-field. For $X=(0,2,4,6)$, compute the conditional expectation martingale and its orthogonal increments.

The overall average is $X_0=3$. The two block averages are $1=(0+2)/2$ and $5=(4+6)/2$, so $X_1=(1,1,5,5)$. Since $\mathcal{F}_2$ reveals the outcome, $X_2=(0,2,4,6)$. The increments are $D_1=X_1-X_0=(-2,-2,2,2)$ and $D_2=X_2-X_1=(-1,1,-1,1)$.

Their inner product is $\langle D_1,D_2\rangle=(2-2-2+2)/4=0$. Their squared norms are $\|D_1\|_2^2=(4+4+4+4)/4=4$ and $\|D_2\|_2^2=(1+1+1+1)/4=1$. Also $\|X-X_0\|_2^2=((-3)^2+(-1)^2+1^2+3^2)/4=5$, so $5=4+1$: the total change from $X_0$ to $X_2$ splits into orthogonal changes. Finally, averaging $X_1$ within either $\mathcal{F}_1$ block gives $1$ or $5$, and averaging $X_2$ on those blocks gives the same values, verifying the martingale conditional-mean property.

Worked Example: Projections Revealing an Infinite Sequence

Let $(\xi_k)_{k\ge1}$ be independent fair Bernoulli random variables, so $\mathbb{P}(\xi_k=1)=\mathbb{P}(\xi_k=0)=1/2$. Let $\mathcal{F}_n=\sigma(\xi_1,\ldots,\xi_n)$, with $\mathcal{F}_0$ trivial, and set $X=\sum_{k=1}^{\infty}2^{-k}\xi_k$. Find $\mathbb{E}[X\mid\mathcal{F}_n]$ and verify its L2 convergence.

The series defining $X$ is bounded by $\sum_{k\ge1}2^{-k}=1$, so $X\in L^2$. Given $\mathcal{F}_n$, the first $n$ terms are known and each remaining $\xi_k$ still has conditional mean $1/2$, by independence. Therefore $$ X_n=\mathbb{E}[X\mid\mathcal{F}_n] =\sum_{k=1}^n2^{-k}\xi_k+\frac12\sum_{k=n+1}^{\infty}2^{-k} =\sum_{k=1}^n2^{-k}\xi_k+2^{-n-1}. $$

The difference $X-X_n$ is $\sum_{k>n}2^{-k}(\xi_k-1/2)$. Its summands are independent, have mean zero, and have variances $4^{-k}/4$, so $$ \|X-X_n\|_2^2=\frac14\sum_{k=n+1}^{\infty}4^{-k} =\frac14\cdot\frac{4^{-n}}{3} =\frac{4^{-n}}{12}. $$

As $n$ tends to infinity, $4^{-n}/12$ tends to zero, verifying convergence in L2. Here $X$ is measurable with respect to $\mathcal{F}_\infty=\sigma(\xi_1,\xi_2,\ldots)$, so the limit theorem says the limit is $\mathbb{E}[X\mid\mathcal{F}_\infty]=X$, in agreement with the direct error calculation.

What the Projection Limit Does—and Does Not—Say

The limit is determined by the information eventually available. It equals $X$ when $X$ is $\mathcal{F}_\infty$-measurable, but in general it is only $\mathbb{E}[X\mid\mathcal{F}_\infty]$. For example, if every $\mathcal{F}_n$ is trivial and $X$ takes values $-1$ and $1$ with equal probability, then every projection is $0$, not $X$. The filtration never reveals which value occurred.

Common pitfall: treating projections as pointwise increasing. In the four-outcome example, $X_0=3$ everywhere, while $X_1=(1,1,5,5)$. At outcomes 1 and 2 the projection changes from $3$ down to $1$; at outcomes 3 and 4 it changes up to $5$. The theorem says the subspaces are nested and the squared norms of the projections are nondecreasing; it does not say the random variables increase at each outcome.

The increasing-filtration assumption also matters. To see why, take a fair sign $X$, with values $-1$ and $1$, and alternate between the trivial sigma-field for even $n$ and the full sigma-field for odd $n$. The corresponding conditional expectations alternate between $0$ and $X$, so they do not converge in L2. This sequence of sigma-fields is not a filtration; the nested projection identity cannot be applied.

Check Your Understanding

Use orthogonality and the nested projection identity to justify your answers.

What condition on $X-Y$ characterizes $Y=\mathbb{E}[X\mid\mathcal{G}]$ among the elements of $L^2(\mathcal{G})$?
Why does $\|X_n\|_2^2$ increase when the sigma-fields $\mathcal{F}_n$ are nested?
In the four-outcome example, compute $\|X_2-X_0\|_2^2$ and compare it with the squared norms of the two increments.
What is the L2 limit of $\mathbb{E}[X\mid\mathcal{F}_n]$ if $X$ is not measurable with respect to $\mathcal{F}_\infty$?
Which assumption fails in the alternating trivial/full sigma-field example, and why does the projection sequence fail to converge?

L2 Martingales as Hilbert Space Projections

What You'll Learn