Conditional Expectation as an L2 Projection

Why Conditioning Has a Geometric Meaning

A conditional expectation retains the part of a random variable that can be expressed using the information in a sigma-algebra. When the random variable is square-integrable, this idea has a precise geometric form: conditional expectation is an orthogonal projection in the Hilbert space \(L^2\).

The key fact is an identity. If \(X\in L^2\), \(\mathcal G\) is a sigma-algebra, and \(P=\mathbb E[X\mid\mathcal G]\), then the residual \(X-P\) is orthogonal to every square-integrable, \(\mathcal G\)-measurable random variable. This tutorial derives that identity and explains why it makes \(P\) the orthogonal projection onto the \(\mathcal G\)-measurable part of \(L^2\).

The \(L^2\) Space and Its Conditioning-Measurable Subspace

Work on a probability space \((\Omega,\mathcal F,\mathbb P)\), with a sub-sigma-algebra \(\mathcal G\subseteq\mathcal F\). We use real-valued random variables and identify variables that are equal almost surely. The space \(L^2=L^2(\Omega,\mathcal F,\mathbb P)\) consists of these equivalence classes with finite second moment. Its inner product and norm are

\langle X,Y\rangle=\mathbb E[XY], \qquad \|X\|_2=\bigl(\mathbb E[X^2]\bigr)^{1/2}.

Define \(L^2(\mathcal G)\) to be the set of \(L^2\) equivalence classes that have a \(\mathcal G\)-measurable representative. This qualification matters: random variables are identified up to almost-sure equality, whereas measurability is a property of a representative.

Definition: A variable \(R\in L^2\) is orthogonal to a subspace \(H\subseteq L^2\) if \(\langle R,Y\rangle=0\) for every \(Y\in H\). The orthogonal complement consists of all variables orthogonal to every element of \(H\).

To call conditional expectation an orthogonal projection onto \(L^2(\mathcal G)\), we need this subspace to be closed. We also need the conditional expectation itself to belong to it. The latter follows from its \(\mathcal G\)-measurability and, for \(X\in L^2\), the \(L^2\) contraction property established in the earlier material on conditional expectation as an \(L^p\) contraction.

Proposition: \(L^2(\mathcal G)\) is a closed subspace of \(L^2\).

Proof. It is a subspace because sums and scalar multiples of \(\mathcal G\)-measurable representatives remain \(\mathcal G\)-measurable. To prove closedness, suppose \(Y_n\in L^2(\mathcal G)\) and \(Y_n\to Y\) in \(L^2\). Choose \(\mathcal G\)-measurable representatives for each \(Y_n\). There is a subsequence \((Y_{n_k})\) such that \[ \mathbb P\bigl(|Y_{n_k}-Y|>2^{-k}\bigr)<2^{-k}. \] Indeed, choose the subsequence so that \(\|Y_{n_k}-Y\|_2^2<2^{-3k}\), and apply Markov's inequality. Since the probabilities on the left have finite sum, the Borel–Cantelli lemma implies \(Y_{n_k}\to Y\) almost surely. The set on which the sequence of chosen representatives converges to a finite limit is \(\mathcal G\)-measurable. Define a variable there by that limit, and define it to be zero elsewhere. This gives a \(\mathcal G\)-measurable representative equal to \(Y\) almost surely. Therefore \(Y\in L^2(\mathcal G)\), so the subspace is closed. \(\square\)

From the Defining Identity to Orthogonality

For \(X\in L^2\), set \(P=\mathbb E[X\mid\mathcal G]\). The defining property of conditional expectation says, in particular, that

\mathbb E[P\mathbf 1_A]=\mathbb E[X\mathbf 1_A] \qquad\text{for every }A\in\mathcal G.

By linearity, the same identity holds with \(\mathbf 1_A\) replaced by any \(\mathcal G\)-measurable simple function. Approximation then gives the identity for bounded \(\mathcal G\)-measurable tests. To test against an arbitrary \(Y\in L^2(\mathcal G)\), we extend from bounded variables by truncation.

Theorem: If \(X\in L^2\), \(\mathcal G\subseteq\mathcal F\), and \(P=\mathbb E[X\mid\mathcal G]\), then \[ \mathbb E[(X-P)Y]=0 \qquad\text{for every }Y\in L^2(\mathcal G). \]

Proof. Since conditional expectation is an \(L^2\) contraction, \(P\in L^2(\mathcal G)\). Thus \(X-P\in L^2\). First let \(Y\) be bounded and \(\mathcal G\)-measurable. The defining identity for conditional expectation, extended from indicators to bounded measurable tests, gives \(\mathbb E[PY]=\mathbb E[XY]\), hence \(\mathbb E[(X-P)Y]=0\). Now let \(Y\) be any element of \(L^2(\mathcal G)\), and choose a \(\mathcal G\)-measurable representative. Define \(Y_n=\max(-n,\min(Y,n))\). Each \(Y_n\) is bounded and \(\mathcal G\)-measurable, and \(Y_n\to Y\) in \(L^2\), since \(|Y_n-Y|^2\leq |Y|^2\) and \(Y_n\to Y\) almost surely. Therefore, \[ \left|\mathbb E[(X-P)(Y-Y_n)]\right| \leq \|X-P\|_2\|Y-Y_n\|_2 \longrightarrow 0 \] by the Cauchy–Schwarz inequality. Since \(\mathbb E[(X-P)Y_n]=0\) for each \(n\), it follows that \(\mathbb E[(X-P)Y]=0\). \(\square\)

The theorem is the defining orthogonality identity. It says more than that the residual has mean zero: it has zero inner product with every square-integrable variable that is measurable with respect to the conditioning information.

Conditional Expectation as an Orthogonal Projection

In a Hilbert space, an orthogonal projection onto a closed subspace is characterized by two properties: the projected element lies in the subspace, and the difference from the original element is orthogonal to the whole subspace. The result just proved supplies the second property for conditional expectation.

Theorem: For \(X\in L^2\), the conditional expectation \(\mathbb E[X\mid\mathcal G]\) is the unique \(P\in L^2(\mathcal G)\) such that \(X-P\) is orthogonal to every element of \(L^2(\mathcal G)\). Thus conditional expectation is the orthogonal projection of \(X\) onto \(L^2(\mathcal G)\).

Proof. The conditional expectation \(P=\mathbb E[X\mid\mathcal G]\) belongs to \(L^2(\mathcal G)\), and the preceding theorem shows that \(X-P\) is orthogonal to every element of that subspace. This proves existence. For uniqueness, suppose \(Q\in L^2(\mathcal G)\) also has \(X-Q\) orthogonal to every element of \(L^2(\mathcal G)\). Subtracting the two orthogonality identities shows that \(P-Q\) is orthogonal to every element of \(L^2(\mathcal G)\). Because \(P-Q\) itself belongs to \(L^2(\mathcal G)\), we may test against \(P-Q\), obtaining \[ \|P-Q\|_2^2=\langle P-Q,P-Q\rangle=0. \] Hence \(P=Q\) almost surely. \(\square\)

Worked Examples

Worked Example: Projection onto a Finite-Partition Subspace

Let \(\Omega=\{1,2,3,4\}\) with equal probabilities, and let \(\mathcal G\) be generated by the partition \(\{\{1,2\},\{3,4\}\}\). Suppose \(X\) takes the values \(1,3,2,6\) at \(1,2,3,4\), respectively. Conditional expectation averages \(X\) within each partition cell, so

P=\mathbb E[X\mid\mathcal G]=(2,2,4,4).

Every \(Y\in L^2(\mathcal G)\) has values \((a,a,b,b)\) for some real \(a,b\). The residual is \(X-P=(-1,1,-2,2)\), and direct calculation gives

\mathbb E[(X-P)Y] =\frac14\bigl(-a+a-2b+2b\bigr)=0.

Thus the residual is orthogonal to every element of the conditioning-measurable subspace. The calculation verifies the projection identity, not just the conditional averages.

Worked Example: Conditioning a Standard Normal Variable on Its Sign

Let \(Z\) have the standard normal distribution and let \(\mathcal G=\sigma(\{Z\geq 0\})\). A \(\mathcal G\)-measurable variable is constant on each of the events \(\{Z\geq0\}\) and \(\{Z<0\}\), up to null sets. Symmetry gives each event probability \(1/2\), and integration of the normal density gives

\mathbb E[Z\mathbf 1_{\{Z\geq0\}}]=\frac{1}{\sqrt{2\pi}}, \qquad \mathbb E[Z\mathbf 1_{\{Z<0\}}]=-\frac{1}{\sqrt{2\pi}}.

The conditional means on the two events are therefore \(\sqrt{2/\pi}\) and \(-\sqrt{2/\pi}\), respectively. Writing \(c=\sqrt{2/\pi}\), we have

P=\mathbb E[Z\mid\mathcal G] =c\mathbf 1_{\{Z\geq0\}}-c\mathbf 1_{\{Z<0\}}.

For any \(Y\in L^2(\mathcal G)\), write \(Y=a\) on \(\{Z\geq0\}\) and \(Y=b\) on \(\{Z<0\}\). Then

\begin{aligned} \mathbb E[(Z-P)Y] &=a\left(\frac{1}{\sqrt{2\pi}}-\frac c2\right) +b\left(-\frac{1}{\sqrt{2\pi}}+\frac c2\right)\\ &=0, \end{aligned}

because \(c/2=1/\sqrt{2\pi}\). This confirms orthogonality for every test in the conditioning-measurable subspace.

Worked Example: An Independent Centered Summand

Let \(U,V\in L^2\) be independent, with \(\mathbb E[V]=0\), and set \(X=U+V\) and \(\mathcal G=\sigma(U)\). For every \(A\in\mathcal G\), independence implies

\mathbb E[X\mathbf 1_A] =\mathbb E[U\mathbf 1_A]+\mathbb E[V\mathbf 1_A] =\mathbb E[U\mathbf 1_A]+\mathbb E[V]\mathbb P(A) =\mathbb E[U\mathbf 1_A].

Since \(U\) is \(\mathcal G\)-measurable, these identities show that \(\mathbb E[X\mid\mathcal G]=U\). The residual is \(V\). For any \(Y\in L^2(\mathcal G)\), \(Y\) is independent of \(V\), and \(VY\) is integrable by Cauchy–Schwarz. Independence therefore gives

\mathbb E[(X-U)Y]=\mathbb E[VY]=\mathbb E[V]\mathbb E[Y]=0.

Here the information in \(U\) retains the \(U\) component, while the independent centered component is orthogonal to all square-integrable tests based on \(U\).

What the Projection Statement Does—and Does Not—Say

The orthogonality identity is useful because it converts conditional expectation into a statement about inner products. It also provides a uniqueness test: if a proposed \(\mathcal G\)-measurable \(P\in L^2\) has residual orthogonal to all of \(L^2(\mathcal G)\), then it must agree almost surely with \(\mathbb E[X\mid\mathcal G]\).

A common pitfall is to test orthogonality only against indicators of events in \(\mathcal G\) and stop there. Those tests express the defining conditional-expectation identity, but the Hilbert-space projection statement concerns every \(Y\in L^2(\mathcal G)\). The truncation argument is what justifies that extension without assuming the test variable is bounded.

The projection interpretation depends on square integrability: it uses the \(L^2\) inner product and Cauchy–Schwarz. Conditional expectation is defined in broader settings, but the specific orthogonal-projection statement belongs to \(L^2\). The next question—how this projection relates to a least-squares criterion—is a separate characterization.

Check Your Understanding

Use the definitions and identities in this tutorial to answer the following questions.

What are the two properties that characterize the orthogonal projection of \(X\) onto \(L^2(\mathcal G)\)?
Why is it necessary to extend the conditional-expectation identity beyond bounded \(\mathcal G\)-measurable tests?
How does truncation show that the orthogonality identity holds for every \(Y\in L^2(\mathcal G)\)?
In the finite-partition example, why does it suffice to check the residual against a variable with values \((a,a,b,b)\)?
Why does orthogonality of \(P-Q\) to \(L^2(\mathcal G)\), together with \(P-Q\in L^2(\mathcal G)\), imply uniqueness?