Radon–Nikodym Construction of Conditional Expectation

Why the Radon–Nikodym theorem enters conditioning

A conditional expectation with respect to a sub-sigma-algebra must be measurable using only the events in that sub-sigma-algebra, while preserving the integrals of the original random variable over those events. The Radon–Nikodym theorem provides exactly the mechanism for constructing such a measurable function: it represents one measure as a density with respect to another.

The key is to use the right measurable space. If $\mathcal{G}$ is a sub-sigma-algebra of $\mathcal{F}$, we work on $(\Omega,\mathcal{G})$ and use the restriction of $\mathbb{P}$ to $\mathcal{G}$ as the reference measure. The random variable determines a second measure on that same space.

Definition: Let $(\Omega,\mathcal{F},\mathbb{P})$ be a probability space, let $\mathcal{G}\subseteq\mathcal{F}$ be a sub-sigma-algebra, and let $X$ be a nonnegative, $\mathcal{F}$-measurable random variable with $\mathbb{E}[X]<\infty$. A conditional expectation of $X$ given $\mathcal{G}$ is a nonnegative, integrable, $\mathcal{G}$-measurable random variable $Y$ such that, for every $A\in\mathcal{G}$, the integral of $Y$ over $A$ equals the integral of $X$ over $A$: $$ \int_A Y\,d\mathbb{P}=\int_A X\,d\mathbb{P}. $$ We write $Y=\mathbb{E}[X\mid\mathcal{G}]$ for a random variable obtained by this construction.

For a signed integrable random variable, the same definition applies with “nonnegative” omitted: the conditional expectation must be integrable and $\mathcal{G}$-measurable, and the integral identity must hold for every $A\in\mathcal{G}$. We will first construct the nonnegative case, where the measure induced by $X$ is positive, and then use positive and negative parts for the signed case.

The measure induced on the sub-sigma-algebra

Define $\mathbb{P}_{\mathcal{G}}$ to be the restriction of $\mathbb{P}$ to $\mathcal{G}$, so $\mathbb{P}_{\mathcal{G}}(A)=\mathbb{P}(A)$ for $A\in\mathcal{G}$. From $X$, define a set function $\nu$ on $\mathcal{G}$ by

\nu(A)=\int_A X\,d\mathbb{P},\qquad A\in\mathcal{G}.

The function $X$ need not be $\mathcal{G}$-measurable. That causes no difficulty: the integral is taken with respect to $\mathbb{P}$ on the original space, and only the sets $A$ are restricted to $\mathcal{G}$.

Proposition: If $X\geq 0$ and $\mathbb{E}[X]<\infty$, then $\nu$ is a finite measure on $(\Omega,\mathcal{G})$, and $\nu$ is absolutely continuous with respect to $\mathbb{P}_{\mathcal{G}}$.

Proof. First, $\nu(\varnothing)=0$, since the integral over the empty set is zero. Let $A_1,A_2,\ldots$ be pairwise disjoint sets in $\mathcal{G}$. Their union is in $\mathcal{G}$, and countable additivity of the integral for a nonnegative function gives $$ \nu\!\left(\bigcup_{n=1}^{\infty}A_n\right) =\int_{\bigcup_{n=1}^{\infty}A_n}X\,d\mathbb{P} =\sum_{n=1}^{\infty}\int_{A_n}X\,d\mathbb{P} =\sum_{n=1}^{\infty}\nu(A_n). $$ Thus $\nu$ is a measure. Also, $$ \nu(\Omega)=\int_\Omega X\,d\mathbb{P}=\mathbb{E}[X]<\infty, $$ so it is finite. Finally, if $A\in\mathcal{G}$ and $\mathbb{P}_{\mathcal{G}}(A)=0$, then $\mathbb{P}(A)=0$, and therefore $\nu(A)=\int_A X\,d\mathbb{P}=0$. This proves $\nu\ll\mathbb{P}_{\mathcal{G}}$. $\square$

Applying Radon–Nikodym

The Radon–Nikodym theorem says that if $\nu$ and $\mu$ are finite measures on the same measurable space and $\nu\ll\mu$, then there is a nonnegative measurable function $h$ such that $\nu(A)=\int_A h\,d\mu$ for every measurable set $A$. Apply it on $(\Omega,\mathcal{G})$, with $\mu=\mathbb{P}_{\mathcal{G}}$ and with $\nu$ as above. The density is $\mathcal{G}$-measurable by the theorem, and its integral identity is the defining condition for conditional expectation.

Theorem: If $X$ is nonnegative, $\mathcal{F}$-measurable, and integrable, then there exists a nonnegative, integrable, $\mathcal{G}$-measurable random variable $Y$ such that, for every $A\in\mathcal{G}$, $$ \int_A Y\,d\mathbb{P}=\int_A X\,d\mathbb{P}. $$

Proof. By the proposition, the finite measure $\nu$ on $(\Omega,\mathcal{G})$ is absolutely continuous with respect to $\mathbb{P}_{\mathcal{G}}$. The Radon–Nikodym theorem therefore supplies a nonnegative, $\mathcal{G}$-measurable function $Y$ satisfying $$ \nu(A)=\int_A Y\,d\mathbb{P}_{\mathcal{G}},\qquad A\in\mathcal{G}. $$ Since $\mathbb{P}_{\mathcal{G}}$ agrees with $\mathbb{P}$ on $\mathcal{G}$, this is precisely $$ \int_A Y\,d\mathbb{P}=\int_A X\,d\mathbb{P},\qquad A\in\mathcal{G}. $$ Taking $A=\Omega$ gives $$ \int_\Omega Y\,d\mathbb{P}=\int_\Omega X\,d\mathbb{P}<\infty. $$ Because $Y\geq 0$, this also shows that $Y$ is integrable. It has all the required properties. $\square$

This argument does not claim that $X$ itself is measurable with respect to $\mathcal{G}$. Instead, the Radon–Nikodym density is $\mathcal{G}$-measurable and matches $X$'s integrals on the events available in $\mathcal{G}$. This distinction is the heart of the construction.

Worked Example: Conditioning on the trivial sigma-algebra

Let $\Omega=[0,1]$ with Lebesgue probability measure, let $X(x)=2x$, and let $\mathcal{G}=\{\varnothing,\Omega\}$. The induced measure on $\mathcal{G}$ has total mass $$ \nu(\Omega)=\int_0^1 2x\,dx=1. $$ A $\mathcal{G}$-measurable function is constant, so the density $Y(x)=1$ is a candidate. Indeed, for $A=\Omega$, $$ \int_A Y\,d\mathbb{P}=1=\int_0^1 2x\,dx=\int_A X\,d\mathbb{P}, $$ and for $A=\varnothing$, both integrals are zero. These are all the sets in $\mathcal{G}$, so $Y=1$ satisfies the defining condition.

Worked Example: A product-space density

Let $\Omega=[0,1]^2$ with product Lebesgue probability measure, write points as $(x,y)$, and let $\mathcal{G}$ be the sigma-algebra generated by the first-coordinate projection. Thus its sets are $C\times[0,1]$ for Borel sets $C\subseteq[0,1]$. Take $X(x,y)=4xy$. This variable is nonnegative and integrable. For $A=C\times[0,1]$, Fubini's theorem gives $$ \nu(A)=\int_C\int_0^1 4xy\,dy\,dx =\int_C 4x\left(\int_0^1 y\,dy\right)dx =\int_C 2x\,dx. $$ Define $Y(x,y)=2x$. It is $\mathcal{G}$-measurable, and $$ \int_A Y\,d\mathbb{P} =\int_C\int_0^1 2x\,dy\,dx =\int_C 2x\,dx =\nu(A). $$ The equality holds for every $C$, hence for every $A\in\mathcal{G}$. The Radon–Nikodym density is therefore represented here by the function $2x$, which retains the first-coordinate dependence and averages out the second-coordinate factor.

Extending the construction to signed variables

Let $X$ now be real-valued, $\mathcal{F}$-measurable, and integrable, meaning $\mathbb{E}[|X|]<\infty$. Its positive and negative parts are

X^+=\max(X,0),\qquad X^-=\max(-X,0),\qquad X=X^+-X^-.

Both $X^+$ and $X^-$ are nonnegative and integrable. Apply the nonnegative construction separately to them. If $U$ is a conditional expectation of $X^+$ given $\mathcal{G}$, and $V$ is a conditional expectation of $X^-$ given $\mathcal{G}$, then the difference $U-V$ is the natural candidate for the conditional expectation of $X$.

Theorem: For every integrable real-valued $X$, there exists an integrable, $\mathcal{G}$-measurable random variable $Z$ such that, for every $A\in\mathcal{G}$, $$ \int_A Z\,d\mathbb{P}=\int_A X\,d\mathbb{P}. $$

Proof. Construct $U$ and $V$ by applying the nonnegative theorem to $X^+$ and $X^-$, respectively. Set $Z=U-V$. Since $U$ and $V$ are $\mathcal{G}$-measurable, so is $Z$. Also, the inequality $|U-V|\leq U+V$ and their integrability give $$ \int_\Omega |Z|\,d\mathbb{P} \leq \int_\Omega U\,d\mathbb{P}+\int_\Omega V\,d\mathbb{P} =\int_\Omega X^+\,d\mathbb{P}+\int_\Omega X^-\,d\mathbb{P} =\mathbb{E}[|X|]<\infty. $$ Thus $Z$ is integrable. For each $A\in\mathcal{G}$, the defining identities for $U$ and $V$ imply $$ \int_A Z\,d\mathbb{P} =\int_A U\,d\mathbb{P}-\int_A V\,d\mathbb{P} =\int_A X^+\,d\mathbb{P}-\int_A X^-\,d\mathbb{P} =\int_A X\,d\mathbb{P}. $$ Hence $Z$ satisfies the required condition. $\square$

Worked Example: A signed variable with zero conditional expectation

On the product space $[0,1]^2$ with product Lebesgue probability measure, let $\mathcal{G}$ again be generated by the first coordinate and set $$ X(x,y)=2x(2y-1). $$ This is signed and integrable because $|X(x,y)|\leq 2$. For $A=C\times[0,1]$, direct integration yields $$ \int_A X\,d\mathbb{P} =\int_C 2x\,dx\int_0^1(2y-1)\,dy =\int_C 2x\,dx\,(1-1) =0. $$ The zero function is $\mathcal{G}$-measurable and integrable, and its integral over every such $A$ is also zero. Since every event in $\mathcal{G}$ has this form, the zero function satisfies the conditional-expectation identity. In the signed construction, the positive and negative parts each contribute a nonnegative density; subtracting those densities allows their contributions to cancel.

What the construction does—and does not—say

The construction is useful because it turns a family of integral constraints into a density problem. The random variable $X$ defines the measure $\nu$ on $\mathcal{G}$; absolute continuity ensures that this measure has a density with respect to the restricted probability measure; and the density is the required $\mathcal{G}$-measurable random variable.

A common mistake is to apply the Radon–Nikodym theorem on $(\Omega,\mathcal{F})$ and expect the result to be measurable with respect to $\mathcal{G}$. The construction instead applies the theorem on $(\Omega,\mathcal{G})$. That choice is what ensures that the density uses only the information represented by $\mathcal{G}$.

Another important point is that the defining condition is an equality of integrals over every event in $\mathcal{G}$, not a pointwise equality between $X$ and its conditional expectation. The examples on the product space show why: the conditional expectation can depend on one coordinate while the original variable depends on both, or it can be zero even when the original variable is not.

Check Your Understanding

Use the measure construction and the integral identity to check your understanding.

For nonnegative integrable $X$, how is the induced measure $\nu$ defined on $\mathcal{G}$?
Why is $\nu$ absolutely continuous with respect to the restriction of $\mathbb{P}$ to $\mathcal{G}$?
Which measurable space and reference measure are used when applying the Radon–Nikodym theorem?
Why does taking $A=\Omega$ in the density identity establish integrability in the nonnegative case?
How do the positive and negative parts extend the construction to an integrable signed random variable?