Conditional Expectation from Joint Densities

From a Joint Density to a Conditional Expectation

When two real-valued random variables have a joint density, conditioning on one variable can be computed by first finding its marginal density and then normalizing the joint density. The resulting conditional density gives an ordinary integral formula for conditional expectations. The normalization is essential: the joint density describes both variables together, while a conditional density must integrate to one in the variable being conditioned.

Let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space, and let \(X,Y:\Omega\to\mathbb{R}\) be real-valued random variables. Assume their joint distribution has a density \(f_{X,Y}:\mathbb{R}^2\to[0,\infty]\) with respect to Lebesgue measure \(m\times m\). Thus \(f_{X,Y}\) is measurable, nonnegative, and has integral one. All density formulas below are understood up to Lebesgue-null sets. The marginal density of \(X\) is obtained by integrating out \(y\); the marginal density of \(Y\) is obtained by integrating out \(x\). These statements follow from Tonelli's theorem, since the joint density is nonnegative.

f_X(x)=\int_{\mathbb{R}} f_{X,Y}(x,y)\,dy, \qquad f_Y(y)=\int_{\mathbb{R}} f_{X,Y}(x,y)\,dx.

For \(x\) with \(f_X(x)>0\), divide the joint density by the marginal density of \(X\). The result is a density in \(y\): its integral over \(y\) is one for almost every such \(x\). If \(f_X(x)=0\), this quotient is undefined and a conditional density at that \(x\) need not be specified by the joint density. Any choice there has no effect on a conditional expectation, because \(X\) takes values in a set on which \(f_X=0\) with probability zero.

Definition: A version of the conditional density of \(Y\) given \(X=x\) is \(f_{Y\mid X}(y\mid x)=f_{X,Y}(x,y)/f_X(x)\) where \(f_X(x)>0\), for almost every such \(x\). Values at exceptional \(x\), including points where \(f_X(x)=0\), may be assigned arbitrarily. For an integrable Borel function \(g:\mathbb{R}\to\mathbb{R}\), the corresponding conditional expectation is \(\mathbb{E}[g(Y)\mid\sigma(X)]=h(X)\) almost surely, where \(h(x)=\int_{\mathbb{R}}g(y)f_{Y\mid X}(y\mid x)\,dy\) for \(f_X(x)\,dx\)-almost every \(x\).

Here \(\sigma(X)\) is the sigma-algebra generated by \(X\), and conditional expectation means an integrable, \(\sigma(X)\)-measurable random variable whose integral on every event in \(\sigma(X)\) equals the integral of the variable being conditioned. Integrable means that the expectation of the absolute value is finite. The function \(g\) is required to satisfy \(\mathbb{E}|g(Y)|<\infty\). Conditional expectations are determined only almost surely, so changing \(h\) on a set having zero probability under the distribution of \(X\) does not change \(h(X)\) almost surely.

Why the Integral Formula Gives Conditional Expectation

Theorem: Under the joint-density and integrability assumptions above, define \(h\) by integrating \(g\) against \(f_{Y\mid X}\) wherever the conditional density is defined, and choose any measurable values on the exceptional set. Then \(h(X)\) is a version of \(\mathbb{E}[g(Y)\mid\sigma(X)]\).

Proof. First, the marginal formula for \(f_X\) holds almost everywhere by Tonelli's theorem. Therefore, wherever the quotient is defined and the marginal formula holds,

\int_{\mathbb{R}} f_{Y\mid X}(y\mid x)\,dy = \frac{\int_{\mathbb{R}}f_{X,Y}(x,y)\,dy}{f_X(x)} =1.

The measurable-integration theorem gives a measurable version of \(h\): apply it to the positive and negative parts of \(g\), whose integrals are finite for \(f_X(x)\,dx\)-almost every \(x\), as shown by Tonelli's theorem and \(\mathbb{E}|g(Y)|<\infty\). For integrability and the defining identity, let \(A\subseteq\mathbb{R}\) be Borel. Fubini's theorem applies to \(g(y)f_{X,Y}(x,y)\), because its absolute integral is \(\mathbb{E}|g(Y)|<\infty\). Thus

\begin{aligned} \mathbb{E}\bigl[|h(X)|\bigr] &\le \int_{\mathbb{R}}\int_{\mathbb{R}}|g(y)|f_{X,Y}(x,y)\,dy\,dx =\mathbb{E}|g(Y)|<\infty,\\ \int_{\{X\in A\}}h(X)\,d\mathbb{P} &=\int_A h(x)f_X(x)\,dx\\ &=\int_A\int_{\mathbb{R}}g(y)f_{X,Y}(x,y)\,dy\,dx\\ &=\mathbb{E}\bigl[g(Y)\chi_{\{X\in A\}}\bigr]. \end{aligned}

In the second equality, the conditional-density quotient cancels \(f_X(x)\) for almost every \(x\) with positive marginal density; points where the marginal is zero contribute zero to the integral. The final equality is Fubini's theorem with the absolute-integrability check already given. Every event in \(\sigma(X)\) has the form \(\{X\in A\}\) for a Borel set \(A\), so this is exactly the defining integral identity for conditional expectation. Hence \(h(X)=\mathbb{E}[g(Y)\mid\sigma(X)]\) almost surely. \(\square\)

Taking \(g=\chi_B\) for a Borel set \(B\) gives the corresponding conditional-probability formula: the conditional probability of \(Y\in B\), given \(X\), is the conditional density integrated over \(B\). The same normalization and almost-everywhere qualifications apply.

Worked Density Calculations

Worked Example: A Uniform Density on a Triangle

Suppose the joint density is \(f_{X,Y}(x,y)=2\) when \(0<y<x<1\), and zero otherwise. The inequalities are strict; changing the density on the boundary does not alter any probability or integral. Find the conditional density of \(Y\) given \(X=x\), and compute \(\mathbb{E}[Y\mid\sigma(X)]\).

For \(0<x<1\), the allowed values of \(y\) are \(0<y<x\). Therefore the marginal density is

f_X(x)=\int_0^x 2\,dy=2x.

Its integral is \(\int_0^1 2x\,dx=1\), as required. For \(0<x<1\), the conditional density is \(2/(2x)=1/x\) on \(0<y<x\), and zero elsewhere. Its integral is \(\int_0^x(1/x)\,dy=x/x=1\). Thus

\mathbb{E}[Y\mid\sigma(X)] =h(X),\qquad h(x)=\int_0^x y\frac1x\,dy =\frac1x\left[\frac{y^2}{2}\right]_0^x =\frac{x}{2}.

Check: The expectation of this conditional mean is \(\int_0^1(x/2)(2x)\,dx=\int_0^1x^2\,dx=1/3\). Directly from the joint density, \(\mathbb{E}Y=\int_0^1\int_0^x2y\,dy\,dx=\int_0^1x^2\,dx=1/3\). The two calculations agree. At \(x=0\), \(f_X(0)=0\), so the conditional density need not be defined by a quotient there.

Worked Example: Independent Exponential Densities

Let \(f_{X,Y}(x,y)=e^{-x-y}\) for \(x>0\) and \(y>0\), and zero otherwise. Compute \(\mathbb{E}[Y\mid\sigma(X)]\). The inequalities defining the support are strict, and endpoints have measure zero.

The joint density has total integral \((\int_0^\infty e^{-x}\,dx)(\int_0^\infty e^{-y}\,dy)=1\). Tonelli's theorem permits this product calculation because the integrand is nonnegative. Marginalizing gives \(f_X(x)=e^{-x}\int_0^\infty e^{-y}\,dy=e^{-x}\) for \(x>0\). Hence \(f_{Y\mid X}(y\mid x)=e^{-y}\) for \(y>0\), whose integral is one. By integration by parts,

h(x)=\int_0^\infty ye^{-y}\,dy =\left[-ye^{-y}\right]_0^\infty+\int_0^\infty e^{-y}\,dy =0+1=1.

The boundary term at infinity is zero since \(ye^{-y}\to0\). Thus \(\mathbb{E}[Y\mid\sigma(X)]=1\) almost surely. Check: the marginal density of \(Y\) is also \(e^{-y}\) for \(y>0\), so direct integration gives \(\mathbb{E}Y=\int_0^\infty ye^{-y}\,dy=1\), agreeing with the expectation of the constant conditional mean.

Worked Example: A Nonconstant Conditional Mean

On the unit square, let \(f_{X,Y}(x,y)=x+y\) for \(0<x<1\) and \(0<y<1\), and zero outside. Find \(\mathbb{E}[Y\mid\sigma(X)]\). Again, the support inequalities are strict; boundary values do not affect the density.

Integrating over \(y\) gives \(f_X(x)=\int_0^1(x+y)\,dy=x+1/2\). This is a probability density because \(\int_0^1(x+1/2)\,dx=1/2+1/2=1\). The conditional density is \((x+y)/(x+1/2)\) for \(0<y<1\). It is normalized, since \(\int_0^1(x+y)\,dy=x+1/2\). Its conditional mean is

\begin{aligned} h(x) &=\int_0^1 y\frac{x+y}{x+1/2}\,dy\\ &=\frac{1}{x+1/2}\left(x\int_0^1y\,dy+\int_0^1y^2\,dy\right)\\ &=\frac{x/2+1/3}{x+1/2}. \end{aligned}

Check: Averaging this conditional mean against \(f_X\) cancels the denominator and gives \(\int_0^1(x/2+1/3)\,dx=1/4+1/3=7/12\). Directly, \(\mathbb{E}Y=\int_0^1\int_0^1 y(x+y)\,dx\,dy=\int_0^1(y/2+y^2)\,dy=1/4+1/3=7/12\), confirming the result.

Normalization Is Not Optional

A frequent error is to integrate \(g(y)f_{X,Y}(x,y)\) and call the result a conditional expectation. That integral is a numerator, not a conditional average: it must be divided by \(f_X(x)\) wherever the marginal density is positive. In the unit-square example, at \(x=0\) the numerator for \(g(y)=y\) is \(\int_0^1 y(0+y)\,dy=1/3\), while \(f_X(0)=1/2\). The correct conditional mean, using the continuous formula at this point, is \((1/3)/(1/2)=2/3\); the unnormalized calculation gives \(1/3\). The conditional density there is \(2y\) on \(0<y<1\), and its integral is \(2\int_0^1y\,dy=1\). This check exposes the missing normalization.

More generally, a conditional-density calculation has three useful checks: first, integrate the joint density to obtain the marginal; second, integrate the proposed conditional density in the variable being conditioned and verify that the result is one for marginal-almost every conditioning value; third, average the resulting conditional expectation against the marginal and compare with the corresponding unconditional expectation. The last comparison is a consequence of the defining identity, but it is also an effective way to detect arithmetic or normalization errors.

Check Your Understanding

In each question, specify the support using strict or non-strict inequalities as appropriate, and state where the conditional-density formula is defined.

For the density \(2\) on \(0<y<x<1\), compute \(\mathbb{E}[\chi_{\{Y<1/2\}}\mid\sigma(X)]\) as a function of \(X\).
For the unit-square density \(f_{X,Y}(x,y)=x+y\), verify the conditional density by integrating it over \(0<y<1\) for a fixed \(x\) with positive marginal density.
Explain why a conditional density cannot be obtained by dividing by \(f_X(x)\) at a point where \(f_X(x)=0\), and why its assigned value there does not affect the conditional expectation.
For a joint density with \(f_X(x)>0\), derive the formula for \(\mathbb{E}[g(Y)\mid\sigma(X)]\) when \(g\) is an integrable indicator function.
In the exponential example, compute \(\mathbb{E}[Y\mid\sigma(X)]\) using the normalized conditional density and compare it with \(\mathbb{E}Y\).

Conditional Expectation from Joint Densities

What You'll Learn