Conditional Expectation from Conditional Densities

From a Conditional Density to a Conditional Expectation

When a joint density is known, it can answer more than questions about probabilities. It can also describe the distribution of one variable after the value of another is observed. Integrating a function against that conditional density gives the conditional expectation; as the observed value varies, the answer is a function of that value.

Let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space, and let \(X\) and \(Y\) be real-valued random variables with joint density \(f_{X,Y}(x,y)\) with respect to two-dimensional Lebesgue measure. Their marginal density for \(X\) is \[ f_X(x)=\int_{\mathbb{R}} f_{X,Y}(x,y)\,dy. \] At values \(x\) where \(f_X(x)>0\), the conditional density of \(Y\) given \(X=x\) is obtained by dividing the joint density by that marginal density.

The division matters: a joint density describes both the likelihood of seeing \(X\) near \(x\) and the distribution of \(Y\) there. Dividing by \(f_X(x)\) removes the first factor and leaves a density over possible \(Y\)-values, for almost every observed \(x\).

The theorem asserts equality almost surely, not equality at every numerical value \(x\). A conditional expectation is defined by its averages over events determined by \(X\); changing the function at values \(x\) that \(X\) takes with probability zero does not change that random variable.

Proof. First consider \(g(Y)\). Since \(\mathbb{E}|g(Y)|<\infty\), integration of \(|g(y)|f_{X,Y}(x,y)\) over \((x,y)\) is finite. Tonelli's theorem applied to this nonnegative integrand implies that for almost every \(x\) with \(f_X(x)>0\), the integral of \(|g(y)|f_{Y\mid X}(y\mid x)\) is finite. Thus \(h(x)\) is finite there. For any Borel set \(B\subseteq\mathbb{R}\), the joint-density formula and Fubini's theorem give \[ \begin{aligned} \mathbb{E}\big[\mathbf{1}_{\{X\in B\}}g(Y)\big] &=\int_B\int_{\mathbb{R}}g(y)f_{X,Y}(x,y)\,dy\,dx\\ &=\int_B h(x)f_X(x)\,dx\\ &=\mathbb{E}\big[\mathbf{1}_{\{X\in B\}}h(X)\big]. \end{aligned} \] The middle equality uses \(f_{X,Y}(x,y)=f_X(x)f_{Y\mid X}(y\mid x)\) wherever \(f_X(x)>0\); the set where \(f_X(x)=0\) contributes zero to the integral. The events \(\{X\in B\}\) generate \(\sigma(X)\). The displayed identity for every such \(B\), together with the integrability just established, is the defining property of conditional expectation, so \(h(X)\) is a version of \(\mathbb{E}[g(Y)\mid\sigma(X)]\). For \(g(X,Y)\), the same calculation replaces \(g(y)\) by \(g(x,y)\); the assumed integrability justifies Fubini and yields the stated formula. \(\square\)

One preliminary check is useful in every application: the proposed conditional density must integrate to one in \(y\), for almost every \(x\) under the marginal distribution of \(X\). This follows directly from its definition.

Proof. For each \(x\) with \(f_X(x)>0\) for which the marginal integral is defined, \[ \int_{\mathbb{R}}f_{Y\mid X}(y\mid x)\,dy =\frac{1}{f_X(x)}\int_{\mathbb{R}}f_{X,Y}(x,y)\,dy =\frac{f_X(x)}{f_X(x)}=1. \] The marginal density identity holds almost everywhere, which gives the asserted qualification. \(\square\)

Worked Calculations

What the Formula Does—and Does Not—Say

The conditional expectation is a function of the conditioning variable because the observed value \(X=x\) determines which conditional density to use. In the examples, integrating against that density produced \(x/2\), \(1/(1+x)\), and \((x+1)/3\). Once this function is evaluated at the random variable \(X\), it represents the conditional expectation as a random variable.

The condition \(\mathbb{E}|g(Y)|<\infty\) ensures the conditional expectation is finite almost surely. For a nonnegative \(g\), the same integral formula is valid as an extended nonnegative conditional expectation, which may be infinite on a set of positive probability. Values of \(x\) outside the support of \(X\), and exceptional values within the support where a density formula is not specified, do not determine the conditional expectation as a random variable. A formula there can be chosen as a version, but it is not uniquely determined by conditioning.

Common mistake: forgetting to divide by the marginal. In the triangular example, a wrong calculation might use \[ \int_0^x y f_{X,Y}(x,y)\,dy =\int_0^x 2y\,dy=x^2 \] as the conditional mean. This is not a conditional mean: it is the joint-density-weighted numerator. The correct calculation divides by \(f_X(x)=2x\), giving \(x^2/(2x)=x/2\) for \(0<x<1\). Equivalently, first form the conditional density \(2/(2x)=1/x\), then integrate \(y\) against it. The denominator is essential because the conditional density must integrate to one.