Tutorials › Multivariate Statistics (Graduate) › Conditional Densities and Conditional Expectations

Home  /  Tutorials  /  Multivariate Laws and Transformations

Multivariate Laws and Transformations · Tutorial 51 of 975

Conditional Densities and Conditional Expectations

Build conditional densities from joint and marginal laws, then use them to calculate conditional probabilities, expectations, and variances.

Beginner 10 min read

What You'll Learn

  • Construct a conditional density by dividing the joint density by the appropriate marginal density.
  • Identify the conditional support from the joint support.
  • Calculate conditional probabilities by integrating over an event.
  • Find conditional expectations and variances from conditional densities.
  • State when these formulas apply and why conditioning at exceptional values needs care.
  • Derive the laws of total expectation and total variance.

From a Joint Density to a Conditional Density

A joint density describes the distribution of a random vector all at once. But many questions are conditional: among individuals with a particular value, or range of values, for one component, how is another component distributed? For continuous random variables, the answer comes from comparing the joint density with the marginal density of the variable being conditioned on.

Let \(X\in\mathbb{R}^p\) and \(Y\in\mathbb{R}^q\) have joint density \(f_{X,Y}(x,y)\), and let \(f_X(x)\) be the marginal density of \(X\), obtained by integrating the joint density over \(y\). Where \(f_X(x)>0\), the ratio of these densities gives a conditional density for \(Y\) given \(X=x\).

Definition: For a joint density \(f_{X,Y}\) and marginal density \(f_X\), a conditional density of \(Y\) given \(X=x\) is $$ f_{Y\mid X}(y\mid x)=\frac{f_{X,Y}(x,y)}{f_X(x)},\qquad f_X(x)>0. $$ It is a density in \(y\), for each relevant \(x\). Its support is the set of \(y\)-values for which the joint density at \((x,y)\) is positive.

The phrase “for each relevant \(x\)” matters. The ratio is defined directly only where \(f_X(x)>0\). Moreover, a density can be changed on a set of measure zero without changing the distribution it describes. Consequently, conditional distributions specified by densities are determined only for almost every \(x\) under the distribution of \(X\). At an individual value where \(f_X(x)=0\), the ratio does not define a conditional density; additional model structure may sometimes specify a useful version there, but the density formula alone does not.

For any measurable set \(B\subseteq\mathbb{R}^q\), the conditional probability is obtained by integrating the conditional density over \(B\):

$$ P(Y\in B\mid X=x)=\int_B f_{Y\mid X}(y\mid x)\,dy. $$

Likewise, if \(g(Y)\) is integrable, its conditional expectation is the integral of \(g\) against that density. For a vector-valued \(Y\), expectations are taken component by component.

$$ E[g(Y)\mid X=x]=\int_{\mathbb{R}^q}g(y)f_{Y\mid X}(y\mid x)\,dy. $$

Worked Conditional Calculations

Worked Example: A Uniform Conditional Distribution on a Triangle

Suppose the joint density is \(f_{X,Y}(x,y)=2\) when \(0<y<x<1\), and is zero elsewhere. Find the marginal density of \(X\), the conditional density of \(Y\) given \(X=x\), and the conditional mean and variance.

First integrate over the possible values of \(y\) at each \(x\). The support requires \(0<y<x\), so

$$ f_X(x)=\int_0^x 2\,dy=2x,\qquad 0<x<1. $$

The marginal integrates to one: \(\int_0^1 2x\,dx=1\). For \(0<x<1\), divide the joint density by this marginal:

$$ f_{Y\mid X}(y\mid x)=\frac{2}{2x}=\frac{1}{x},\qquad 0<y<x. $$

This is the uniform density on \((0,x)\), and its integral is \(x(1/x)=1\). For example, the conditional probability that \(Y<x/2\) is

$$ P(Y<x/2\mid X=x)=\int_0^{x/2}\frac{1}{x}\,dy=\frac12. $$

The conditional mean and second moment are

$$ E[Y\mid X=x]=\int_0^x \frac{y}{x}\,dy=\frac{x}{2}, \qquad E[Y^2\mid X=x]=\int_0^x \frac{y^2}{x}\,dy=\frac{x^2}{3}. $$

Therefore,

$$ \operatorname{Var}(Y\mid X=x) =E[Y^2\mid X=x]-\{E[Y\mid X=x]\}^2 =\frac{x^2}{3}-\frac{x^2}{4} =\frac{x^2}{12}. $$

Worked Example: A Conditional Support That Changes with \(x\)

Let \(f_{X,Y}(x,y)=2\) on \(0<x<y<1\), and zero elsewhere. Find \(f_{Y\mid X}(y\mid x)\), then calculate the conditional mean and the probability that \(Y>3/4\) when \(X=1/2\).

For fixed \(x\), the joint support allows \(x<y<1\). Integrating over that interval gives

$$ f_X(x)=\int_x^1 2\,dy=2(1-x),\qquad 0<x<1. $$

This is a valid marginal density because \(\int_0^1 2(1-x)\,dx=1\). Thus

$$ f_{Y\mid X}(y\mid x)=\frac{2}{2(1-x)} =\frac{1}{1-x},\qquad x<y<1. $$

The conditional support depends on \(x\): it is the interval from \(x\) to \(1\), not the entire unit interval. Integrating gives

$$ E[Y\mid X=x] =\int_x^1 \frac{y}{1-x}\,dy =\frac{1-x^2}{2(1-x)} =\frac{1+x}{2},\qquad 0<x<1. $$

At \(x=1/2\), the conditional density is \(2\) on \((1/2,1)\). Hence

$$ P(Y>3/4\mid X=1/2) =\int_{3/4}^1 2\,dy =2\left(1-\frac34\right)=\frac12, \qquad E[Y\mid X=1/2]=\frac{1+1/2}{2}=\frac34. $$

Worked Example: An Exponential Conditional Density

Suppose \(f_{X,Y}(x,y)=xe^{-x}e^{-xy}\) for \(x>0\) and \(y>0\), and is zero elsewhere. Find the conditional distribution of \(Y\) given \(X=x\), its mean and variance, and \(P(Y>2\mid X=3)\).

Integrating out \(y\) yields

$$ f_X(x)=\int_0^\infty xe^{-x}e^{-xy}\,dy =xe^{-x}\left[\frac{1}{x}\right] =e^{-x},\qquad x>0. $$

Here \(\int_0^\infty e^{-xy}\,dy=1/x\) because \(x>0\). The marginal is normalized, since \(\int_0^\infty e^{-x}\,dx=1\). Dividing by it gives

$$ f_{Y\mid X}(y\mid x)=xe^{-xy},\qquad y>0. $$

This is an exponential density with rate \(x\). Direct integration, or the standard exponential moments, gives \(E[Y\mid X=x]=1/x\) and \(E[Y^2\mid X=x]=2/x^2\). Therefore,

$$ \operatorname{Var}(Y\mid X=x)=\frac{2}{x^2}-\left(\frac{1}{x}\right)^2 =\frac{1}{x^2}. $$

For \(x=3\), integrating the conditional density over \(y>2\) gives

$$ P(Y>2\mid X=3)=\int_2^\infty 3e^{-3y}\,dy =\left[-e^{-3y}\right]_2^\infty=e^{-6}. $$

Why the Density Ratio Works

The ratio is not just a convenient recipe. It is normalized for almost every \(x\) with positive marginal density, and it reproduces joint probabilities when integrated against the marginal density of \(X\).

Theorem: Suppose \((X,Y)\) has a joint density \(f_{X,Y}\) and marginal density \(f_X(x)=\int f_{X,Y}(x,y)\,dy\). For almost every \(x\) with \(f_X(x)>0\), the ratio \(f_{X,Y}(x,y)/f_X(x)\) integrates to one in \(y\). For measurable sets \(A\subseteq\mathbb{R}^p\) and \(B\subseteq\mathbb{R}^q\), $$ P(X\in A,Y\in B)=\int_A P(Y\in B\mid X=x)f_X(x)\,dx. $$

Proof. Since the joint density is nonnegative, Tonelli’s theorem permits the iterated integrals below. For any \(x\) with \(0<f_X(x)<\infty\),

$$ \int_{\mathbb{R}^q}\frac{f_{X,Y}(x,y)}{f_X(x)}\,dy =\frac{1}{f_X(x)}\int_{\mathbb{R}^q}f_{X,Y}(x,y)\,dy =1. $$

A marginal density is finite almost everywhere; thus this calculation applies for almost every \(x\) where it is positive. For the probability identity, first express the joint probability as an integral of its density and then substitute the ratio wherever \(f_X(x)>0\):

$$ \begin{aligned} P(X\in A,Y\in B) &=\int_A\int_B f_{X,Y}(x,y)\,dy\,dx\\ &=\int_A f_X(x)\left\{\int_B \frac{f_{X,Y}(x,y)}{f_X(x)}\,dy\right\}dx\\ &=\int_A P(Y\in B\mid X=x)f_X(x)\,dx. \end{aligned} $$

The set of \(x\)-values where \(f_X(x)=0\) contributes zero to the joint integral, so values assigned to the conditional probability there do not change the identity. \(\square\)

Conditional Expectations and Variances

For a scalar \(Y\), define the conditional mean \(m(x)=E[Y\mid X=x]\) by integrating \(y\) against the conditional density, whenever that integral exists. The conditional variance measures the remaining variation in \(Y\) at that value of \(X\):

$$ \operatorname{Var}(Y\mid X=x) =E[(Y-m(x))^2\mid X=x] =E[Y^2\mid X=x]-m(x)^2. $$

These quantities are functions of \(x\), not generally constants. For a random vector \(Y\), the conditional mean is a vector and the conditional covariance matrix is the conditional expectation of \((Y-m(x))(Y-m(x))^\mathsf{T}\), when its entries exist.

Theorem: If \(E[|Y|]<\infty\), then $$ E[Y]=E[E[Y\mid X]]. $$ If also \(E[Y^2]<\infty\), then $$ \operatorname{Var}(Y) =E[\operatorname{Var}(Y\mid X)] +\operatorname{Var}(E[Y\mid X]). $$

Proof. Write \(m(x)=E[Y\mid X=x]\). Under the first moment condition, Fubini’s theorem applies to the integrable function \(y f_{X,Y}(x,y)\). Using the conditional-density ratio gives

$$ E[m(X)] =\int m(x)f_X(x)\,dx =\int\int y f_{X,Y}(x,y)\,dy\,dx =E[Y]. $$

For the variance identity, finite second moment ensures the terms below are integrable. At each \(x\), expand \(Y-E[Y]\) as \((Y-m(x))+(m(x)-E[Y])\), square, and take the conditional expectation. The cross term is zero because \(E[Y-m(x)\mid X=x]=0\). Thus

$$ E[(Y-E[Y])^2\mid X=x] =\operatorname{Var}(Y\mid X=x)+(m(x)-E[Y])^2. $$

Integrating both sides with respect to \(f_X(x)\,dx\), and using the first identity to identify \(E[m(X)]=E[Y]\), gives

$$ \operatorname{Var}(Y) =E[\operatorname{Var}(Y\mid X)] +E[(m(X)-E[m(X)])^2] =E[\operatorname{Var}(Y\mid X)] +\operatorname{Var}(E[Y\mid X]). $$

This proves both identities under the stated moment conditions. \(\square\)

Common Pitfall: Ignoring the Conditional Support

A frequent error is to divide the densities correctly but then integrate over a support that is too large. In the triangular example, \(Y\mid X=x\) is supported on \(0<y<x\), not on all positive \(y\). Integrating \(1/x\) from \(0\) to \(1\) would give \(1/x\), not \(1\), and would fail the basic test for a density. The conditional support must be read from the joint support after fixing \(x\).

Another mistake is to use the ratio at an \(x\) where \(f_X(x)=0\), or to assume a conditional distribution at a single point is uniquely determined by the joint law. The ratio is justified where the marginal density is positive and determines the conditional law only almost everywhere. This distinction is especially important when an expression is evaluated at a specific conditioning value.

Conditional densities are useful because they turn questions about dependence into calculations at fixed values of one component. They also separate within-group variability, represented by the conditional variance, from variability in the conditional mean across groups. The total-variance identity makes that separation precise.

Check Your Understanding

Use the density ratio and keep track of the support in each calculation.

  1. For a joint density supported on \(0<y<x<1\), what is the range of \(y\) when calculating \(f_{Y\mid X}(y\mid x)\)?
  2. Why does a conditional density obtained by the ratio need to integrate to one for each relevant \(x\)?
  3. What condition is required to define the density ratio \(f_{X,Y}(x,y)/f_X(x)\) at a given value of \(x\)?
  4. State the integral formula for \(E[g(Y)\mid X=x]\).
  5. In the law of total variance, what do the two terms on the right-hand side represent mathematically?

Previous

Next

Back to Tutorials
Clinical Biostats