Continuous and Mixed Conditional Expectation Workshop

Choosing the Right Conditional Calculation

A conditional mean can look like a familiar integral and still be wrong if it uses the wrong support or leaves out part of the conditioning event. For continuous variables, the conditional density is obtained by dividing the joint density by a marginal density, but that ratio depends on where the joint density is positive. For a mixed distribution, a probability at an atom must be handled as a probability, not as a density value.

The calculations in this workshop use two basic strategies. First, find the conditional law and average with respect to it. Second, when conditioning on an event such as $Y>c$, integrate the relevant quantity over that event and divide by its probability. When a model has stages or a mixture of discrete and continuous parts, compute the mean within each part and then average those means with the part’s probabilities.

Conditional Means from Joint Densities

Definition: Suppose $(X,Y)$ has joint density $f_{X,Y}(x,y)$, and let $f_X(x)=\int_{-\infty}^{\infty} f_{X,Y}(x,y)\,dy$ be the marginal density of $X$. At values $x$ for which $f_X(x)>0$, a conditional density of $Y$ given $X=x$ is $$ f_{Y\mid X}(y\mid x)=\frac{f_{X,Y}(x,y)}{f_X(x)}. $$ If $E[|h(Y)|]<\infty$, the corresponding conditional mean is $$ E[h(Y)\mid X=x]=\int_{-\infty}^{\infty} h(y)f_{Y\mid X}(y\mid x)\,dy, $$ where the integral is effectively over the conditional support.

The ratio is defined only where the marginal density is positive, and conditional densities are specified only up to sets of marginal probability zero. In calculations, the support matters just as much as the ratio: a joint density may be positive only on a region whose limits depend on $x$.

Theorem: If $(X,Y)$ has joint density $f_{X,Y}$ and $E[|h(Y)|]<\infty$, then the function $$ g(x)=\frac{\int h(y)f_{X,Y}(x,y)\,dy}{f_X(x)} $$ where $f_X(x)>0$, satisfies $g(X)=E[h(Y)\mid X]$ almost surely.

Proof. The defining property of $E[h(Y)\mid X]$ is that its integral over every event determined by $X$ agrees with the integral of $h(Y)$ over that event. Let $A$ be any Borel set. Since $E[|h(Y)|]<\infty$, Fubini’s theorem applies to the integrals below. The joint density gives

E[h(Y)\mathbf{1}_{\{X\in A\}}] =\int_A\int h(y)f_{X,Y}(x,y)\,dy\,dx =\int_A g(x)f_X(x)\,dx =E[g(X)\mathbf{1}_{\{X\in A\}}].

Also, $\int |g(x)|f_X(x)\,dx\leq E[|h(Y)|]$, by the triangle inequality applied inside the integral. Thus $g(X)$ is integrable and satisfies the defining property for conditional expectation with respect to $X$. Therefore $g(X)=E[h(Y)\mid X]$ almost surely. $\square$

Worked Example: A Conditional Support That Depends on the Given Value

Let $(X,Y)$ have density $f_{X,Y}(x,y)=2$ on $0<x<y<1$, and zero elsewhere. Find $E[X\mid Y=y]$, then find $E[X\mid Y>1/2]$.

First, find the marginal density of $Y$. For a fixed $y$ between 0 and 1, the allowed values of $x$ run from 0 to $y$, not from 0 to 1. Hence

f_Y(y)=\int_0^y 2\,dx=2y,\qquad 0<y<1.

Now form the conditional density. On the support $0<x<y$,

f_{X\mid Y}(x\mid y)=\frac{2}{2y}=\frac{1}{y},\qquad 0<x<y.

This is a uniform density on $(0,y)$, so direct integration gives

E[X\mid Y=y]=\int_0^y x\frac{1}{y}\,dx =\frac{1}{y}\left[\frac{x^2}{2}\right]_0^y =\frac{y}{2}.

Condition on the threshold event. The probability of the event is

P(Y>1/2)=\int_{1/2}^1 2y\,dy =\left[y^2\right]_{1/2}^1 =\frac{3}{4}.

The numerator for the conditional mean is the integral of $x$ over the part of the joint support where $y>1/2$:

E[X\mathbf{1}_{\{Y>1/2\}}] =\int_{1/2}^1\int_0^y 2x\,dx\,dy =\int_{1/2}^1 y^2\,dy =\left[\frac{y^3}{3}\right]_{1/2}^1 =\frac{7}{24}.

Therefore

E[X\mid Y>1/2] =\frac{E[X\mathbf{1}_{\{Y>1/2\}}]}{P(Y>1/2)} =\frac{7/24}{3/4} =\frac{7}{18}.

The density ratio changes with $y$ through its support: $X$ is uniform on $(0,y)$, not on a fixed interval. The threshold calculation also restricts the outer integral to $y>1/2$.

Conditioning on an Event

Result: If $A$ is an event with $P(A)>0$ and $E[|X|]<\infty$, then $$ E[X\mid A]=\frac{E[X\mathbf{1}_A]}{P(A)}. $$ For a joint density and $A=\{Y>c\}$, this becomes $$ E[X\mid Y>c] =\frac{\int\int_{\{y>c\}} x f_{X,Y}(x,y)\,dx\,dy} {\int\int_{\{y>c\}} f_{X,Y}(x,y)\,dx\,dy}, $$ provided $P(Y>c)>0$.

Justification. Conditional on the event $A$, expectations are computed with the conditional probability $P(B\mid A)=P(B\cap A)/P(A)$. Integrating $X$ with respect to this conditional probability gives $E[X\mathbf{1}_A]/P(A)$. With a joint density, expectations over $A$ are integrals restricted to the region defining $A$. This is why the numerator and denominator must use the same conditioning region.

An equivalent route, when a conditional mean is already available, is to average it over the conditional distribution of the variable being conditioned on. In the previous example,

E[X\mid Y>1/2] =\frac{\int_{1/2}^1 E[X\mid Y=y]f_Y(y)\,dy}{P(Y>1/2)} =\frac{\int_{1/2}^1 (y/2)(2y)\,dy}{3/4} =\frac{7/24}{3/4} =\frac{7}{18}.

Both routes agree because they integrate the same joint density over the same event. The direct route is often safer when the conditional support or the event’s geometry is easy to see from the joint density.

Mixed Laws and Staged Averaging

A conditioning variable need not have a density everywhere. It may have an atom, such as positive probability at zero, as well as a continuous component. Conditional expectations are still computed by averaging over the law of the conditioning variable: the atom contributes its probability times its conditional mean, and the continuous part contributes an integral.

Result: Suppose $X$ has probability $p$ at 0 and, with probability $1-p$, has density $q(x)$. Let $m(x)=E[Y\mid X=x]$, including the conditional mean $m(0)$ under the conditional law at the atom. If $E[|Y|]<\infty$, then $$ E[Y]=p\,m(0)+(1-p)\int m(x)q(x)\,dx. $$

Proof. Split the expectation according to whether $X=0$ or $X$ lies in the continuous component. On the atom, the contribution is $P(X=0)E[Y\mid X=0]=p\,m(0)$. On the continuous component, first condition on the value $x$, then integrate over its density and multiply by the probability of entering that component. Thus

E[Y] =E[Y\mathbf{1}_{\{X=0\}}]+E[Y\mathbf{1}_{\{X\ne 0\}}] =p\,m(0)+(1-p)\int m(x)q(x)\,dx.

The integrability assumption ensures these expectations and the integral are well-defined. This is staged averaging: average within each conditional stage, then average across the possible stages. $\square$

Worked Example: Averaging Across an Atom and a Continuous Component

Let $P(X=0)=1/3$. With the remaining probability $2/3$, let $X$ have a uniform distribution on $(0,1)$. Given $X=0$, let $Y$ be Bernoulli with success probability $1/2$. Given $X=x>0$, let $Y$ be uniform on $(0,x)$. Find $E[Y\mid X]$ and $E[Y]$.

At the atom, $E[Y\mid X=0]=1/2$. For $x>0$, the conditional density of $Y$ is $1/x$ on $(0,x)$, so

E[Y\mid X=x]=\int_0^x y\frac{1}{x}\,dy =\frac{1}{x}\left[\frac{y^2}{2}\right]_0^x =\frac{x}{2}.

Thus a version of the conditional mean is $E[Y\mid X]=1/2$ when $X=0$, and $E[Y\mid X]=X/2$ when $X>0$. Averaging these conditional means across the two components gives

E[Y] =\frac{1}{3}\left(\frac{1}{2}\right) +\frac{2}{3}\int_0^1 \frac{x}{2}\,dx =\frac{1}{6}+\frac{2}{3}\left(\frac{1}{4}\right) =\frac{1}{3}.

The first term is the atom’s contribution. The second term is the continuous component’s contribution, including its mixture probability $2/3$. Integrating against the uniform density alone and forgetting that mixture probability would overstate the continuous contribution.

Worked Example: A Threshold Event That Includes an Atom

Use the same marginal distribution for $X$ as above: probability $1/3$ at 0 and probability $2/3$ on a uniform distribution over $(0,1)$. Now let $Y\mid X=0$ be Bernoulli with success probability $3/4$, and let $Y\mid X=x>0$ be uniform on $(0,x)$. Find $E[X\mid Y>1/2]$.

At $X=0$, the event $Y>1/2$ occurs with probability $3/4$. Its contribution to $P(Y>1/2)$ is therefore $(1/3)(3/4)=1/4$. The atom contributes nothing to $E[X\mathbf{1}_{\{Y>1/2\}}]$, because $X=0$ there.

For $x>0$, the event can occur only when $x>1/2$. Given such an $x$, its probability is the length of $(1/2,x)$ divided by the length of $(0,x)$:

P(Y>1/2\mid X=x)=\frac{x-1/2}{x},\qquad x>1/2.

The continuous component of $X$ has unconditional density $2/3$ on $(0,1)$. Its contribution to the event probability is

\frac{2}{3}\int_{1/2}^1\frac{x-1/2}{x}\,dx =\frac{2}{3}\left[\;x-\frac{1}{2}\ln x\;\right]_{1/2}^1 =\frac{1-\ln 2}{3}.

Hence $P(Y>1/2)=1/4+(1-\ln 2)/3=(7-4\ln 2)/12$. For the numerator, the atom again contributes zero, while the continuous component gives

E[X\mathbf{1}_{\{Y>1/2\}}] =\frac{2}{3}\int_{1/2}^1 x\frac{x-1/2}{x}\,dx =\frac{2}{3}\int_{1/2}^1(x-1/2)\,dx =\frac{2}{3}\left[\frac{(x-1/2)^2}{2}\right]_{1/2}^1 =\frac{1}{12}.

Dividing numerator by denominator,

E[X\mid Y>1/2] =\frac{1/12}{(7-4\ln 2)/12} =\frac{1}{7-4\ln 2}.

Numerically, this is about $0.237$. It is less than $1/2$ because the event includes the atom $X=0$, even though the continuous part contributing to the numerator has $X>1/2$.

Common Mistakes and Practical Checks

Using a fixed support when the support varies. If the joint density is positive only when $0<x<y$, the conditional integral at a fixed $y$ runs from 0 to $y$, not from 0 to 1.
Treating an atom as a density value. A probability mass at $X=0$ contributes its probability directly. It is not found by evaluating a continuous density at zero.
Dropping the mixture weight. If a continuous component is entered with probability $2/3$, its contribution to an unconditional expectation must include the factor $2/3$.
Ignoring an atom in a threshold event. Check whether the event can occur at the atom. In the last example, it can; that mass belongs in the denominator even though its contribution to the numerator is zero.
Changing the event between numerator and denominator. Both $E[X\mathbf{1}_A]$ and $P(A)$ must refer to the same event $A$.

A reliable calculation begins by writing the conditioning event or conditional support explicitly. Then determine which components contribute to the probability, which contribute to the weighted numerator, and only then divide or average. In multistage models, keep the probability of each stage visible until the final sum.

Check Your Understanding

Work through each question by identifying the relevant support or mixture component before integrating.

If a joint density is positive only on $0<x<y<1$, what are the limits for integrating over $x$ to find $f_Y(y)$?
For a mixed $X$ with mass $p$ at zero and a continuous component of probability $1-p$, what two terms make up $E[Y]$ when $m(x)=E[Y\mid X=x]$?
In a threshold calculation $E[X\mid Y>c]$, why must the probability in the denominator use the same region as the weighted integral in the numerator?
Suppose an atom has $X=0$ and the conditioning event can occur there. Explain how the atom contributes to $P(A)$ and to $E[X\mathbf{1}_A]$.
In the first worked example, verify that $\int_{1/2}^1 y^2\,dy=7/24$, and use it to recover the conditional mean.