Conditional Means With Mixed Distributions

Why Mixed Conditional Laws Need Two Contributions

A conditional distribution need not be purely discrete or purely continuous. For example, given a covariate value, an outcome might equal zero with positive probability and otherwise vary continuously. A conditional mean calculated from just the density would omit the point mass; a sum over point masses alone would omit the continuous outcomes. Correct calculation requires accounting for both parts of the law.

The same issue arises when a joint distribution combines discrete and continuous components. Conditioning on a value that has positive probability can lead to a different conditional law from conditioning on values in a continuous range. The calculations are related, but the source of the conditional distribution matters.

Definition: A mixed distribution on the real line has point masses at a finite or countable set of values and a continuous component described by a density, with no other component. If the atoms are $a_j$, their probabilities are $p_j$, and the continuous component has density $g$, then $$ P(Y\in B)=\sum_j p_j\,\mathbf{1}_{\{a_j\in B\}}+\int_B g(y)\,dy $$ for every Borel set $B$. The components must satisfy $p_j\geq 0$, $g(y)\geq 0$ almost everywhere, and $\sum_j p_j+\int_{\mathbb R}g(y)\,dy=1$.

The density here describes the continuous component, whose total probability can be less than one. It is not necessarily a density that integrates to one by itself. The definition excludes singular-continuous components, such as a distribution concentrated on a set of Lebesgue measure zero with no atoms. If such a component is present, its contribution must also be included.

Expectation Under a Mixed Law

For an integrable function $h$, the expectation under a mixed law is a sum over its point masses plus an integral over its density component. In particular, the conditional mean uses $h(y)=y$. Absolute integrability, rather than mere cancellation between positive and negative contributions, ensures that this calculation is well-defined.

Theorem: Suppose $Y$ has the mixed distribution above and $E[|h(Y)|]<\infty$. Then $$ E[h(Y)]=\sum_j h(a_j)p_j+\int_{\mathbb R}h(y)g(y)\,dy. $$ In particular, if $E[|Y|]<\infty$, then $$ E[Y]=\sum_j a_jp_j+\int_{\mathbb R}y g(y)\,dy. $$

To see why the terms have this form, integrate $h$ against the distribution of $Y$. Each atom contributes its function value multiplied by the probability at that point. The continuous part contributes the usual density integral.

List the atoms.
For each atom $a_j$, record its probability $p_j=P(Y=a_j)$.

Identify the continuous component.
Write its density $g$ and check that its integral equals the probability not accounted for by the atoms.

Calculate each contribution.
Sum $h(a_j)p_j$ over the atoms and add $\int h(y)g(y)\,dy$.

Worked Example: The Mean and an Event-Conditional Mean

Suppose $Y$ has probability $1/4$ at $-1$, probability $1/4$ at $2$, and continuous density $g(y)=1/4$ for $0<y<2$, with density zero elsewhere. Find $E[Y]$, and then find $E[Y\mid Y>1]$.

Check the total probability. The atoms contribute $1/4+1/4=1/2$. The density contributes $$ \int_0^2 \frac14\,dy=\frac12. $$ Thus the total probability is one.

Unconditional mean. The atomic contributions are $(-1)(1/4)$ and $2(1/4)$. The continuous contribution is $$ \int_0^2 y\frac14\,dy =\frac14\left[\frac{y^2}{2}\right]_0^2 =\frac12. $$ Consequently, $$ E[Y]=-\frac14+\frac12+\frac12=\frac34. $$

Restrict to $Y>1$. The event includes the atom at $2$, but not the atom at $-1$. Its probability is $$ P(Y>1)=\frac14+\int_1^2\frac14\,dy=\frac14+\frac14=\frac12. $$ The unnormalized first-moment contribution on this event is $$ 2\left(\frac14\right)+\int_1^2 y\frac14\,dy =\frac12+\frac14\left[\frac{y^2}{2}\right]_1^2 =\frac12+\frac38=\frac78. $$ Dividing by the event probability gives $E[Y\mid Y>1]=(7/8)/(1/2)=7/4$. The density component and the atom both affect this conditional mean.

Conditional Mixed Laws

Let $X$ and $Y$ be real-valued random variables. A conditional distribution of $Y$ given $X=x$ can be described by a probability kernel $Q_x$: for each $x$, $Q_x$ is a probability distribution for $Y$, and for each Borel set $B$, $Q_x(B)$ is a measurable function of $x$. For real-valued random variables, such conditional distributions exist. At values $x$ having probability zero, the conditional distribution is generally specified only up to changes on a set of $X$-probability zero. Thus a formula for $E[Y\mid X=x]$ is understood for $X$-almost every $x$, unless a particular conditional model specifies it at additional values.

Result: Suppose, for $X$-almost every $x$, the conditional distribution of $Y$ given $X=x$ consists only of atoms $a_j(x)$, with probabilities $p_j(x)$, and a density $g(y\mid x)$. Assume $$ \sum_j p_j(x)+\int_{\mathbb R}g(y\mid x)\,dy=1. $$ Whenever the conditional first absolute moment is finite, the conditional mean is $$ E[Y\mid X=x] =\sum_j a_j(x)p_j(x)+\int_{\mathbb R}y g(y\mid x)\,dy. $$

The probabilities and density in this formula are conditional components: their total mass is one for each $x$ where the conditional law is defined. The atoms can depend on $x$, and the density can also vary with $x$. For an unconditional integrability condition, $E[|Y|]<\infty$ ensures the conditional first absolute moment is finite for $X$-almost every $x$.

The result follows directly by applying the mixed-law expectation formula to $Q_x$. Specifically, for each such $x$, integrate $y$ against its conditional law: each atom contributes its location times its conditional probability, while the continuous component contributes the integral of $y$ against its conditional density. This yields the displayed formula for $E[Y\mid X=x]$.

Worked Example: A Conditional Atom and a Conditional Density

Let $X$ be uniform on $(0,1)$. Given $X=x$, suppose $Y$ has probability $x$ at $2$, and has conditional density $g(y\mid x)=1-x$ for $0<y<1$, zero elsewhere. Find $E[Y\mid X=x]$, evaluate it at $x=0.4$, and check the unconditional mean.

For $0<x<1$, the conditional atom has mass $x$. The continuous component has total mass $$ \int_0^1(1-x)\,dy=1-x, $$ so the conditional probabilities sum to one. The atom contributes $2x$ to the mean. The density contributes $$ \int_0^1 y(1-x)\,dy=(1-x)\left[\frac{y^2}{2}\right]_0^1=\frac{1-x}{2}. $$ Therefore $$ E[Y\mid X=x]=2x+\frac{1-x}{2}=\frac12+\frac32x. $$

At $x=0.4$, this is $0.5+1.5(0.4)=1.1$. To check the unconditional mean, integrate the conditional mean over the uniform distribution of $X$: $$ E[Y]=\int_0^1\left(\frac12+\frac32x\right)\,dx =\frac12+\frac34=\frac54. $$ The same value results by integrating the atom and density contributions over $x$: $\int_0^1 2x\,dx=1$ and $\int_0^1(1-x)/2\,dx=1/4$, giving $1+1/4=5/4$.

When the Joint Law Has Different Components

A joint distribution can combine probability masses and a two-dimensional density. Conditional calculations must use the component that applies at the conditioning value. In particular, a value of $X$ with positive probability is handled by dividing joint probabilities on that slice by $P(X=x)$. At values in a continuous component of $X$, conditional densities are obtained from the joint density and marginal density, where the marginal density is positive. This distinction matters because a density formula alone does not describe probability placed on an atom of $X$.

Worked Example: Conditioning on an Atomic or Continuous Value

Consider a joint law with $P(X=0,Y=0)=0.2$ and $P(X=0,Y=2)=0.1$. Its remaining probability, $0.7$, has joint density $f(x,y)=0.7$ on $0<x<1,\ 0<y<1$, and zero elsewhere. Find the conditional mean of $Y$ given $X=0$, and the conditional mean given a continuous value $X=x$, $0<x<1$.

The joint density has total integral $$ \int_0^1\int_0^1 0.7\,dy\,dx=0.7, $$ so together with the two joint atoms the law has total probability $0.7+0.2+0.1=1$.

At $X=0$, the marginal probability is $P(X=0)=0.2+0.1=0.3$. The conditional probabilities are $0.2/0.3=2/3$ at $Y=0$ and $0.1/0.3=1/3$ at $Y=2$. Hence $$ E[Y\mid X=0]=0\left(\frac23\right)+2\left(\frac13\right)=\frac23. $$

For $0<x<1$, the marginal density of $X$ is $$ f_X(x)=\int_0^1 0.7\,dy=0.7. $$ Thus the conditional density of $Y$ at such an $x$ is $f(x,y)/f_X(x)=0.7/0.7=1$ for $0<y<1$. It is uniform on $(0,1)$, so $$ E[Y\mid X=x]=\int_0^1 y\,dy=\frac12. $$ The conditional means differ because the joint law has one kind of component at $X=0$ and another for $0<X<1$.

Conditioning Within a Mixed Law

A useful calculation arises when an event selects only part of a mixed distribution. Let $A$ be a Borel set and suppose $P(Y\in A)>0$. If the law has atoms $a_j$ with probabilities $p_j$ and continuous density $g$, then the conditional mean on $A$ is $$ E[Y\mid Y\in A] =\frac{\sum_{j:a_j\in A}a_jp_j+\int_A y g(y)\,dy} {\sum_{j:a_j\in A}p_j+\int_A g(y)\,dy}, $$ provided the absolute first moment on $A$ is finite. The denominator is the probability of the event, and the numerator is its unnormalized first-moment contribution. This formula makes boundary details important: if $A$ includes an atom at its boundary, that atom belongs in both the numerator and denominator.

Common Mistake: Normalizing the Density to One

Wrong: Treat the density as if it must integrate to one, then add the atom probabilities on top. This can make the total probability exceed one.
Correction: For a mixed law, the density component integrates to the probability left after accounting for the atoms. Check $\sum_j p_j+\int g=1$ before calculating a mean.
Wrong: Use only a conditional density to calculate a conditional mean even though the conditional law has an atom.
Correction: Add every conditional atom's location times its conditional probability, as well as the integral against the continuous density.
Wrong: Apply a density ratio at a value $X=x$ that is an atom of $X$, ignoring joint probability masses at that value.
Correction: At an atom, form conditional probabilities from the joint masses on that slice. Use the joint-to-marginal density ratio for continuous conditioning values where the marginal density is positive.

Key takeaway: A mixed conditional mean is the sum of two contributions: the conditional mean contribution from point masses and the first-moment integral from the continuous density. Verify the conditional probabilities add to one and include exactly the atoms belonging to any conditioning event.

Check Your Understanding

For each question, identify the atomic and continuous contributions before combining them.

A law has probability $0.2$ at $0$ and density $0.4$ on $(0,2)$. Does this specify a probability distribution? Explain by checking the total mass.
A conditional law has probability $1/3$ at $3$ and density $2/3$ on $(0,1)$. Calculate its conditional mean.
In the first worked example, why does the event $Y>1$ include the atom at $2$ but exclude the atom at $-1$?
Why is a conditional density ratio not sufficient to calculate $E[Y\mid X=0]$ in the final worked example?
State the integrability condition used to ensure that the conditional first-moment formula is finite.