Conditional Laws in Mixed Distributions

Two Different Ways to Condition

A conditioning variable need not be purely discrete or purely continuous. For example, a measurement may be exactly zero with positive probability, while otherwise taking values over an interval. In that setting, conditioning on the atom and conditioning on a particular value in the continuous component require different calculations: the first uses probabilities, while the second uses densities.

The distinction matters even when the variable being conditioned on is the same. At an atom, the event being conditioned on has positive probability, so the ordinary conditional-probability ratio applies. At a continuous value, the event usually has probability zero. A conditional distribution there is instead specified by a density ratio, interpreted as a version of the conditional law for almost every value in the continuous component.

Definition: A real-valued random variable $X$ has a mixed distribution of the discrete–continuous type if its distribution can be written as $$ P(X\in A)=\sum_{a\in D:\,a\in A}p_a+\int_{A\cap C} f_X(x)\,dx, $$ where $D$ is a finite or countable set of atoms, $p_a=P(X=a)>0$, $C$ is the continuous part of the support, and $f_X$ is a density for that component. The atom probabilities and the continuous density together account for total probability one.

The displayed decomposition describes a distribution with discrete and absolutely continuous parts; it does not include a singular continuous component. To construct conditional laws, we also need to know how the joint distribution behaves on each part. Suppose $Y$ is real-valued. At an atom $a$ of $X$, define the joint slice $Q_a(B)=P(X=a,Y\in B)$. On the continuous component, suppose the joint law has density $g(x,y)$ with respect to $dx\,\nu(dy)$, where $\nu$ is a measure that can accommodate the possible values of $Y$. For a continuous $Y$, $\nu$ may be Lebesgue measure; for a mixed $Y$, it may combine point masses and Lebesgue measure.

Theorem: Under the preceding decomposition, let $m(x)=\int g(x,y)\,\nu(dy)$ be the marginal density of the continuous component of $X$. For every atom $a$ with $p_a>0$, a conditional law of $Y$ given $X=a$ is $$ P(Y\in B\mid X=a)=\frac{Q_a(B)}{p_a}. $$ For almost every $x$ in the continuous component with $m(x)>0$, a conditional law is $$ P(Y\in B\mid X=x)=\frac{\int_B g(x,y)\,\nu(dy)}{m(x)}. $$ On continuous-component values where $m(x)=0$, the conditional law can be assigned arbitrarily without changing the joint distribution.

The atom formula is the usual conditional-probability ratio applied to a positive-probability event. The continuous formula is a ratio of densities: its denominator is the marginal density of $X$, and its numerator is the joint density integrated over the requested values of $Y$. It is not the ratio $P(Y\in B,X=x)/P(X=x)$, since both probabilities in that expression are zero at a typical continuous value.

Why the Density Ratio Gives a Conditional Law

A conditional law must be a probability distribution in $Y$ for each relevant value of $X$, and it must reproduce the joint probabilities when integrated over $X$. The ratio construction has both properties. The continuous formula is a probability measure in $B$ whenever $m(x)>0$, because it is nonnegative, countably additive, and has total mass $m(x)/m(x)=1$.

For completeness, the integral identity that connects the conditional law to the joint law can be checked directly. Let $A$ be a measurable subset of the continuous component and let $B$ be a measurable set of values for $Y$. Using the marginal density $m$,

\begin{aligned} \int_A P(Y\in B\mid X=x)m(x)\,dx &=\int_A \frac{\int_B g(x,y)\,\nu(dy)}{m(x)}m(x)\,dx\\ &=\int_A\int_B g(x,y)\,\nu(dy)\,dx\\ &=P(X\in A,Y\in B). \end{aligned}

The cancellation is made only where $m(x)>0$; points with $m(x)=0$ contribute zero to the integral. The change in order of integration is valid because $g$ is nonnegative, by Tonelli’s theorem. At an atom $a$, the corresponding identity follows directly:

p_a P(Y\in B\mid X=a)=p_a\frac{Q_a(B)}{p_a}=Q_a(B)=P(X=a,Y\in B).

Together these identities verify the conditional-law construction on both components. As usual for conditioning on continuous values, a density ratio determines the conditional law only up to sets of $x$-values having marginal probability zero; different versions on such sets describe the same joint distribution. At a positive-probability atom, by contrast, the conditional law is fixed by the joint slice. $\square$

Worked Example: An Atom and a Continuous Conditioning Value

Let $P(X=0)=0.4$, and let the continuous part of $X$ have density $0.3$ on $0<x<2$. These components have total probability $0.4+0.3(2)=1$. Given $X=0$, suppose $Y=0$ with probability $1/4$ and $Y=2$ with probability $3/4$. Given $X=x>0$, suppose $Y$ is uniform on $(0,x)$.

At the atom, the joint probabilities are

P(X=0,Y=0)=0.4\left(\frac14\right)=0.1,\qquad P(X=0,Y=2)=0.4\left(\frac34\right)=0.3.

Dividing these joint masses by $P(X=0)=0.4$ gives $P(Y=0\mid X=0)=0.25$ and $P(Y=2\mid X=0)=0.75$. Thus, for example, $P(Y\leq 1\mid X=0)=0.25$.

On the continuous component, the marginal density of $X$ is $m(x)=0.3$ for $0<x<2$. The conditional density of $Y$ given $X=x$ is $1/x$ on $0<y<x$, so the joint density there is

g(x,y)=0.3\frac{1}{x}\,\mathbf{1}_{\{0<y<x<2\}}.

At $x=1/2$, the conditional density is $g(1/2,y)/m(1/2)=2$ on $0<y<1/2$. Therefore

P\left(Y\leq\frac14\mid X=\frac12\right) =\int_0^{1/4}2\,dy =\frac12.

The two calculations use different objects: joint masses at $X=0$, but a ratio of densities at $X=1/2$. The same conditional-probability ratio cannot be used for both.

When the Conditional Outcome Is Mixed Too

The variable $Y$ may also have a mixed conditional distribution. The same formulas apply, but its conditional law can contain both point masses and a continuous density. A convenient reference measure for such a $Y$, when its only atom is at zero and its continuous part is positive, is $\nu=\delta_0+\lambda$, where $\delta_0$ assigns unit mass to zero and $\lambda$ is Lebesgue measure on $(0,1)$. A density with respect to this measure records the mass at zero and the ordinary density on $(0,1)$.

Worked Example: A Conditional Law with a Point Mass and a Density

Suppose $P(X=0)=0.25$, and the continuous part of $X$ has density $0.75$ on $0<x<1$. Given $X=0$, let the conditional law of $Y$ be one-half point mass at zero and one-half uniform on $(0,1)$. Given $X=x$ with $0<x<1$, let $Y$ equal zero with probability $1/(1+x)$; otherwise let it be uniform on $(0,1)$.

At the atom of $X$, for a measurable set $B$,

P(Y\in B\mid X=0)=\frac12\mathbf{1}_{\{0\in B\}}+\frac12\int_{B\cap(0,1)}dy.

For $0<x<1$, the conditional law has mass $1/(1+x)$ at zero and density $x/(1+x)$ on $(0,1)$. To check normalization, add the atom and integrate the density:

\frac{1}{1+x}+\int_0^1\frac{x}{1+x}\,dy =\frac{1+x}{1+x}=1.

The joint density on the continuous $X$ component, with respect to $dx\,(\delta_0+\lambda)(dy)$, is $0.75/(1+x)$ at $y=0$, and $0.75x/(1+x)$ for $0<y<1$. Its marginal density in $x$ is $0.75$, since

\frac{0.75}{1+x}+\int_0^1\frac{0.75x}{1+x}\,dy=0.75.

Dividing each joint component by this marginal density gives exactly the stated conditional mass and density. For example, at $x=1/2$, the conditional mass at zero is $2/3$, and the conditional density on $(0,1)$ is $1/3$. Hence, for $0\leq t\leq1$,

P(Y\leq t\mid X=1/2)=\frac23+\int_0^t\frac13\,dy=\frac23+\frac{t}{3}.

The conditional expectation is also obtained from both parts of the law:

E[Y\mid X=1/2]=0\left(\frac23\right)+\int_0^1 y\frac13\,dy=\frac16.

At $X=0$, the conditional expectation is instead $0(1/2)+(1/2)\int_0^1 y\,dy=1/4$. The two conditional laws need not agree, even if one can examine continuous-component values arbitrarily close to the atom.

Checking the Construction by Integration

A conditional density ratio should recover joint probabilities when integrated against the marginal density of the conditioning variable. The following example checks that identity numerically on a continuous interval while also contrasting it with conditioning at an atom.

Worked Example: Verifying a Conditional Tail Probability

Suppose $P(X=0)=1/2$, with the remaining probability having density $1/2$ on $0<x<1$. Given $X=0$, let $Y$ have an exponential distribution with rate $1$. Given $X=x$ for $0<x<1$, let $Y$ have an exponential distribution with rate $1+x$. An exponential distribution with rate $r>0$ has density $r e^{-ry}$ for $y>0$ and tail probability $e^{-rt}$ at $t\geq0$.

At the atom, $P(Y>1\mid X=0)=e^{-1}$. On the continuous component, the joint density is $g(x,y)=\tfrac12(1+x)e^{-(1+x)y}$ for $0<x<1$ and $y>0$; the marginal density of $X$ is $m(x)=1/2$. The ratio $g(x,y)/m(x)$ is therefore $(1+x)e^{-(1+x)y}$, the stated conditional density. Integrating its tail gives

P(Y>1\mid X=x)=\int_1^\infty(1+x)e^{-(1+x)y}\,dy=e^{-(1+x)}.

In particular, at $x=1/2$, this probability is $e^{-3/2}$. Now check a joint probability over an interval of continuous values:

\begin{aligned} P\left(\frac14<X<\frac34,Y>1\right) &=\int_{1/4}^{3/4}P(Y>1\mid X=x)m(x)\,dx\\ &=\frac12\int_{1/4}^{3/4}e^{-(1+x)}\,dx\\ &=\frac12\left(e^{-5/4}-e^{-7/4}\right). \end{aligned}

The factor $1/2$ is the marginal density of $X$, not a conditional probability. The calculation verifies that the conditional tail probabilities, weighted by the continuous marginal density, reproduce the joint probability.

Common Mistakes and Practical Checks

A common error is to use a joint density ratio at an atom. In the exponential example, $X=0$ has positive probability, but the continuous-component density formula does not describe its conditional law. The correct calculation at zero is the joint slice divided by $P(X=0)$. Conversely, applying a ratio of event probabilities at $X=x$ for a continuous value is invalid when $P(X=x)=0$; the appropriate construction uses the joint and marginal densities.

Another pitfall is to assume the conditional law at an atom must be the limit of conditional laws at nearby continuous values. The mixed-outcome example shows why that is not justified: the atom had its own specified joint slice, while the density ratio described the continuous component. Finally, if the outcome $Y$ is mixed, normalization must include both its point masses and the integral of its density. Checking only the density can miss probability assigned to an atom.

Key takeaway: Separate the conditioning variable into its atoms and continuous component. At an atom, divide the joint slice by the atom probability. On the continuous component, divide the joint density by the marginal density, and check that the resulting conditional law integrates or sums to one.

Check Your Understanding

Use the distinction between joint masses and density ratios in each question.

If $P(X=a)>0$, what joint quantity is divided by $P(X=a)$ to obtain $P(Y\in B\mid X=a)$?
For a continuous-component value $x$ with marginal density $m(x)>0$, write the density-ratio formula for $P(Y\in B\mid X=x)$.
Suppose $Y\mid X=x$ has mass $0.6$ at zero and density $0.4$ on $(0,1)$. Check that this is a probability distribution.
Why does a conditional density ratio determine the conditional law only almost everywhere on the continuous component?
In the exponential example, compute $P(Y>2\mid X=0)$ and $P(Y>2\mid X=1/2)$.