Conditional Distributions Characterize Independence

Conditional Distributions and Independence

Independence says that learning one random variable gives no information about the other. A conditional distribution makes this idea precise: compare the distribution of $X$ after learning $Y$ with the marginal distribution of $X$. For discrete variables, this comparison can be made at every value of $Y$ that has positive probability. For continuous variables, individual values usually have probability zero, so the corresponding statement requires more care.

Throughout, $X$ and $Y$ are real-valued random variables. We use the standard meaning of independence: $X$ and $Y$ are independent if $P(X\in A,Y\in B)=P(X\in A)P(Y\in B)$ for all Borel sets $A,B\subseteq\mathbb R$.

The Criterion for Discrete Variables

Theorem: Suppose $X$ and $Y$ are discrete. They are independent if and only if, for every value $y$ with $P(Y=y)>0$ and every set $A$ of possible values of $X$, $$ P(X\in A\mid Y=y)=P(X\in A). $$ Equivalently, at every such $y$, the conditional PMF of $X$ given $Y=y$ equals the marginal PMF of $X$ at every possible $x$.

The restriction $P(Y=y)>0$ is essential: the elementary conditional probability $P(E\mid Y=y)$ is defined by dividing by $P(Y=y)$, so it is not defined by this formula when that probability is zero.

To prove the forward direction, suppose $X$ and $Y$ are independent and $P(Y=y)>0$. For any set $A$,

P(X\in A\mid Y=y) =\frac{P(X\in A,Y=y)}{P(Y=y)} =\frac{P(X\in A)P(Y=y)}{P(Y=y)} =P(X\in A).

For the reverse direction, suppose the stated conditional equality holds for every positive-probability value $y$. For any sets $A$ and $B$, sum over the values $y$ in $B$ with positive probability:

\begin{aligned} P(X\in A,Y\in B) &=\sum_{y\in B}P(X\in A,Y=y)\\ &=\sum_{\substack{y\in B\\P(Y=y)>0}} P(X\in A\mid Y=y)P(Y=y)\\ &=\sum_{\substack{y\in B\\P(Y=y)>0}} P(X\in A)P(Y=y)\\ &=P(X\in A)P(Y\in B). \end{aligned}

Values with zero probability contribute nothing to either sum. The resulting factorization for every $A,B$ is the definition of independence. $\square$

Worked Example: Checking a Discrete Conditional Distribution

Let $X,Y\in\{0,1\}$ have joint PMF $p(x,y)=P(X=x,Y=y)$ given by

	$Y=0$	$Y=1$	Marginal for $X$
$X=0$	0.42	0.28	0.70
$X=1$	0.18	0.12	0.30
Marginal for $Y$	0.60	0.40	1.00

Both values of $Y$ have positive probability. Conditional on $Y=1$, the conditional probabilities are

P(X=0\mid Y=1)=\frac{0.28}{0.40}=0.70,\qquad P(X=1\mid Y=1)=\frac{0.12}{0.40}=0.30.

These are the marginal probabilities $P(X=0)=0.70$ and $P(X=1)=0.30$. Conditional on $Y=0$, they are

P(X=0\mid Y=0)=\frac{0.42}{0.60}=0.70,\qquad P(X=1\mid Y=0)=\frac{0.18}{0.60}=0.30.

Thus the conditional PMF of $X$ equals its marginal PMF for both positive-probability values of $Y$. The theorem establishes that $X$ and $Y$ are independent. Equivalently, each joint cell is the product of its row and column marginals; for example, $0.12=(0.30)(0.40)$.

Continuous Variables: Equality Almost Everywhere

If $Y$ is continuous, then typically $P(Y=y)=0$ for every individual value $y$. The ratio used for elementary conditioning on an event of positive probability therefore cannot define $P(X\in A\mid Y=y)$. Instead, probability theory uses a regular conditional distribution: a conditional law of $X$ given $Y=y$, defined as a probability distribution in $X$ and measurable as a function of $y$, that satisfies

P(X\in A,Y\in B)=\int_B K(y,A)\,P_Y(dy)

for all Borel sets $A,B$. Here $P_Y$ is the marginal distribution of $Y$, and $K(y,A)$ denotes the conditional probability assigned to $A$ at $y$. For real-valued variables such conditional distributions exist. They are determined only for $P_Y$-almost every $y$: their values can be changed on a set of $y$'s having marginal probability zero without changing the defining identity.

Theorem: For real-valued $X,Y$ and a regular conditional distribution $K$ of $X$ given $Y$, $X$ and $Y$ are independent if and only if $K(y,\cdot)=P_X(\cdot)$ for $P_Y$-almost every $y$. Here equality means equality of probability distributions.

For the forward direction, assume independence. Fix a Borel set $A$. The defining identity for $K$ gives, for every Borel $B$,

\int_B K(y,A)\,P_Y(dy) =P(X\in A,Y\in B) =P_X(A)P_Y(B) =\int_B P_X(A)\,P_Y(dy).

If two measurable functions have equal integrals over every measurable set, they are equal almost everywhere. Thus $K(y,A)=P_X(A)$ for $P_Y$-almost every $y$, for each fixed $A$. To obtain equality of the whole distributions on one common full-probability set, first take $A$ from the countable collection $(-\infty,q]$ with rational $q$. Intersect the corresponding countably many full-probability sets. On that intersection the two distribution functions agree at every rational $q$, which determines a probability distribution on $\mathbb R$. Therefore $K(y,\cdot)=P_X(\cdot)$ there.

Conversely, suppose these conditional distributions equal $P_X$ for $P_Y$-almost every $y$. For arbitrary Borel $A,B$, the defining identity then gives

P(X\in A,Y\in B) =\int_B K(y,A)\,P_Y(dy) =\int_B P_X(A)\,P_Y(dy) =P_X(A)P_Y(B).

This is independence. $\square$

Conditional Densities

If $(X,Y)$ has a joint density $f_{X,Y}$ and $Y$ has marginal density $f_Y$, then, wherever the ratio is defined, the conditional density is given by

f_{X\mid Y}(x\mid y)=\frac{f_{X,Y}(x,y)}{f_Y(y)}.

This formula supplies a version of the conditional density for $P_Y$-almost every $y$ with $f_Y(y)>0$; it does not turn the event $Y=y$ into a positive-probability event. In this setting independence is equivalent to the conditional density being $f_X(x)$ for Lebesgue-almost every $x$, for $P_Y$-almost every $y$. The almost-everywhere qualifications matter because densities themselves are defined only up to changes on sets of Lebesgue measure zero.

Worked Example: Independent Continuous Variables

Suppose $X$ and $Y$ are independent Uniform$(0,1)$ random variables. Their joint density and the marginal density of $Y$ are

f_{X,Y}(x,y)=\mathbf{1}_{(0,1)}(x)\mathbf{1}_{(0,1)}(y), \qquad f_Y(y)=\mathbf{1}_{(0,1)}(y).

For $0<y<1$, the conditional density formula gives

f_{X\mid Y}(x\mid y) =\frac{\mathbf{1}_{(0,1)}(x)\mathbf{1}_{(0,1)}(y)} {\mathbf{1}_{(0,1)}(y)} =\mathbf{1}_{(0,1)}(x) =f_X(x).

Thus the conditional law is Uniform$(0,1)$ for almost every $y$ under the distribution of $Y$, just as the marginal law is. One may define the conditional law to be Uniform$(0,1)$ at every $y$, including values outside $(0,1)$, to obtain a convenient version; those outside values have no effect on the conditional-distribution identity because $P(Y\notin(0,1))=0$.

Worked Example: A Conditional Normal Distribution Reveals Dependence

Let $U,V$ be independent $N(0,1)$ random variables and define

X=U,\qquad Y=\frac12 U+\frac{\sqrt{3}}{2}V.

Both marginals are $N(0,1)$. The joint density, obtained from this linear transformation, is

f_{X,Y}(x,y)=\frac{1}{\pi\sqrt{3}} \exp\left\{-\frac{2}{3}(x^2-xy+y^2)\right\}.

Since $f_Y(y)=(2\pi)^{-1/2}\exp(-y^2/2)$, dividing the joint density by the marginal density and completing the square gives

\begin{aligned} f_{X\mid Y}(x\mid y) &=\frac{1}{\sqrt{2\pi(3/4)}} \exp\left\{-\frac{(x-y/2)^2}{2(3/4)}\right\}. \end{aligned}

Hence the conditional law is $N(y/2,3/4)$, whereas the marginal law is $N(0,1)$. For example, at $y=1$, the conditional probability of $X>0$ is

P(X>0\mid Y=1)=\Phi\left(\frac{1}{\sqrt{3}}\right)>\frac12=P(X>0),

where $\Phi$ is the standard normal distribution function. More importantly, the conditional mean $y/2$ differs from the marginal mean for every $y\ne 0$. Since $Y\sim N(0,1)$, the set $y\ne0$ has probability one. The conditional law therefore differs from the marginal law on a set of full $P_Y$-probability, so $X$ and $Y$ are dependent.

Common Pitfalls and Why the Criterion Matters

Conditioning on a zero-probability value as if it were an ordinary event. For a continuous $Y$, the expression $P(X\in A,Y=y)/P(Y=y)$ is not a definition of the conditional probability at $y$. Use a regular conditional distribution or the conditional-density formula.
Demanding pointwise equality at every continuous conditioning value. Conditional distributions are only determined up to $P_Y$-null sets. Independence requires equality almost everywhere, not equality for every possible version at every real $y$.
Confusing a positive density with a positive point probability. The fact that $f_Y(y)>0$ does not mean $P(Y=y)>0$. A density describes probability across intervals, not probability at a single point.
Checking only one summary of a conditional law. Equal conditional means alone do not establish that two distributions are equal. The criterion compares the full conditional distribution with the full marginal distribution.

The central test is whether conditioning changes the distribution, not merely whether it changes a particular average. In discrete problems, compare the conditional PMF at every positive-probability conditioning value. In continuous problems, compare conditional densities or conditional laws with the appropriate almost-everywhere interpretation. This gives a direct way to translate the meaning of independence into a statement about what is learned from observing $Y$.

Check Your Understanding

Use the distinction between positive-probability conditioning events and continuous conditional distributions in your answers.

For discrete $X,Y$, why is the conditional-distribution criterion restricted to values $y$ with $P(Y=y)>0$?
A discrete variable $Y$ has two positive-probability values. If $X\mid Y=y$ has the same PMF as $X$ for both, what theorem lets you conclude independence?
Why does $f_Y(y)>0$ not imply $P(Y=y)>0$ when $Y$ is continuous?
In the independent Uniform$(0,1)$ example, what is the conditional density of $X$ for $0<y<1$, and how does it compare with $f_X$?
In the normal example, what feature of the conditional law of $X$ given $Y=y$ shows that it differs from the marginal law for almost every $y$?

	\(Y=0\)	\(Y=1\)	Marginal for \(X\)
\(X=0\)	0.42	0.28	0.70
\(X=1\)	0.18	0.12	0.30
Marginal for \(Y\)	0.60	0.40	1.00