Tutorials › Statistical Theory I - Probability and Distribution Theory › Conditioning on a Continuous Value

Conditional Distribution Theory · Tutorial 254 of 994

Conditioning on a Continuous Value

Learn how density ratios define conditional distributions at continuous conditioning values, and why those pointwise laws require careful interpretation.

Intermediate 11 min read

What You'll Learn

  • Distinguish conditioning on an event from specifying a value of a continuous random variable.
  • Construct conditional laws from a joint density and its marginal density.
  • Verify the probability-kernel and conditional-distribution properties.
  • Interpret density-ratio laws at values with positive, finite marginal density.
  • Recognize why pointwise conditional laws are versions rather than uniquely determined probabilities.
  • Identify and handle exceptional conditioning values without sacrificing measurability.

Why an Exact Continuous Value Needs a Different Definition

For an event \(B\) with \(P(B)>0\), ordinary conditional probability is defined by \(P(A\mid B)=P(A\cap B)/P(B)\). But if \(Y\) has a continuous distribution, then \(P(Y=y)=0\) for every fixed \(y\). The ratio \(P(X\in A,\ Y=y)/P(Y=y)\) is therefore not a definition of \(P(X\in A\mid Y=y)\): it has a zero denominator.

Conditional distributions given a continuous value are instead described using a conditional probability kernel. When a joint density is available, a useful version of this kernel is obtained by dividing the joint density by the marginal density. This gives a probability distribution in \(x\) at each conditioning value where the marginal density is positive and finite. It does not turn the probability of the zero-probability event \(\{Y=y\}\) into an ordinary conditional probability.

Definition: A conditional distribution of \(X\) given \(Y\) is a function \(K(y,A)\), for \(y\in\mathbb R\) and Borel sets \(A\subseteq\mathbb R\), such that: (1) for each fixed \(y\), \(A\mapsto K(y,A)\) is a probability measure; (2) for each fixed Borel \(A\), \(y\mapsto K(y,A)\) is measurable; and (3) for every pair of Borel sets \(A,B\), $$ P(X\in A,\ Y\in B)=\int_B K(y,A)\,P_Y(dy), $$ where \(P_Y\) is the distribution of \(Y\). The function \(K(y,A)\) is a version of the conditional probability of \(X\in A\) given \(Y=y\).

The integral identity is the defining link to the joint distribution. It averages the conditional laws over the values of \(Y\), using the actual distribution \(P_Y\) as the weighting measure. A single value \(y\) with \(P(Y=y)=0\) receives no weight in this identity. Consequently, the identity cannot in general determine \(K(y,A)\) uniquely at each such value.

The Density-Ratio Conditional Law

Suppose \((X,Y)\) has a jointly measurable joint density \(f_{X,Y}(x,y)\) with respect to two-dimensional Lebesgue measure. Choose this density as a measurable function and define the marginal density by integrating its sections: $$ f_Y(y)=\int_{\mathbb R} f_{X,Y}(x,y)\,dx. $$

This marginal is a density of \(Y\), and the displayed definition fixes a particular value of \(f_Y(y)\) from the chosen joint-density version. Let \(D=\{y:0<f_Y(y)<\infty\}\). For \(y\in D\), define $$ K(y,A)=\frac{\displaystyle\int_A f_{X,Y}(x,y)\,dx}{f_Y(y)} =\int_A \frac{f_{X,Y}(x,y)}{f_Y(y)}\,dx. $$

The function \(f_{X,Y}(x,y)/f_Y(y)\), as a function of \(x\) at a fixed \(y\in D\), is a conditional density of \(X\) given \(Y=y\) for this version. Its integral over the entire real line is one by the definition of \(f_Y(y)\). Thus, at these values, it defines a probability distribution rather than an unnormalized slice of the joint density.

The set outside \(D\) needs explicit handling. Since \(f_Y\) is measurable, \(D\) is measurable. Fix any probability measure \(Q\) on \(\mathbb R\), and set \(K(y,A)=Q(A)\) for \(y\notin D\). This constant assignment is measurable in \(y\) for every Borel \(A\). Other assignments are allowed only if they preserve that measurability. In particular, an arbitrary point-by-point assignment on the exceptional set is not automatically a probability kernel.

Theorem: With the construction above, \(K\) is a conditional distribution of \(X\) given \(Y\). At every \(y\in D\), its conditional density is \(f_{X,Y}(x,y)/f_Y(y)\).

Proof. For \(y\in D\), nonnegativity of the joint density and the identity \(\int_{\mathbb R}f_{X,Y}(x,y)\,dx=f_Y(y)\) show that \(A\mapsto K(y,A)\) is a probability measure. For \(y\notin D\), this holds because \(K(y,\cdot)=Q(\cdot)\). For a fixed Borel set \(A\), the parameter integral \(y\mapsto\int_A f_{X,Y}(x,y)\,dx\) is measurable by the measurability of the nonnegative integrand and Tonelli's theorem. Dividing by the positive measurable \(f_Y\) on \(D\), and using the constant \(Q(A)\) outside \(D\), shows that \(y\mapsto K(y,A)\) is measurable.

The density \(f_Y\) integrates to one, so \(P_Y(B)=\int_B f_Y(y)\,dy\). Also, \(P_Y(D^c)=0\): on \(\{f_Y=0\}\) its integral is zero, and \(\{f_Y=\infty\}\) has Lebesgue measure zero because \(f_Y\) is integrable. Therefore, for Borel \(A,B\), $$ \begin{aligned} \int_B K(y,A)\,P_Y(dy) &=\int_{B\cap D} \frac{\int_A f_{X,Y}(x,y)\,dx}{f_Y(y)}f_Y(y)\,dy\\ &=\int_{B\cap D}\int_A f_{X,Y}(x,y)\,dx\,dy\\ &=\int_B\int_A f_{X,Y}(x,y)\,dx\,dy\\ &=P(X\in A,\ Y\in B). \end{aligned} $$ The third equality holds because \(D^c\) has zero \(P_Y\)-probability, and hence carries no joint probability. Tonelli's theorem justifies the iterated integral of the nonnegative density. All three kernel properties hold, proving the theorem. \(\square\)

Worked Examples

Worked Example: Independent Normal Variables

Let \(X\sim N(0,1)\) and \(Y\sim N(2,4)\) be independent. Find the density-ratio conditional law of \(X\) given \(Y=y\), for a value \(y\) where \(f_Y(y)>0\).

Independence gives \(f_{X,Y}(x,y)=f_X(x)f_Y(y)\). The two densities are $$ f_X(x)=\frac{1}{\sqrt{2\pi}}e^{-x^2/2}, \qquad f_Y(y)=\frac{1}{\sqrt{8\pi}}e^{-(y-2)^2/8}. $$

Since the marginal density is positive and finite for every real \(y\), division gives $$ f_{X\mid Y=y}(x)=\frac{f_{X,Y}(x,y)}{f_Y(y)} =\frac{f_X(x)f_Y(y)}{f_Y(y)} =f_X(x). $$

Thus this version assigns \(N(0,1)\) to \(X\) at every \(y\). The result agrees with independence: learning the value of \(Y\) does not change the conditional law. It is the density-ratio kernel that gives the precise meaning to that statement here, not a ratio involving \(P(Y=y)\).

Worked Example: A Sum with Gaussian Noise

Let \(X\sim N(0,1)\) and \(\varepsilon\sim N(0,1)\) be independent, and set \(Y=X+\varepsilon\). Find the conditional density of \(X\) given \(Y=y\).

The transformation from \((X,\varepsilon)\) to \((X,Y)\) has Jacobian determinant one, since \(\varepsilon=y-x\). Therefore $$ f_{X,Y}(x,y)=\frac{1}{2\pi}\exp\left(-\frac{x^2+(y-x)^2}{2}\right). $$

To integrate out \(x\), complete the square: $$ x^2+(y-x)^2=2x^2-2xy+y^2 =2\left(x-\frac y2\right)^2+\frac{y^2}{2}. $$

It follows that \(Y\sim N(0,2)\), with density $$ f_Y(y)=\frac{1}{\sqrt{4\pi}}e^{-y^2/4}. $$

Now divide the joint density by this marginal density: $$ \begin{aligned} f_{X\mid Y=y}(x) &=\frac{1}{2\pi}e^{-(x-y/2)^2-y^2/4} \left(\sqrt{4\pi}e^{y^2/4}\right)\\ &=\frac{1}{\sqrt{\pi}}e^{-(x-y/2)^2}. \end{aligned} $$

This is the density of \(N(y/2,1/2)\): the normal density with mean \(y/2\) and variance \(1/2\). For example, at \(y=2\), the conditional law in this version is \(N(1,1/2)\). The calculation uses the joint and marginal densities, not the undefined ordinary conditional-probability ratio for the event \(Y=2\).

Worked Example: A Conditional Law on a Triangular Support

Suppose \((X,Y)\) has joint density \(f_{X,Y}(x,y)=2\) when \(0<x<y<1\), and zero otherwise. Find the conditional density for \(0<y<1\), and explain what happens at \(y=0\).

For \(0<y<1\), integrate over the possible \(x\)-values: $$ f_Y(y)=\int_0^y 2\,dx=2y. $$

Consequently, for \(0<y<1\), $$ f_{X\mid Y=y}(x)=\frac{2}{2y}=\frac1y,\qquad 0<x<y, $$

and it is zero outside that interval. It is normalized because \(\int_0^y (1/y)\,dx=1\). At \(y=0\), the marginal density is zero, so the ratio is not defined. The distribution of \(Y\) gives that value probability zero; a conditional kernel can be completed there by a fixed probability measure, such as a point mass at zero. This choice does not alter the kernel identity, and choosing the same measure at every exceptional value ensures measurability.

What the Formula Does—and Does Not—Determine

The density-ratio theorem gives a valid conditional-distribution version. It does not say that a uniquely determined probability distribution is attached to every individual value \(y\). Indeed, if \(K_1\) and \(K_2\) are two conditional distributions, then for each fixed Borel set \(A\) their values \(K_1(y,A)\) and \(K_2(y,A)\) agree for \(P_Y\)-almost every \(y\), but they may differ on a \(P_Y\)-null set. The defining integral averages over \(P_Y\) and cannot distinguish changes confined to such a set, provided the resulting kernel remains measurable.

For a continuous \(Y\), every singleton is \(P_Y\)-null. The density-ratio formula supplies a particularly useful choice at values with \(0<f_Y(y)<\infty\), but its pointwise value still depends on the chosen density version. To make the construction precise, the theorem defined \(f_Y(y)\) by integrating the chosen joint-density version. If one instead changes a density on a set of Lebesgue measure zero, its values on a particular horizontal slice can change, even though the joint distribution is unchanged. Such changes can affect a ratio at that specific \(y\).

Worked Example: Why a Null-Event Ratio Fails

Take the independent normal variables of the first example and fix \(y=1\). Independence gives \(P(X\in[0,1],Y=1)=0\) and \(P(Y=1)=0\), since \(Y\) has a density. The ordinary event-conditioning ratio would be \(0/0\), which has no value. In contrast, the density ratio gives $$ \frac{f_{X,Y}(x,1)}{f_Y(1)}=f_X(x), $$

so the chosen conditional kernel at \(y=1\) is \(N(0,1)\). This is a well-defined value of a measurable kernel satisfying the integral identity. It is not obtained by assigning an ordinary conditional probability to the event \(Y=1\).

Common Pitfall and Why the Distinction Matters

Wrong reasoning: “Because \(\{Y=y\}\) has probability zero, the conditional law given \(Y=y\) is undefined, so there can be no meaningful conditional distribution at that value.” The first clause is correct for ordinary event conditioning, but the conclusion is too strong. A conditional distribution is defined through a kernel and an integral identity; a density ratio gives a useful version at values where the marginal density is positive and finite.

A related error is to assign arbitrary probability distributions on exceptional values and assume the result is automatically a conditional distribution. Even if those assignments do not change the integral identity, a nonmeasurable assignment can fail the kernel requirement. A safe construction is to assign one fixed probability measure throughout the measurable exceptional set. More generally, any exceptional-value assignments must make \(y\mapsto K(y,A)\) measurable for every Borel \(A\).

This distinction matters whenever a statistical model describes the distribution of one quantity at a specified value of another—for example, a response at a fixed covariate value. The density ratio supplies a coherent conditional law, while the version and measurability qualifications prevent us from treating a zero-probability event as though it had positive probability.

Check Your Understanding

Answer each question using the definitions and reasoning above.

  1. Why is \(P(X\in A,\ Y=y)/P(Y=y)\) not a definition of a conditional probability when \(Y\) has a continuous distribution?
  2. For a joint density, what condition on \(f_Y(y)\) makes the density-ratio formula a probability density in \(x\)?
  3. Why does assigning the same fixed probability measure throughout the exceptional set preserve measurability of the kernel?
  4. In the triangular-support example, verify that the conditional density integrates to one for \(0<y<1\).
  5. What aspect of a conditional distribution can differ between versions, and with respect to which measure must the versions agree almost everywhere?
Back to Tutorials