Tutorials › Multivariate Statistics (Graduate) › Worked Problems in Marginalization and Conditioning

Home  /  Tutorials  /  Multivariate Laws and Transformations

Multivariate Laws and Transformations · Tutorial 70 of 975

Worked Problems in Marginalization and Conditioning

Learn to derive marginal and conditional distributions from joint densities whose support is not rectangular, and verify each result carefully.

Beginner 10 min read

What You'll Learn

  • Find marginal densities by integrating over the correct cross-section of a nonrectangular support
  • Derive conditional densities from joint and marginal densities
  • Identify the support of a conditional law from the joint support
  • Check marginal and conditional normalization directly
  • Recognize why pointwise conditional-density claims require care at exceptional conditioning values

Why the Support Determines the Calculation

For a joint density on a rectangle, integration limits often follow directly from the endpoints of each coordinate. On a nonrectangular support, those limits change with the coordinate being held fixed. The key task in marginalization and conditioning is therefore to identify the relevant cross-section of the joint support before integrating or forming a density ratio.

This tutorial develops that method through worked calculations on triangular and curved regions. In each case, the bounds are part of the probability model: integrating over a rectangular region that includes points outside the support can give incorrect marginal and conditional distributions.

Definitions and a Normalization Result

Definition: Let \((X,Y)\) have joint density \(f_{X,Y}(x,y)\) with respect to two-dimensional Lebesgue measure. Its marginal densities are $$ f_X(x)=\int_{\mathbb R} f_{X,Y}(x,y)\,dy, \qquad f_Y(y)=\int_{\mathbb R} f_{X,Y}(x,y)\,dx. $$ For a value \(x\) such that \(0<f_X(x)<\infty\), a conditional density of \(Y\) given \(X=x\) is $$ f_{Y\mid X}(y\mid x)=\frac{f_{X,Y}(x,y)}{f_X(x)}. $$ The analogous formula defines \(f_{X\mid Y}(x\mid y)\) when \(0<f_Y(y)<\infty\). These ratio formulas specify conditional laws for almost every conditioning value under the corresponding marginal distribution; values at exceptional points can be assigned separately.

The joint density is understood up to changes on sets of two-dimensional measure zero. Consequently, the marginal density, including its value at a particular point, can depend on the chosen version of the joint density. The marginal \(f_X\) is finite for almost every \(x\), since its integral is one, but it need not be finite at every individual \(x\). This is why the pointwise ratio formula is stated only when the denominator is positive and finite.

Theorem: For almost every \(x\) under the distribution of \(X\), the conditional density \(f_{Y\mid X}(y\mid x)\) integrates to one. More precisely, this holds at every \(x\) for which \(0<f_X(x)<\infty\). For every Borel set \(A\subseteq\mathbb R\), $$ P(Y\in A\mid X=x)=\int_A f_{Y\mid X}(y\mid x)\,dy $$ is a version of the conditional probability, in the sense that for every Borel set \(B\subseteq\mathbb R\), $$ P(X\in B,\ Y\in A) =\int_B P(Y\in A\mid X=x)f_X(x)\,dx. $$

Proof. Since the joint density is nonnegative, Tonelli’s theorem gives $$ \int_{\mathbb R} f_X(x)\,dx =\int_{\mathbb R}\int_{\mathbb R}f_{X,Y}(x,y)\,dy\,dx=1. $$ Thus \(f_X(x)<\infty\) for almost every \(x\). At any \(x\) with \(0<f_X(x)<\infty\), the ratio formula and the definition of the marginal yield $$ \int_{\mathbb R} f_{Y\mid X}(y\mid x)\,dy =\frac{\int_{\mathbb R} f_{X,Y}(x,y)\,dy}{f_X(x)} =1. $$ For the conditional-probability identity, define the ratio at such \(x\), and assign any probability distribution at other \(x\). The exceptional values with \(f_X(x)=0\) have zero measure under the distribution of \(X\); values where \(f_X(x)=\infty\) also form a Lebesgue-null set and hence have zero probability under that distribution. Tonelli’s theorem now gives $$ \begin{aligned} \int_B P(Y\in A\mid X=x)f_X(x)\,dx &=\int_{B\cap\{0<f_X<\infty\}}\int_A f_{X,Y}(x,y)\,dy\,dx\\ &=\int_B\int_A f_{X,Y}(x,y)\,dy\,dx\\ &=P(X\in B,\ Y\in A). \end{aligned} $$ The omitted exceptional \(x\)-values contribute zero to the probability integral. This is the defining identity for a version of the conditional probability. \(\square\)

In an actual calculation, begin by writing the support as inequalities. To obtain \(f_X(x)\), hold \(x\) fixed and determine the allowed values of \(y\); to obtain \(f_Y(y)\), hold \(y\) fixed and determine the allowed values of \(x\). A conditional density has the same cross-section as the joint density at the conditioning value, with its height divided by the appropriate marginal.

Worked Problems on Triangular Supports

Worked Example: A Triangle with \(0<y<x<1\)

Suppose the joint density is \(f_{X,Y}(x,y)=2\) on \(0<y<x<1\), and zero elsewhere. First check that it is a density: the support has area \(\int_0^1 x\,dx=1/2\), so the integral of the constant density is \(2(1/2)=1\).

Marginal of \(X\). Fix \(x\). The support inequalities require \(0<y<x\), which is possible for \(0<x<1\). Therefore

$$ f_X(x)=\int_0^x 2\,dy=2x,\qquad 0<x<1. $$

The marginal is zero outside this interval, and \(\int_0^1 2x\,dx=1\). For the other marginal, fix \(y\). The inequalities require \(y<x<1\), so

$$ f_Y(y)=\int_y^1 2\,dx=2(1-y),\qquad 0<y<1. $$

Again, \(\int_0^1 2(1-y)\,dy=1\). Dividing by the appropriate marginal gives

$$ f_{Y\mid X}(y\mid x)=\frac{1}{x},\quad 0<y<x,\ 0<x<1; \qquad f_{X\mid Y}(x\mid y)=\frac{1}{1-y},\quad y<x<1,\ 0<y<1. $$

Each conditional density is uniform on its support interval. Direct checks give \(\int_0^x (1/x)\,dy=1\) and \(\int_y^1 (1/(1-y))\,dx=1\). For example, at a fixed \(x\in(0,1)\), the conditional probability \(P(Y<x/2\mid X=x)\) is \(\int_0^{x/2}(1/x)\,dy=1/2\).

Worked Example: A Uniform Density on a Simplex

Let \(f_{X,Y}(x,y)=2\) on the triangle \(x>0,\ y>0,\ x+y<1\), and zero elsewhere. Its area is \(\int_0^1(1-x)\,dx=1/2\), so the density integrates to one. Fixing \(x\) leaves \(0<y<1-x\), giving

$$ f_X(x)=\int_0^{1-x}2\,dy=2(1-x),\qquad 0<x<1. $$

Fixing \(y\) instead leaves \(0<x<1-y\), and hence

$$ f_Y(y)=\int_0^{1-y}2\,dx=2(1-y),\qquad 0<y<1. $$

Both marginals integrate to one. The conditional densities are

$$ f_{Y\mid X}(y\mid x)=\frac{1}{1-x},\quad 0<y<1-x,\ 0<x<1; \qquad f_{X\mid Y}(x\mid y)=\frac{1}{1-y},\quad 0<x<1-y,\ 0<y<1. $$

For example, given \(X=x\), the conditional probability that \(Y\) exceeds half the available interval is $$ P\left(Y>\frac{1-x}{2}\mid X=x\right) =\int_{(1-x)/2}^{1-x}\frac{1}{1-x}\,dy =\frac{1}{2}. $$ The upper endpoint \(1-x\) is essential: using the fixed bound 1 would include points outside the joint support whenever \(x>0\).

A Curved Support and Reverse Conditioning

Worked Example: The Region \(x^2<y<x\)

Consider the region \(0<x<1\) and \(x^2<y<x\). Its area is $$ \int_0^1(x-x^2)\,dx=\frac12-\frac13=\frac16. $$ Thus the uniform density on this region is \(f_{X,Y}(x,y)=6\), and its total integral is \(6(1/6)=1\).

For \(f_X\), hold \(x\) fixed. The allowed vertical section is \(x^2<y<x\), so

$$ f_X(x)=\int_{x^2}^x6\,dy=6(x-x^2),\qquad 0<x<1. $$

Its integral is \(6(1/2-1/3)=1\). To find \(f_Y\), solve both support inequalities for \(x\): \(x^2<y\) with \(x>0\) implies \(x<\sqrt y\), and \(y<x\) implies \(x>y\). For \(0<y<1\), the horizontal section is therefore \(y<x<\sqrt y\). It follows that

$$ f_Y(y)=\int_y^{\sqrt y}6\,dx=6(\sqrt y-y),\qquad 0<y<1. $$

This marginal also integrates to one: $$ \int_0^1 6(\sqrt y-y)\,dy =6\left(\frac23-\frac12\right)=1. $$ The conditional densities are consequently

$$ f_{Y\mid X}(y\mid x)=\frac{1}{x-x^2},\quad x^2<y<x,\ 0<x<1; \qquad f_{X\mid Y}(x\mid y)=\frac{1}{\sqrt y-y},\quad y<x<\sqrt y,\ 0<y<1. $$

The first conditional density integrates to \((x-x^2)/(x-x^2)=1\); the second integrates to \((\sqrt y-y)/(\sqrt y-y)=1\). These checks also reveal the correct interval lengths. For instance, given \(Y=y\), the allowable \(X\)-values lie between \(y\) and \(\sqrt y\), not between \(0\) and \(1\).

How to Check a Marginal or Conditional Result

A reliable solution checks three things before interpreting the answer. First, verify that the bounds describe exactly the relevant cross-section of the joint support. Second, verify that each marginal integrates to one. Third, verify that each conditional density integrates to one over its conditional support. A formula that is algebraically plausible but fails any of these checks is not the desired distribution.

1
Write the support as inequalities.
Keep track of the allowed ranges of both coordinates, including how one coordinate constrains the other.
2
Take the appropriate cross-section.
Fix the coordinate whose marginal or conditional distribution is being considered, then solve the inequalities for the remaining coordinate.
3
Integrate or divide.
Integrate the joint density over the cross-section to get a marginal; divide the joint density by that marginal to get the conditional density where the marginal is positive and finite.
4
Check normalization and support.
Integrate the result over its stated range and confirm that points outside the relevant support receive density zero.

The conditioning value affects the allowed range of the other variable. This makes the support check especially useful: in the simplex example, for fixed \(x\), the conditional range is \(0<y<1-x\); in the curved example, for fixed \(y\), it is \(y<x<\sqrt y\). A conditional density cannot assign positive density beyond those ranges.

Common Pitfalls

  • Using rectangular bounds automatically. Integrating \(y\) from 0 to 1 in the simplex example ignores the constraint \(y<1-x\). The correction is to solve the support inequalities after fixing \(x\).
  • Keeping the joint density’s height without dividing. A joint density is not generally a conditional density. Divide by the marginal density at the conditioning value, then check that the conditional density integrates to one.
  • Claiming pointwise normalization when the marginal is infinite. Although \(f_X\) is finite almost everywhere, it can be infinite at an individual point under a particular density version. The ratio argument proves pointwise normalization only where \(0<f_X(x)<\infty\); conditional laws are specified only up to marginal-null conditioning values.
  • Ignoring the conditional support. A density ratio does not make values outside the joint support possible. Its numerator is zero there, and the conditional density must be interpreted on the corresponding cross-section.
Key takeaway: For a nonrectangular joint support, derive the marginal and conditional laws from its cross-sections. State the resulting support explicitly, divide by a positive finite marginal for the conditional density, and verify each normalization directly.

Check Your Understanding

For each problem, identify the relevant cross-section before carrying out the calculation.

  1. For a density constant on \(0<y<x<1\), what are the bounds for computing \(f_Y(y)\), and why?
  2. In the support \(x>0,\ y>0,\ x+y<1\), state the support of \(Y\mid X=x\) for a fixed \(x\in(0,1)\).
  3. For the region \(x^2<y<x\), derive the allowable interval for \(x\) when \(y\) is fixed in \((0,1)\).
  4. Why does dividing by \(f_X(x)\) give a normalized conditional density when \(0<f_X(x)<\infty\)?
  5. Why should a pointwise ratio formula not be asserted at every \(x\) without qualification?

Previous

Next

Back to Tutorials
Clinical Biostats