Tutorials › Probability Theory (Graduate) › Marginal Laws from Joint Distributions

Home  /  Tutorials  /  Product Measures and Fubini

Product Measures and Fubini · Tutorial 198 of 946

Marginal Laws from Joint Distributions

Use coordinate projections to define marginal laws and section integrals to compute marginal probabilities from joint laws.

Intermediate 10 min read

What You'll Learn

  • Define coordinate and block marginals as pushforwards of a joint law
  • Compute marginal event probabilities from joint probabilities on coordinate cylinders
  • Apply section integrals to joint laws represented relative to product measures
  • Work out marginals in finite and continuous examples
  • Recognize why singular joint laws need not have a product-measure density

From a Joint Law to a Marginal Law

A joint distribution describes several random coordinates together. Often, however, a question concerns just one coordinate or a block of coordinates. The corresponding distribution is called a marginal law. Its definition is simple: retain the joint law, but view it through a coordinate projection.

This viewpoint works for discrete, continuous, and singular distributions. When a joint law can also be represented by a function integrated against a product measure, Tonelli’s theorem gives a useful section-integral formula for marginal event probabilities. The projection definition is more general; the integral formula requires the additional representation.

Coordinate Projections and Marginal Laws

Let \(S_1,\ldots,S_d\) be measurable spaces, with product sigma-algebra \(\bigotimes_{j=1}^d\mathcal S_j\). Let \(X=(X_1,\ldots,X_d)\) be a random vector taking values in their product, and let \(\mu\) be its joint law. Thus, for a measurable set \(B\) in the product space, \(\mu(B)=\mathbb P(X\in B)\).

For a coordinate \(i\), the projection \(\pi_i\) sends a point of the product space to its \(i\)th coordinate. For a nonempty set of coordinates \(I\subseteq\{1,\ldots,d\}\), the block projection \(\pi_I\) retains exactly the coordinates indexed by \(I\), in their original order. These maps are measurable for the product sigma-algebras.

Definition: The marginal law \(\mu_I\) of the coordinate block \(X_I=(X_i)_{i\in I}\) is the pushforward of the joint law \(\mu\) under \(\pi_I\). Equivalently, for every measurable set \(A\) in the product space of the coordinates in \(I\), $$ \mu_I(A)=\mu\bigl(\pi_I^{-1}(A)\bigr). $$ For a single coordinate \(i\), this is \(\mu_i(A)=\mu(\pi_i^{-1}(A))\).

In particular, for two coordinates \(X\) and \(Y\) with joint law \(\mu\), the marginal probabilities can be read from coordinate cylinders:

$$ \mu_X(A)=\mu(A\times T), \qquad \mu_Y(C)=\mu(S\times C), $$

where \(A\subseteq S\) and \(C\subseteq T\) are measurable. A cylinder restricts one coordinate and places no restriction on the other. This is why marginalization means summing or integrating over all possible values of the coordinates that are not being retained.

A Basic Result About Marginal Laws

The projection definition immediately ensures that a marginal law is itself a probability measure. It also gives a precise formula for probabilities involving a coordinate block.

Theorem: Let \(\mu\) be a probability measure on the product of the measurable spaces \(S_1,\ldots,S_d\), and let \(I\) be a nonempty set of coordinate indices. The pushforward \(\mu_I=\mu\circ\pi_I^{-1}\) is a probability measure. For every measurable set \(A\) in the block space, $$ \mu_I(A)=\mu\bigl(\pi_I^{-1}(A)\bigr). $$
Proof. The projection \(\pi_I\) is measurable, so \(\pi_I^{-1}(A)\) is measurable whenever \(A\) is measurable in the block space. Thus the formula defines a measure: inverse images preserve complements and countable unions, and countable additivity follows from countable additivity of \(\mu\). More explicitly, if the sets \(A_n\) are pairwise disjoint, then their inverse images are pairwise disjoint, and $$ \mu_I\left(\bigcup_{n=1}^{\infty}A_n\right) =\mu\left(\bigcup_{n=1}^{\infty}\pi_I^{-1}(A_n)\right) =\sum_{n=1}^{\infty}\mu\bigl(\pi_I^{-1}(A_n)\bigr) =\sum_{n=1}^{\infty}\mu_I(A_n). $$ The whole block space has inverse image equal to the whole product space. Therefore its marginal probability is \(\mu_I(\text{whole block space})=\mu(\text{whole product space})=1\). Hence \(\mu_I\) is a probability measure, and the stated formula is its definition. \(\square\)

Worked Example: Marginals from a Finite Joint Law

Worked Example: Marginals from a Finite Joint Law

Suppose \(X\) takes values \(0,1\), and \(Y\) takes values \(a,b,c\), with joint probabilities given by the table below.

\(X\backslash Y\)\(a\)\(b\)\(c\)
\(0\)\(0.1\)\(0.2\)\(0.1\)
\(1\)\(0.2\)\(0.1\)\(0.3\)

The entries sum to \(0.1+0.2+0.1+0.2+0.1+0.3=1\), as required for a joint probability law. The marginal probability of \(X=0\) is the probability of the cylinder \(\{0\}\times\{a,b,c\}\), so it is the sum of the first row: $$ \mathbb P(X=0)=0.1+0.2+0.1=0.4. $$ Similarly, \(\mathbb P(X=1)=0.2+0.1+0.3=0.6\). For \(Y\), sum down the columns: $$ \mathbb P(Y=a)=0.1+0.2=0.3,\quad \mathbb P(Y=b)=0.2+0.1=0.3,\quad \mathbb P(Y=c)=0.1+0.3=0.4. $$ Each marginal sums to one. For example, the marginal event \(Y\in\{a,c\}\) has probability \(0.3+0.4=0.7\). The joint law also gives \(\mathbb P(X=1,Y\in\{a,c\})=0.2+0.3=0.5\); this is a joint event, not the marginal probability \(\mathbb P(X=1)=0.6\).

Section Integrals for Marginal Probabilities

In a continuous or mixed setting, summing over the omitted coordinate may be expressed as integration. Here is the relevant setup for two coordinates. Let \((S,\mathcal S,\lambda)\) and \((T,\mathcal T,\kappa)\) be sigma-finite measure spaces. Suppose a joint probability law \(\mu\) on \(S\times T\) has a nonnegative measurable representation \(h\) relative to the product measure \(\lambda\otimes\kappa\): for measurable \(B\subseteq S\times T\), let \(\mu(B)=\int_B h\,d(\lambda\otimes\kappa)\), where \(\int_{S\times T}h\,d(\lambda\otimes\kappa)=1\).

For \(B\subseteq S\times T\) and \(x\in S\), the section \(B_x\) is the set of \(y\in T\) such that \((x,y)\in B\). Applying Tonelli’s theorem to the nonnegative measurable function \(\mathbf 1_B h\) expresses the probability of \(B\) by integrating over these sections. In particular, a marginal event for \(X\) corresponds to \(B=A\times T\).

Theorem: Under the assumptions above, for every measurable \(A\subseteq S\), $$ \mu_X(A) =\int_{A\times T}h(x,y)\,(\lambda\otimes\kappa)(d(x,y)) =\int_A\left(\int_T h(x,y)\,\kappa(dy)\right)\lambda(dx). $$ The inner integral is interpreted as a nonnegative extended-valued integral; the displayed outer integral is finite because \(\mu\) is a probability measure.
Proof. By the coordinate projection formula, \(\mu_X(A)=\mu(A\times T)\). The representation of \(\mu\) therefore gives $$ \mu_X(A)=\int_{S\times T}\mathbf 1_{A\times T}(x,y)h(x,y)\,(\lambda\otimes\kappa)(d(x,y)). $$ The integrand is nonnegative and measurable. Tonelli’s theorem applies because the two reference measures are sigma-finite, and yields $$ \mu_X(A)=\int_S\left(\int_T\mathbf 1_{A\times T}(x,y)h(x,y)\,\kappa(dy)\right)\lambda(dx). $$ For each \(x\), the indicator \(\mathbf 1_{A\times T}(x,y)\) equals \(\mathbf 1_A(x)\) for every \(y\). Thus the inner integral is \(\mathbf 1_A(x)\int_T h(x,y)\,\kappa(dy)\), and the outer integral reduces to the stated integral over \(A\). \(\square\)

The same reasoning applies to a marginal of \(Y\), integrating over \(S\) instead. More generally, for a measurable joint event \(B\), Tonelli gives $$ \mu(B)=\int_S\left(\int_T\mathbf 1_B(x,y)h(x,y)\,\kappa(dy)\right)\lambda(dx). $$ The section \(B_x\) determines which values of \(y\) contribute for each fixed \(x\). This formula evaluates event probabilities from the joint law; it does not require the event to be a rectangle.

Worked Example: A Joint Law on a Triangle

Worked Example: A Joint Law on a Triangle

Let \((X,Y)\) have joint representation \(h(x,y)=2\) on the region \(0<x<y<1\), and \(h(x,y)=0\) elsewhere, relative to two-dimensional Lebesgue measure. The region has area \(1/2\), so the integral of \(h\) is \(2(1/2)=1\). To calculate the marginal probability \(\mathbb P(X\leq 1/2)\), fix \(x\). Its section in the support is \(x<y<1\), which is nonempty for \(0<x<1/2\). Therefore $$ \mathbb P(X\leq 1/2) =\int_0^{1/2}\int_x^1 2\,dy\,dx =\int_0^{1/2}2(1-x)\,dx =\left[2x-x^2\right]_0^{1/2} =\frac34. $$ For comparison, fixing \(y\) gives the section \(0<x<y\). Hence $$ \mathbb P(Y\leq 1/2) =\int_0^{1/2}\int_0^y 2\,dx\,dy =\int_0^{1/2}2y\,dy =\left[y^2\right]_0^{1/2} =\frac14. $$ The different section limits reflect the support condition \(x<y\). The two probabilities are not obtained by integrating over the same triangular slices.

When the Joint Law Has No Product-Measure Density

The section-integral formula just used requires a representation relative to a product measure. It is not part of the definition of a marginal, and it does not apply to every joint law. The projection formula still applies without change.

Worked Example: A Singular Joint Law on the Diagonal

Let \(U\) be uniform on \([0,1]\), and define \(X=U\) and \(Y=U\). The joint law is concentrated on the diagonal \(\{(x,y):x=y\}\). For every measurable \(A\subseteq[0,1]\), the projection definition gives $$ \mathbb P(X\in A)=\mathbb P(U\in A)=\operatorname{Leb}(A), \qquad \mathbb P(Y\in A)=\mathbb P(U\in A)=\operatorname{Leb}(A). $$ Thus both marginals are uniform. Yet the two-dimensional Lebesgue measure of the diagonal is zero, while the joint law assigns it probability one. If the joint law had a nonnegative density with respect to two-dimensional Lebesgue measure, the integral of that density over the diagonal would be zero, because the integral over a null set is zero. This contradicts probability one. The joint law therefore has no such density, even though its marginals are well-defined by projections.

What Marginalization Does—and Does Not—Say

A marginal law records probabilities involving the retained coordinates alone. It does not preserve the full information in the joint law: different joint laws can have the same marginals but different probabilities for joint events. In the finite example, row and column sums discard how probability is paired across cells. More generally, knowing \(\mu_X\) and \(\mu_Y\) does not determine \(\mu\).

A common error is to treat a marginal formula as if it always came from integrating a joint density. The projection definition needs only a joint probability measure and measurable coordinate maps. A section-integral expression additionally needs an appropriate product-measure representation and the hypotheses for Tonelli or Fubini. In particular, a singular joint law may have perfectly ordinary marginal laws while lacking a density with respect to product Lebesgue measure.

For a discrete law, the sums across rows or columns are the direct counterparts of section integrals. For a represented continuous law, the sections describe the allowed values of the omitted coordinates and the integral accumulates their contribution. In both cases, the underlying operation is the same: evaluate the joint law on the inverse image of a marginal event.

Check Your Understanding

Use the projection definition and the section-integral formula to answer these questions.

  1. For a joint law on \(S\times T\), write the inverse image under the first coordinate projection of a measurable set \(A\subseteq S\).
  2. Why is the marginal law of a coordinate a probability measure?
  3. In the triangular example, what section in the \(y\)-coordinate corresponds to a fixed \(x\) in the support?
  4. Which assumptions allow Tonelli’s theorem to express a marginal event probability as a section integral?
  5. Why does a joint law concentrated on the diagonal not have a density with respect to two-dimensional Lebesgue measure?

Previous

Next

Back to Tutorials
Clinical Biostats