Joint Laws on Coordinate Spaces
A finite-dimensional distribution records the joint law of finitely many coordinates. It retains their dependence: knowing the individual coordinate laws alone generally does not determine the joint law. Marginalization asks a more limited question: what is the law of a selected subcollection of coordinates, without recording the others?
Let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space, and let \(T\) be an index set. For each \(t\in T\), suppose \(X_t:\Omega\to S_t\) is measurable, where \((S_t,\mathcal{S}_t)\) is a measurable space. Choose distinct indices \(t_1,\ldots,t_k\), with \(k\ge1\), and write \(S_{t_1}\times\cdots\times S_{t_k}\) for the coordinate space, equipped with the product sigma-algebra \(\mathcal{S}_{t_1}\otimes\cdots\otimes\mathcal{S}_{t_k}\). The coordinate vector is \(X_I=(X_{t_1},\ldots,X_{t_k})\), where \(I=\{t_1,\ldots,t_k\}\); its law is the probability measure \(\mu_I\) defined by \(\mu_I(C)=\mathbb{P}(X_I\in C)\) for measurable \(C\) in that product space. This law is the finite-dimensional distribution for the chosen coordinates.
A pushforward records the probability that the retained coordinates fall in a specified set. In particular, if \(\mu_I\) is the law of \(X_I\), then \(\mu_J\) is the law of the subvector consisting of the coordinates in \(J\). The order of coordinates must be fixed when writing a vector or a table; the projection keeps the selected coordinates in that order. Marginalization sums or integrates over the omitted values. It is not conditioning: a marginal is an unconditional law, whereas conditioning restricts or reweights probabilities using additional information.
Projection and Marginalization
Proof. The projection \(\pi_J\) is measurable: the inverse image of a measurable rectangle in the \(J\)-coordinate space is a measurable rectangle in the full product space, and such rectangles generate the product sigma-algebras. Therefore inverse images of measurable sets are measurable. Define \(\mu_J(B)=\mu(\pi_J^{-1}(B))\). The inverse image of the empty set is empty, so \(\mu_J(\varnothing)=0\). For pairwise disjoint measurable sets \(B_n\), their inverse images are pairwise disjoint and \(\pi_J^{-1}(\bigcup_n B_n)=\bigcup_n\pi_J^{-1}(B_n)\). Countable additivity of \(\mu\) therefore gives \(\mu_J(\bigcup_n B_n)=\sum_n\mu_J(B_n)\). Finally, \(\pi_J^{-1}(S_J)\) is the whole product space, so \(\mu_J(S_J)=\mu(S_1\times\cdots\times S_k)=1\). If \(\mu\) is the law of \(X\), then \(\mu_J(B)=\mathbb{P}(X\in\pi_J^{-1}(B))=\mathbb{P}(\pi_J(X)\in B)\), which is exactly the law of the selected subvector. For \(J=\varnothing\), the projection is constant and the same formula gives mass one to the singleton. \(\square\)
For discrete coordinates, take a set of possible values and sum the joint probabilities over all values of omitted coordinates. For coordinates with a joint density relative to Lebesgue measure, the corresponding operation is integration. The following density statement makes that operation precise.
Proof. Since \(f\ge0\) is measurable, Tonelli's theorem applies on the product Lebesgue space and shows that \(g\) is measurable and that \(\int_{\mathbb{R}^p}g\,dm=\int_{\mathbb{R}^{p+q}}f\,dm=1\). For any Lebesgue-measurable set \(A\subseteq\mathbb{R}^p\), the probability that the first \(p\) coordinates lie in \(A\) is \(\mu(A\times\mathbb{R}^q)=\int_{A\times\mathbb{R}^q}f\,dm\). Applying Tonelli's theorem to the nonnegative function \(\chi_A(x)f(x,y)\) gives \(\mu(A\times\mathbb{R}^q)=\int_A\left(\int_{\mathbb{R}^q}f(x,y)\,dm(y)\right)dm(x)=\int_Ag(x)\,dm(x)\). This is the defining property of a marginal density. The integral-one calculation shows that it is a probability density. \(\square\)
Worked Marginal Calculations
Worked Example: A Dependent Bernoulli Pair
Let \((X_1,X_2)\) take values in \(\{0,1\}^2\) with the following joint probabilities. Every entry is a probability of the indicated ordered pair.
| \(X_1\backslash X_2\) | \(0\) | \(1\) | Row sum |
|---|---|---|---|
| \(0\) | \(\frac12\) | \(\frac16\) | \(\frac23\) |
| \(1\) | \(\frac16\) | \(\frac16\) | \(\frac13\) |
| Column sum | \(\frac23\) | \(\frac13\) | \(1\) |
The joint probabilities add to \(\frac12+\frac16+\frac16+\frac16=\frac{3+1+1+1}{6}=1\), so the table defines a probability law. To find the marginal of \(X_1\), sum across each row: \(\mathbb{P}(X_1=0)=\frac12+\frac16=\frac23\), and \(\mathbb{P}(X_1=1)=\frac16+\frac16=\frac13\). Summing down columns gives \(\mathbb{P}(X_2=0)=\frac12+\frac16=\frac23\) and \(\mathbb{P}(X_2=1)=\frac16+\frac16=\frac13\).
Check: each marginal totals \(\frac23+\frac13=1\). These coordinates are not independent: independence would require \(\mathbb{P}(X_1=1,X_2=1)=\mathbb{P}(X_1=1)\mathbb{P}(X_2=1)=\frac13\cdot\frac13=\frac19\), but the joint table gives \(\frac16\). Thus, the marginals do not recover the joint law.
Worked Example: Marginal of a Bivariate Gaussian
For \(-1<\rho<1\), consider the centered bivariate Gaussian density on \((\mathbb{R}^2,\mathcal{L},m)\): \(f(x,y)=\dfrac{1}{2\pi\sqrt{1-\rho^2}}\exp\!\left(-\dfrac{x^2-2\rho xy+y^2}{2(1-\rho^2)}\right)\). To compute the first-coordinate marginal, complete the square: \(x^2-2\rho xy+y^2=(y-\rho x)^2+(1-\rho^2)x^2\). The density therefore factors, for each fixed \(x\), as \(f(x,y)=\dfrac{1}{\sqrt{2\pi}}\exp(-x^2/2)\cdot\dfrac{1}{\sqrt{2\pi(1-\rho^2)}}\exp\!\left(-\dfrac{(y-\rho x)^2}{2(1-\rho^2)}\right)\).
Integrate over \(y\). The second factor is a normal density in \(y\) with mean \(\rho x\) and variance \(1-\rho^2>0\), so its integral over \(\mathbb{R}\) is one. Equivalently, the substitution \(z=(y-\rho x)/\sqrt{1-\rho^2}\) reduces its integral to \(\frac{1}{\sqrt{2\pi}}\int_{\mathbb{R}}e^{-z^2/2}\,dz=1\). Thus \(g(x)=\int_{\mathbb{R}}f(x,y)\,dy=\frac{1}{\sqrt{2\pi}}e^{-x^2/2}\), the standard normal density. This integration is justified by the density marginal theorem, whose nonnegativity hypothesis holds for \(f\).
Check: \(\int_{\mathbb{R}}g(x)\,dx=1\), by the standard normal integral. The same calculation integrating in \(x\) gives the standard normal marginal for \(Y\), since the joint density is symmetric in \(x\) and \(y\). The parameter \(\rho\) affects the joint law but disappears from either one-coordinate marginal.
Worked Example: Marginals of a Discrete Pair
Let \(X\) take values \(0,1,2\) and \(Y\) take values \(a,b\), with joint law given below.
| \(X\backslash Y\) | \(a\) | \(b\) | Row sum |
|---|---|---|---|
| \(0\) | \(\frac16\) | \(\frac13\) | \(\frac12\) |
| \(1\) | \(\frac14\) | \(\frac1{12}\) | \(\frac13\) |
| \(2\) | \(\frac1{12}\) | \(\frac1{12}\) | \(\frac16\) |
| Column sum | \(\frac12\) | \(\frac12\) | \(1\) |
The row sums are obtained by adding the probabilities for each fixed value of \(X\): for \(X=0\), \(\frac16+\frac13=\frac12\); for \(X=1\), \(\frac14+\frac1{12}=\frac3{12}+\frac1{12}=\frac13\); and for \(X=2\), \(\frac1{12}+\frac1{12}=\frac16\). The column sums give \(\mathbb{P}(Y=a)=\frac16+\frac14+\frac1{12}=\frac2{12}+\frac3{12}+\frac1{12}=\frac12\), and \(\mathbb{P}(Y=b)=\frac13+\frac1{12}+\frac1{12}=\frac4{12}+\frac1{12}+\frac1{12}=\frac12\).
Check: the total from the \(X\)-marginal is \(\frac12+\frac13+\frac16=\frac3{6}+\frac2{6}+\frac1{6}=1\), and the total from the \(Y\)-marginal is \(\frac12+\frac12=1\). Both agree with the sum of all six joint entries.
A Common Marginalization Error
A frequent mistake is to read one joint-table entry as a marginal probability. In the Bernoulli example, the entry \(\mathbb{P}(X_1=1,X_2=1)=\frac16\) is not \(\mathbb{P}(X_1=1)\). The latter event includes both possibilities \(X_2=0\) and \(X_2=1\), so its probability is \(\frac16+\frac16=\frac13\). The correction is to sum over every possible value of the omitted coordinate. For a density, the parallel correction is to integrate over the omitted coordinate, not to substitute a fixed value such as \(y=0\).
Marginalization is useful whenever a question concerns only selected coordinates of a joint model. It allows calculations about a subvector without changing the underlying joint law. The probability check is direct: the resulting marginal must assign total mass one. A discrete marginal can be checked by summing its entries; a density marginal can be checked by integrating it, with Tonelli's theorem available for nonnegative joint densities.
Check Your Understanding
For each question, identify which coordinates are omitted and use a sum or integral over their possible values.
- A joint law on \(\{0,1\}^2\) assigns probabilities \(\frac14,\frac14,\frac18,\frac38\) to \((0,0),(0,1),(1,0),(1,1)\), respectively. Find both marginals and check their totals.
- Explain why the marginal law of a selected subvector is the pushforward under a coordinate projection.
- For a nonnegative joint density \(f(x,y)\) on \(\mathbb{R}^2\) with integral one, state the formula for the first-coordinate marginal density and the theorem that justifies the iterated integral.
- In the Bernoulli example, compare \(\mathbb{P}(X_1=1,X_2=1)\) with \(\mathbb{P}(X_1=1)\). Why are they different?