Why Marginals Do Not Tell the Whole Story
The marginal distribution of a random variable describes its probabilities without specifying how it is paired with another variable. For a pair \((X,Y)\), the two marginals tell us how \(X\) behaves on its own and how \(Y\) behaves on its own. They do not, in general, tell us which values of \(X\) occur alongside which values of \(Y\).
That missing information is consequential. Two pairs of random variables can have exactly the same marginal distributions but assign different probabilities to joint events such as “both values are high.” They can also have different covariances. The joint distribution contains information about the relationship between the variables that cannot be recovered from the marginals alone.
For a discrete pair, the marginal probabilities are obtained by summing the joint probabilities over the other variable. For a continuous pair with density \(f_{X,Y}\), they are obtained by integrating the joint density over the other variable. Either operation removes information about how the values are paired. The examples below construct different joint laws and verify their shared marginals directly.
Discrete Joint Tables with the Same Marginals
Suppose \(X\) and \(Y\) are binary random variables, each taking values 0 and 1, with \(P(X=1)=0.4\) and \(P(Y=1)=0.3\). Write \(t=P(X=1,Y=1)\). The other three cell probabilities are forced by these marginals and the requirement that the probabilities sum to 1.
All four probabilities must be nonnegative. This requires \(0\leq t\leq 0.3\). Thus there is a range of possible joint tables, even though the two marginals are fixed. Choosing two different values of \(t\) within this range produces two joint distributions with the same marginals.
Worked Example: Two Binary Laws with Fixed Marginals
For the binary marginals above, compare the joint tables obtained with \(t=0.10\) and \(t=0.25\). For each table, verify both marginals, calculate \(P(X=Y)\), and find \(\operatorname{Cov}(X,Y)\).
Method. Fill in the remaining cells from the marginal probabilities, then sum the appropriate cells for each requested probability. For binary variables, \(E[X]=P(X=1)\), \(E[Y]=P(Y=1)\), and \(E[XY]=P(X=1,Y=1)\).
First law, \(t=0.10\). Its four probabilities are \(P(1,1)=0.10\), \(P(1,0)=0.4-0.10=0.30\), \(P(0,1)=0.3-0.10=0.20\), and \(P(0,0)=0.3+0.10=0.40\). They sum to \(0.10+0.30+0.20+0.40=1.00\).
The row sum for \(X=1\) is \(0.30+0.10=0.40\); the column sum for \(Y=1\) is \(0.20+0.10=0.30\). Also, \(P(X=Y)=P(0,0)+P(1,1)=0.40+0.10=0.50\). Since \(E[XY]=0.10\), the covariance is \(0.10-(0.40)(0.30)=-0.02\).
Second law, \(t=0.25\). The cells are \(P(1,1)=0.25\), \(P(1,0)=0.4-0.25=0.15\), \(P(0,1)=0.3-0.25=0.05\), and \(P(0,0)=0.3+0.25=0.55\). These probabilities also sum to \(1.00\).
The row and column sums again give \(P(X=1)=0.40\) and \(P(Y=1)=0.30\). But \(P(X=Y)=0.55+0.25=0.80\), and \(\operatorname{Cov}(X,Y)=0.25-(0.40)(0.30)=0.13\). Thus the marginals agree, while both the probability of equal values and the covariance differ.
Check. Each table has nonnegative entries summing to 1, and its row and column sums reproduce the specified marginals. The common mistake is to assume that fixed marginals determine the joint cells; the different valid values of \(t\) demonstrate otherwise.
A Continuous Family with Uniform Marginals
The same phenomenon occurs for continuous random variables. Consider a joint density on the unit square, equal to zero outside \(0\leq x\leq1\), \(0\leq y\leq1\), and defined inside the square by
The parameter \(\theta\) changes the joint law. Yet, as the next example verifies, every allowed value gives the same uniform marginal distribution for each variable. The restriction on \(\theta\) ensures the density is nonnegative: each factor \(2x-1\) and \(2y-1\) lies between \(-1\) and 1, so their product lies between \(-1\) and 1, making \(f_{X,Y}(x,y)\geq1-|\theta|\geq0\).
Worked Example: Different Densities with Uniform Marginals
Compare the cases \(\theta=0\) and \(\theta=1\). Verify that each defines a joint density with uniform marginals, then find \(P(X<0.5,Y<0.5)\) under each law.
Method. Integrate the joint density over the square to check normalization, integrate over \(y\) to obtain the marginal density of \(X\), and integrate over the requested region to find its probability.
Normalization. For either parameter value, and in fact for any \(\theta\) in the stated range, the integral over the square is
For fixed \(x\), integrating over \(y\) gives \(f_X(x)=\int_0^1 f_{X,Y}(x,y)\,dy=1+\theta(2x-1)(0)=1\), for \(0\leq x\leq1\). By the same calculation, \(f_Y(y)=1\) for \(0\leq y\leq1\). Both marginals are therefore Uniform\((0,1)\) for both parameter values.
Case \(\theta=0\). The joint density is 1 throughout the unit square. The requested event is a square of side length \(0.5\), so its probability is its area: \(0.5(0.5)=0.25\).
Case \(\theta=1\). Integrate the density over that same square. The integral of \(2x-1\) from 0 to 0.5 is \([x^2-x]_0^{0.5}=0.25-0.5=-0.25\); the corresponding integral for \(y\) is also \(-0.25\). Therefore,
The event probabilities, \(0.25\) and \(0.3125\), differ even though both laws have uniform marginals. The common mistake is to integrate only the constant part of the density in the second case; the interaction term contributes \(0.0625\). As a further comparison, direct integration gives \(E[XY]=0.25+\theta/36\), so the covariance is \(\theta/36\), because both means are \(0.5\). It is 0 when \(\theta=0\) and \(1/36\) when \(\theta=1\).
Continuous Laws That Concentrate on Curves
A joint distribution need not have a density over a two-dimensional region. For example, if \(U\sim\operatorname{Uniform}(0,1)\), the pair \((U,U)\) is a valid random pair whose values all lie on the diagonal \(y=x\). The pair \((U,1-U)\) also has valid uniform marginals, but its values lie on the line \(y=1-x\). These are distinct joint laws despite the identical marginal distributions.
Worked Example: Two Uniform Marginals on Different Lines
Let \(U\sim\operatorname{Uniform}(0,1)\). Compare the pairs \((X,Y)=(U,U)\) and \((X,Y)=(U,1-U)\). Verify their marginals, calculate \(P(X<Y)\) in each case, and compare their covariances.
Method. Use the defining transformations to find each marginal and translate the event \(X<Y\) into a condition on \(U\). Use the definition of covariance and the variance of a Uniform\((0,1)\) variable.
Marginals. In both pairs, \(X=U\), so \(X\) is uniform on \((0,1)\). In the first pair, \(Y=U\) is uniform as well. In the second, for \(0\leq y\leq1\), \(P(Y\leq y)=P(1-U\leq y)=P(U\geq1-y)=y\), so \(Y\) is also uniform. Thus both pairs have the same two marginals.
Event probability. Under \((U,U)\), \(X<Y\) becomes \(U<U\), which never occurs, so the probability is 0. Under \((U,1-U)\), the event becomes \(U<1-U\), or \(U<0.5\). Its probability is \(0.5\).
Covariance. For \((U,U)\), \(\operatorname{Cov}(X,Y)=\operatorname{Var}(U)=1/12\). For \((U,1-U)\), \(\operatorname{Cov}(X,Y)=\operatorname{Cov}(U,1-U)=-\operatorname{Var}(U)=-1/12\). The marginals match, but the event probabilities and covariances differ.
Check. Each \(Y\) is uniform, but the two constructions pair its values with \(X\) in different ways. A common mistake is to assume that a continuous pair must have an ordinary density on a two-dimensional region; a random transformation such as these can instead define a joint law concentrated on a curve.
Why the Joint Law Matters
If a calculation concerns only \(X\), its marginal distribution is enough; the same is true for a calculation concerning only \(Y\). A quantity involving both variables generally needs the joint law. For example, the probability that both binary variables equal 1 is \(t\), which varied from \(0.10\) to \(0.25\) while the marginals remained fixed. Likewise, the probability of a region in the continuous examples depended on which joint law was used.
This distinction matters whenever a decision depends on two outcomes occurring together. In a hypothetical setting, an analyst might study whether two separate indicators exceed chosen thresholds at the same time. Knowing each indicator’s marginal distribution does not determine the chance that both thresholds are exceeded, nor does it determine the distribution of their sum. Those questions require information about the joint distribution or additional assumptions that specify it.
Check Your Understanding
For each problem, distinguish the marginal information from the joint information it does not determine.
- Binary \(X,Y\) have \(P(X=1)=0.6\) and \(P(Y=1)=0.5\). Compare valid joint laws with \(P(X=1,Y=1)=0.2\) and \(0.4\). For each, give \(P(X=Y)\) and \(\operatorname{Cov}(X,Y)\). (Answer: For \(t=0.2\), \(P(X=Y)=0.3\) and covariance \(=-0.1\); for \(t=0.4\), \(P(X=Y)=0.7\) and covariance \(=0.1\).)
- Under the continuous density \(1+\theta(2x-1)(2y-1)\) on the unit square, take \(\theta=-0.5\). Find the probability that both variables are below \(0.5\), and state the marginals. (Answer: \(7/32\); both marginals are Uniform\((0,1)\).)
- Let \(U\sim\operatorname{Uniform}(0,1)\) and set \(X=U,\ Y=1-U\). Find \(P(X<Y)\) and \(P(X+Y>1)\). (Answer: \(P(X<Y)=0.5\) and \(P(X+Y>1)=0\).)
- Binary \(X,Y\) have \(P(X=1)=0.35\), \(P(Y=1)=0.20\), and \(P(X=1,Y=1)=0.08\). Construct the four joint probabilities and find \(P(X=Y)\). (Answer: \(P(1,1)=0.08,\ P(1,0)=0.27,\ P(0,1)=0.12,\ P(0,0)=0.53\); \(P(X=Y)=0.61\).)
- For the continuous density family above with \(\theta=1\), find both \(P(X<0.5,Y<0.5)\) and \(\operatorname{Cov}(X,Y)\). (Answer: \(P(X<0.5,Y<0.5)=5/16\) and \(\operatorname{Cov}(X,Y)=1/36\).)