Comparing Dependence with Fixed Marginals

Introduction

Knowing the distribution of each variable separately does not tell us how their outcomes occur together. For example, two populations can have the same possible values and the same proportions at each value, yet one joint distribution may pair large values of one variable with large values of the other, while another pairs large values with small values. The marginal distributions are unchanged, but covariance and correlation can differ substantially.

This comparison is useful whenever the individual behavior of two quantities is understood but their dependence is uncertain. The key is to distinguish what the marginals determine from what only the joint distribution determines.

Definition: The marginal distribution of \(X\) gives the probabilities of its values without specifying \(Y\); the marginal distribution of \(Y\) is defined similarly. A joint distribution specifies probabilities for pairs \((X,Y)\) and must reproduce both marginal distributions when probabilities are summed over the other variable. A joint distribution having specified marginals is one possible coupling of those marginals.

What the Marginals Determine—and What They Do Not

For any joint distribution with finite second moments, the marginal distributions determine \(E[X]\), \(E[Y]\), \(\operatorname{Var}(X)\), and \(\operatorname{Var}(Y)\). They do not, in general, determine \(E[XY]\). Since \(\operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y]\), different joint distributions with the same marginals can therefore have different covariance.

\operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y], \qquad \operatorname{Corr}(X,Y)= \frac{\operatorname{Cov}(X,Y)} {\sqrt{\operatorname{Var}(X)\operatorname{Var}(Y)}}.

When the marginals are fixed and both variances are positive, the denominator in the correlation formula is fixed. Thus changing the joint distribution changes correlation in step with covariance. If either variable has variance zero, its correlation with the other variable is undefined; the formula would require division by zero.

Worked Example: Two Tables With the Same Marginals

Let \(X\) take values 0 and 2, and let \(Y\) take values 1 and 3. Consider the following joint table, with rows corresponding to \(X=0,2\) and columns to \(Y=1,3\):

\begin{array}{c|cc} & Y=1 & Y=3\\ \hline X=0 & 0.1 & 0.4\\ X=2 & 0.4 & 0.1 \end{array}

Method: calculate the marginals first, then use the joint probabilities for \(E[XY]\). The row sums give \(P(X=0)=0.1+0.4=0.5\) and \(P(X=2)=0.4+0.1=0.5\). The column sums give \(P(Y=1)=0.1+0.4=0.5\) and \(P(Y=3)=0.4+0.1=0.5\).

The means are \(E[X]=0(0.5)+2(0.5)=1\) and \(E[Y]=1(0.5)+3(0.5)=2\). The second moments are \(E[X^2]=0^2(0.5)+2^2(0.5)=2\) and \(E[Y^2]=1^2(0.5)+3^2(0.5)=5\). Therefore, \(\operatorname{Var}(X)=2-1^2=1\) and \(\operatorname{Var}(Y)=5-2^2=1\).

Now calculate the product expectation from all four cells: \(E[XY]=0(1)(0.1)+0(3)(0.4)+2(1)(0.4)+2(3)(0.1)=0+0+0.8+0.6=1.4\). Thus \(\operatorname{Cov}(X,Y)=1.4-(1)(2)=-0.6\), and \(\operatorname{Corr}(X,Y)=-0.6/\sqrt{1\cdot1}=-0.6\).

For comparison, a joint table with 0.25 in every cell has exactly the same marginals. Under that table, \(E[XY]=0+0+2(1)(0.25)+2(3)(0.25)=0.5+1.5=2\). Its covariance is \(2-(1)(2)=0\), and its correlation is 0. The second table represents independence; the first does not.

Check: both tables have row and column sums 0.5, so their marginal distributions agree. The covariance changes from \(-0.6\) to 0 solely because the pairing probabilities change. The common mistake is to calculate \(E[XY]\) from the marginals alone; the first table shows why the joint cells are necessary.

Ordered Pairings and the Covariance Extremes

For fixed discrete marginals, pairing low values of \(X\) with low values of \(Y\), and high values with high values, tends to make \(E[XY]\) large. Pairing low values of one variable with high values of the other tends to make it small. The following swap argument explains this result.

Theorem: For finite marginal distributions, a pairing that orders the values of \(X\) and \(Y\) in the same direction maximizes \(E[XY]\), and hence covariance, among all couplings of those marginals. A pairing that orders them in opposite directions minimizes \(E[XY]\), and hence covariance. If probabilities do not match in equal-sized groups, probability mass can be split as needed to make the pairings.

To justify the ordering claim, compare two equal amounts of probability mass. Suppose \(x_1\leq x_2\) and \(y_1\leq y_2\). Pairing in the same order contributes \(x_1y_1+x_2y_2\), while the crossed pairing contributes \(x_1y_2+x_2y_1\). Subtracting gives

(x_1y_1+x_2y_2)-(x_1y_2+x_2y_1) =(x_2-x_1)(y_2-y_1)\geq 0.

Thus replacing a crossed pair by a same-order pair cannot reduce the total product expectation. Repeating such swaps gives a same-order pairing with at least as large an \(E[XY]\). Reversing the comparison shows that opposite-order pairings minimize it. Because \(E[X]\) and \(E[Y]\) are fixed by the marginals, the same ordering also maximizes or minimizes covariance.

The correlation extremes need not be \(1\) and \(-1\). Correlation equals \(1\) or \(-1\) only when one variable is an exact positive or negative affine function of the other with probability 1. Ordered pairings maximize or minimize correlation for fixed nonzero marginal variances, but the resulting values may lie strictly between \(-1\) and \(1\).

Worked Example: Same Marginals, Perfect and Opposite Linear Pairings

Let \(X\) have probabilities \(1/3\) at each of \(0,1,2\), and let \(Y\) have probabilities \(1/3\) at each of \(10,20,30\). Compare the same-order pairing, the opposite-order pairing, and the independent coupling.

Same-order pairing. The joint outcomes are \((0,10),(1,20),(2,30)\), each with probability \(1/3\). The marginal means are \(E[X]=(0+1+2)/3=1\) and \(E[Y]=(10+20+30)/3=20\). The product expectation is \(E[XY]=(0+20+60)/3=80/3\). Therefore \(\operatorname{Cov}(X,Y)=80/3-(1)(20)=20/3\).

Opposite-order pairing. The joint outcomes are \((0,30),(1,20),(2,10)\), each with probability \(1/3\). The marginals and their means have not changed. Now \(E[XY]=(0+20+20)/3=40/3\), so \(\operatorname{Cov}(X,Y)=40/3-20=-20/3\).

To calculate the correlations, first obtain the marginal variances, which are the same for both pairings: \(E[X^2]=(0+1+4)/3=5/3\), so \(\operatorname{Var}(X)=5/3-1^2=2/3\). Also \(E[Y^2]=(100+400+900)/3=1400/3\), so \(\operatorname{Var}(Y)=1400/3-20^2=200/3\). The denominator for correlation is \(\sqrt{(2/3)(200/3)}=\sqrt{400/9}=20/3\). Thus the same-order correlation is \((20/3)/(20/3)=1\), and the opposite-order correlation is \((-20/3)/(20/3)=-1\).

Independent coupling. Assign probability \(1/9\) to each of the nine possible pairs. Independence gives \(E[XY]=E[X]E[Y]=20\), so covariance and correlation are both 0. All three couplings have the same marginals.

Check: the same-order outcomes satisfy \(Y=10X+10\), while the opposite-order outcomes satisfy \(Y=30-10X\). These exact affine relationships account for correlations \(1\) and \(-1\). The independent table has the same marginal means and variances but a different product expectation. The common mistake is to infer correlation from the marginal distributions without examining how outcomes are paired.

When the Extremes Are Not Plus or Minus One

A coupling can give the largest or smallest covariance possible for its marginals without producing a perfect linear relationship. This happens when the two sets of values and their probabilities do not fit an affine rule. The marginals still fix the correlation denominator, so calculating covariance and dividing by that denominator gives the correlation for each pairing.

Worked Example: Dependence Strength With Unequal Marginal Shapes

Suppose a hypothetical study records two measurements, \(X\) and \(Y\). Let \(X\) equal 0, 1, or 3 with probabilities \(0.50, 0.25, 0.25\), respectively, and let \(Y\) equal 0 or 2 with probabilities \(0.50,0.50\). Compare two joint distributions:

\begin{array}{c|cc} \text{Pairing A} & Y=0 & Y=2\\ \hline X=0 & 0.50 & 0\\ X=1 & 0 & 0.25\\ X=3 & 0 & 0.25 \end{array} \qquad \begin{array}{c|cc} \text{Pairing B} & Y=0 & Y=2\\ \hline X=0 & 0 & 0.50\\ X=1 & 0.25 & 0\\ X=3 & 0.25 & 0 \end{array}

The row sums in both tables are \(0.50,0.25,0.25\), and the column sums are \(0.50,0.50\). Thus both tables have the stated marginals. The means are \(E[X]=0(0.50)+1(0.25)+3(0.25)=1\) and \(E[Y]=0(0.50)+2(0.50)=1\).

For either table, \(E[X^2]=0^2(0.50)+1^2(0.25)+3^2(0.25)=0.25+2.25=2.50\), giving \(\operatorname{Var}(X)=2.50-1^2=1.50\). Also \(E[Y^2]=0^2(0.50)+2^2(0.50)=2\), so \(\operatorname{Var}(Y)=2-1^2=1\).

In Pairing A, \(E[XY]=1(2)(0.25)+3(2)(0.25)=0.50+1.50=2\). Hence its covariance is \(2-(1)(1)=1\), and its correlation is \(1/\sqrt{1.50\cdot1}=1/\sqrt{1.5}\), approximately \(0.8165\).

In Pairing B, every outcome with positive probability has either \(X=0\) or \(Y=0\), so each product \(XY\) is zero. Thus \(E[XY]=0\), covariance is \(0-(1)(1)=-1\), and correlation is \(-1/\sqrt{1.5}\), approximately \(-0.8165\). The values are rounded to four decimal places.

Check: the marginal means and variances are identical in the two pairings, but covariance changes from 1 to \(-1\). Neither correlation is \(1\) or \(-1\): the values of \(X\) and \(Y\) do not lie on a single exact affine line in either pairing. The common mistake is to assume that a maximizing or minimizing pairing must always have perfect correlation.

A Reliable Comparison Procedure

When a problem asks how dependence changes while the marginals remain fixed, keep the marginal calculations separate from the joint calculation. A short, systematic procedure reduces errors.

Verify the marginals.
Sum each row and column of every joint table. Confirm that corresponding marginal probabilities agree across the tables and each full table sums to 1.

Calculate marginal moments once.
Find \(E[X]\), \(E[Y]\), \(\operatorname{Var}(X)\), and \(\operatorname{Var}(Y)\) from the marginal distributions. These quantities are unchanged when only the coupling changes.

Use each joint distribution for the product moment.
Calculate \(E[XY]=\sum_x\sum_y xyP(X=x,Y=y)\) using that table's cells. Do not substitute a product of means unless independence has been established.

Compare covariance and correlation.
Subtract the fixed value \(E[X]E[Y]\) from each product expectation. If both marginal variances are positive, divide each covariance by the same fixed standard-deviation product to obtain correlation.

Common Mistakes and Interpretation

Using only the marginals to calculate \(E[XY]\). Marginals determine the means and variances, but generally do not determine the product expectation. Use the joint probabilities.
Changing a marginal while intending to change only dependence. Check row and column sums before comparing covariance. Otherwise the comparison changes both the margins and the pairing.
Assuming independence from zero covariance. Zero covariance does not generally imply independence. The independent coupling in an example has zero covariance, but other dependent couplings can also have zero covariance.
Expecting extreme correlation in every ordered pairing. The ordered coupling gives a covariance extreme, but correlation reaches \(1\) or \(-1\) only for an exact affine relationship.
Forgetting the correlation denominator. With fixed nondegenerate marginals, the denominator is constant across couplings. If a marginal is degenerate, correlation is undefined rather than zero.

The practical lesson is that marginal risk descriptions alone cannot characterize joint behavior. In actuarial and statistical work, conclusions about combined outcomes can depend on how the underlying quantities are associated. Comparing couplings makes that dependence visible without changing either variable's individual distribution. This treatment is aligned with the SOA VEE Mathematical Statistics syllabus.

Check Your Understanding

For each problem, distinguish the marginal information from the joint pairing and show the calculation needed for the requested measure.

\(X\) takes values \(-1\) and 2 with probability \(0.5\) each, and \(Y\) takes values 0 and 4 with probability \(0.5\) each. In one coupling the outcomes are \((-1,0)\) and \((2,4)\), each with probability \(0.5\). Find covariance and correlation. (Answer: covariance \(=3\), correlation \(=1\).)
Suppose \(X\) and \(Y\) have positive variances and are independent, with \(E[X]=3\) and \(E[Y]=-2\). Find \(E[XY]\), covariance, and correlation. (Answer: \(E[XY]=-6\), covariance \(=0\), correlation \(=0\).)
A joint table has rows \(X=0,1\), columns \(Y=2,5\), and rows \((0.3,0.2)\) and \((0.2,0.3)\). Find the marginal distributions, covariance, and correlation. (Answer: \(P(X=0)=P(X=1)=0.5\), \(P(Y=2)=P(Y=5)=0.5\), covariance \(=0.15\), correlation \(=0.2\).)
\(X\) has values 0 and 4 with probability \(0.5\) each; \(Y\) has values 1 and 7 with probability \(0.5\) each. Compare covariance and correlation for the same-order outcomes \((0,1),(4,7)\) and opposite-order outcomes \((0,7),(4,1)\), each pair equally likely. (Answer: same order: covariance \(=6\), correlation \(=1\); opposite order: covariance \(=-6\), correlation \(=-1\).)
For fixed nondegenerate finite marginals, why does maximizing \(E[XY]\) also maximize covariance and correlation? State the role of the marginal distributions. (Answer: \(E[X]\), \(E[Y]\), and both variances are fixed by the marginals; covariance subtracts the fixed product \(E[X]E[Y]\), and correlation divides covariance by a fixed positive standard-deviation product.)

What You'll Learn