Covariance Cancellation in Dependent Pairs

When Covariance Cancels

Covariance summarizes whether two random variables tend to be above or below their respective means together. A positive covariance means that positive centered products contribute more, on balance, than negative ones; a negative covariance means the reverse. But the contributions can also cancel exactly. In that case the covariance is zero even though the variables are dependent.

This distinction matters in exam problems: independence implies zero covariance when the relevant expectations exist, but zero covariance alone does not establish independence. A finite joint distribution makes the distinction visible because every contribution can be calculated and checked.

Definition: For random variables $X$ and $Y$ with finite expectations of the required products, their covariance is $$ \operatorname{Cov}(X,Y)=E[(X-E[X])(Y-E[Y])]. $$ For a finite joint distribution with possible pairs $(x_j,y_j)$ and probabilities $p_j$, this becomes $$ \operatorname{Cov}(X,Y)=\sum_j (x_j-E[X])(y_j-E[Y])p_j. $$ The pair is dependent if its joint distribution does not equal the product of its marginal distributions. In particular, dependence can be shown by finding values $x,y$ for which $P(X=x,Y=y)\ne P(X=x)P(Y=y)$.

Each outcome contributes its probability times a product of deviations from the means. If $x_j$ and $y_j$ are both above their means, or both below, the centered product is positive. If one is above its mean and the other is below, the product is negative. A zero centered deviation produces a zero contribution. Zero covariance says only that the sum of these weighted products is zero; it does not say that every product is zero or that the joint probabilities factor.

Why the Product-Moment Identity Works

The centered-product definition is useful for seeing cancellation directly. For arithmetic, it is often easier to use the equivalent product-moment identity. Its derivation follows by expanding the product and using the linearity of expectation.

Expand the centered product.
$(X-E[X])(Y-E[Y])=XY-XE[Y]-YE[X]+E[X]E[Y]$.

Take expectations.
$E[XY]-E[X]E[Y]-E[Y]E[X]+E[X]E[Y]$.

Combine the constant terms.
The result is $E[XY]-E[X]E[Y]$, because the three mean-product terms have net value $-E[X]E[Y]$.

\operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y].

For finite distributions, $E[XY]$ is the sum of $xy$ times the joint probability of each pair. The identity and the centered-product formula must give the same result. Calculating both is a useful check, especially when the covariance is zero: a small arithmetic error in $E[XY]$ can change the conclusion.

Worked Example: Cancellation in Three Equally Likely Outcomes

A hypothetical device records two scores $(X,Y)$. Its possible score pairs are $(-1,1)$, $(0,-2)$, and $(1,1)$, each with probability $1/3$. Find the covariance and determine whether $X$ and $Y$ are independent.

Method. First calculate the marginal means and $E[XY]$, then use the product-moment identity. Finally, test dependence using one joint probability.

The mean of $X$ is $E[X]=(-1+0+1)/3=0$. The mean of $Y$ is $E[Y]=(1-2+1)/3=0$. For the product, the three values of $XY$ are $-1$, $0$, and $1$, so $E[XY]=(-1+0+1)/3=0$. Therefore, $\operatorname{Cov}(X,Y)=0-(0)(0)=0$.

The centered-product calculation shows the cancellation explicitly. Since both means are zero, the centered products are $(-1)(1)=-1$, $(0)(-2)=0$, and $(1)(1)=1$. Their probability-weighted sum is $(-1)(1/3)+0(1/3)+(1)(1/3)=0$. The negative contribution from the first pair cancels the positive contribution from the third.

For dependence, $P(X=-1,Y=1)=1/3$, while $P(X=-1)P(Y=1)=(1/3)(2/3)=2/9$. These are unequal, so $X$ and $Y$ are dependent. Check: the covariance is zero, but this joint probability disproves independence. The method avoids treating zero covariance as proof of independence.

A Joint Table with Marginals That Do Not Reveal Independence

A table can make dependence less obvious than a deterministic rule such as $Y=g(X)$. The next example has several possible values for each variable. Its marginals are uniform, but a joint cell differs from the probability that independence would require.

Worked Example: A Three-by-Three Distribution

A hypothetical inspection system reports $X,Y\in\{0,1,2\}$. The joint probabilities are the following entries divided by 36. Compute the covariance and decide whether the variables are independent.

$X\backslash Y$	0	1	2	Row total
0	5/36	2/36	5/36	12/36
1	2/36	8/36	2/36	12/36
2	5/36	2/36	5/36	12/36
Column total	12/36	12/36	12/36	36/36

The table entries sum to $36/36=1$, so they form a probability distribution. Each marginal value of $X$ has probability $12/36=1/3$, and the same is true for each value of $Y$. Thus $E[X]=0(1/3)+1(1/3)+2(1/3)=1$ and $E[Y]=1$.

To calculate $E[XY]$, cells with $x=0$ or $y=0$ contribute zero. The other contributions, shown as $xy$ times the cell probability, are $1\cdot1\cdot(8/36)=8/36$, $1\cdot2\cdot(2/36)=4/36$, $2\cdot1\cdot(2/36)=4/36$, and $2\cdot2\cdot(5/36)=20/36$. Their sum is $36/36=1$. Consequently, $\operatorname{Cov}(X,Y)=1-(1)(1)=0$.

The centered-product formula confirms the result. In the row $X=0$, $X-E[X]=-1$, and the three $Y$-centered values are $-1,0,1$. The weighted contributions sum to $((-1)(-1))(5/36)+((-1)(0))(2/36)+((-1)(1))(5/36)=0$. In the row $X=1$, the $X$-centered value is zero, so all contributions are zero. In the row $X=2$, the $X$-centered value is $1$, and the weighted contributions sum to $(1)(-1)(5/36)+(1)(0)(2/36)+(1)(1)(5/36)=0$. The total is zero.

For independence, the table gives $P(X=1,Y=1)=8/36=2/9$. If the variables were independent, that probability would be $P(X=1)P(Y=1)=(1/3)(1/3)=1/9$. Since $2/9\ne1/9$, the variables are dependent. Check: the row and column totals each give the stated marginal probabilities, and the joint entries sum to one. The method avoids assuming that uniform marginals force independence.

An Unequal-Probability Example

Cancellation does not require equally likely outcomes. The means depend on the probabilities as well as the possible values, so both must be included when calculating centered products.

Worked Example: Unequal Probabilities and Exact Cancellation

For a hypothetical classification rule, the possible pairs $(X,Y)$ are $(0,1)$ with probability $1/2$, $(1,0)$ with probability $1/4$, and $(3,1)$ with probability $1/4$. Find the covariance and determine whether the variables are independent.

First, the probabilities add to $1/2+1/4+1/4=1$. The marginal means are $E[X]=0(1/2)+1(1/4)+3(1/4)=1$ and $E[Y]=1(1/2)+0(1/4)+1(1/4)=3/4$. The product values $XY$ are $0,0,3$, respectively, so $E[XY]=0(1/2)+0(1/4)+3(1/4)=3/4$. Therefore, $\operatorname{Cov}(X,Y)=3/4-(1)(3/4)=0$.

For a direct check, the centered products at the three outcomes are $(0-1)(1-3/4)=-1/4$, $(1-1)(0-3/4)=0$, and $(3-1)(1-3/4)=1/2$. After weighting by their probabilities, the sum is $(-1/4)(1/2)+0(1/4)+(1/2)(1/4)=-1/8+1/8=0$.

The first and third outcomes both have $Y=1$, so $P(Y=1)=3/4$. But $P(X=0,Y=1)=1/2$, whereas $P(X=0)P(Y=1)=(1/2)(3/4)=3/8$. Since $1/2\ne3/8$, the variables are dependent. Check: the centered contributions $-1/8$ and $1/8$ cancel, matching the product-moment calculation. The method avoids omitting an outcome's probability when summing centered products.

Common Mistakes and a Reliable Check

The main conceptual trap is to reverse the implication between independence and covariance. Independence gives zero covariance when the expectations exist, but a zero covariance does not establish independence. A separate check of the joint distribution is needed.

There is also a frequent arithmetic trap in $E[XY]$: each product must be multiplied by the probability of its own joint outcome. In the three-by-three example, the $X=2,Y=2$ cell contributes $2\cdot2\cdot(5/36)=20/36$, not $4/36$. The latter is the product $xy$ without its probability weight; using it would give an incorrect expectation and covariance.

A sound solution therefore keeps the two questions separate: first, does the weighted sum of centered products equal zero? Second, does the joint distribution factor into the marginals? A probability mismatch in even one cell answers the second question, regardless of the covariance.

Key takeaway: Zero covariance means that positive and negative probability-weighted centered products balance. It does not mean that the variables are independent. Verify covariance with the means and joint probabilities, then test independence separately using the joint distribution.

Check Your Understanding

For each finite distribution, calculate or verify the requested quantity and distinguish zero covariance from independence.

The equally likely pairs are $(-1,1)$, $(0,-2)$, and $(1,1)$. Find the covariance and determine whether the variables are independent. (Answer: $\operatorname{Cov}(X,Y)=0$; dependent.)
A joint table has entries, in row order $X=0,1,2$ and column order $Y=0,1,2$, equal to $(5,2,5)$, $(2,8,2)$, and $(5,2,5)$, all divided by 36. Find the covariance and compare $P(X=1,Y=1)$ with $P(X=1)P(Y=1)$. (Answer: $\operatorname{Cov}(X,Y)=0$; $P(X=1,Y=1)=2/9$, while $P(X=1)P(Y=1)=1/9$, so the variables are dependent.)
The pairs $(0,0),(0,2),(2,0),(2,2)$ have probabilities $1/5,3/10,3/10,1/5$, respectively. Find the covariance. (Answer: $E[X]=1$, $E[Y]=1$, $E[XY]=4/5$, so $\operatorname{Cov}(X,Y)=-1/5$.)
The pairs $(0,1),(1,0),(3,1)$ have probabilities $1/2,1/4,1/4$, respectively. Verify the covariance and test independence. (Answer: $\operatorname{Cov}(X,Y)=0$; dependent, since $P(X=0,Y=1)=1/2\ne3/8=P(X=0)P(Y=1)$.)
The equally likely pairs are $(-2,3),(0,-1),(2,3)$. Find $E[X]$, $E[Y]$, $E[XY]$, and the covariance; then state whether the variables are independent. (Answer: $E[X]=0$, $E[Y]=5/3$, $E[XY]=0$, $\operatorname{Cov}(X,Y)=0$; dependent.)

What You'll Learn