Independence, Uncorrelatedness, and Counterexamples

Two Different Questions About Dependence

Covariance summarizes whether two variables tend to vary together in a linear way. Independence asks a stronger question: whether knowing one variable changes the distribution of the other. These ideas are related, but they are not interchangeable. In particular, zero covariance does not generally establish independence.

The distinction matters when interpreting a correlation analysis. A zero covariance rules out a particular kind of association in the second moments; it does not rule out every possible relationship. The examples below make that distinction concrete, including cases where the variables are dependent despite having zero covariance.

Definitions and the Implication That Always Holds

Definition: Random variables \(X\) and \(Y\) are independent if \(P(X\in A,\;Y\in B)=P(X\in A)P(Y\in B)\) for every pair of Borel sets \(A,B\). They are uncorrelated if both have finite second moments and \(\operatorname{Cov}(X,Y)=0\), where \(\operatorname{Cov}(X,Y)=E[(X-E[X])(Y-E[Y])]\). For random vectors \(X\in\mathbb{R}^p\) and \(Y\in\mathbb{R}^q\), their cross-covariance matrix is \(\operatorname{Cov}(X,Y)=E[(X-E[X])(Y-E[Y])^T]\), when the componentwise second moments are finite.

For scalar variables with finite second moments, the covariance can also be written \(\operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y]\). The finite-moment condition matters: without it, a covariance may not be defined. Independence, by contrast, is a statement about the joint distribution and does not itself require moments.

Theorem: If \(X\) and \(Y\) are independent and have finite second moments, then \(\operatorname{Cov}(X,Y)=0\). More generally, independent random vectors with finite componentwise second moments have a zero cross-covariance matrix.

Proof. Finite second moments imply finite first moments. Also, the Cauchy–Schwarz inequality gives \(E[|XY|]\leq \sqrt{E[X^2]E[Y^2]}<\infty\), so the product expectation exists. Independence means the joint distribution of \((X,Y)\) is the product of its marginal distributions. Integrating \(xy\) with respect to that product distribution is legitimate because the integral of \(|xy|\) is finite; hence

E[XY] =\int\!\!\int xy\,dF_X(x)\,dF_Y(y) =\left(\int x\,dF_X(x)\right)\left(\int y\,dF_Y(y)\right) =E[X]E[Y].

It follows that \(\operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y]=0\). For random vectors, apply the same argument to each component pair \(X_i,Y_j\); every entry of the cross-covariance matrix is zero. \(\square\)

The theorem gives a one-way implication. Its assumptions and conclusion should be read carefully: independence plus finite second moments guarantees zero covariance. It does not say that zero covariance guarantees independence.

Worked Examples: Checking Both Concepts

Worked Example: Independent Bernoulli Variables With Nonzero Means

Let \(X\sim\operatorname{Bernoulli}(0.6)\) and \(Y\sim\operatorname{Bernoulli}(0.25)\), generated independently. Thus each variable is 1 with its stated probability and 0 otherwise. Their means are \(E[X]=0.6\) and \(E[Y]=0.25\), both nonzero.

Independence gives \(P(X=1,Y=1)=0.6(0.25)=0.15\). Since \(XY=1\) only when both variables equal 1, \(E[XY]=0.15\). Therefore

\operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y] =0.15-(0.6)(0.25)=0.

The raw cross-moment \(E[XY]\) is not zero, even though the covariance is. Covariance subtracts the product of the means. This example also shows that independence does not require either variable to have mean zero.

Result: Adding constants to either variable does not change covariance: for constants \(a,b\), \(\operatorname{Cov}(X+a,Y+b)=\operatorname{Cov}(X,Y)\), provided the original covariance is defined.

Derivation. Since \(E[X+a]=E[X]+a\) and \(E[Y+b]=E[Y]+b\), the centered variables satisfy \((X+a)-E[X+a]=X-E[X]\) and \((Y+b)-E[Y+b]=Y-E[Y]\). Substituting these identities into the definition of covariance gives

\operatorname{Cov}(X+a,Y+b) =E[(X-E[X])(Y-E[Y])] =\operatorname{Cov}(X,Y).

This identity helps separate covariance from the location of the variables: changing their means does not, by itself, change their covariance.

Worked Example: A Discrete Dependent Pair With Zero Covariance

Let \(X\) take each of \(-2,-1,1,2\) with probability \(1/4\), and define \(Y=X^2\). Both variables have finite moments. First,

E[X]=\frac{-2-1+1+2}{4}=0,\qquad E[Y]=\frac{4+1+1+4}{4}=\frac{5}{2}.

Because \(XY=X^3\),

E[XY]=E[X^3] =\frac{-8-1+1+8}{4}=0, \qquad \operatorname{Cov}(X,Y)=0-(0)\left(\frac{5}{2}\right)=0.

Nevertheless, \(X\) and \(Y\) are dependent: \(Y\) is determined exactly by \(X\). To verify dependence using the definition, \(P(X=1,Y=1)=1/4\), whereas \(P(X=1)P(Y=1)=(1/4)(1/2)=1/8\). The probabilities are unequal, so the required factorization fails. The zero covariance results from cancellation between positive and negative values of \(X\), not from independence.

Worked Example: A Continuous Dependent Pair With Zero Covariance

Let \(X\sim\operatorname{Uniform}(-1,1)\), let \(Z\sim\operatorname{Uniform}(-1,1)\) be independent of \(X\), and define \(Y=X^2+Z\). Both \(X\) and \(Y\) are continuous random variables, and their second moments are finite. Symmetry gives \(E[X]=0\), while \(E[X^2]=1/3\) and \(E[Z]=0\), so \(E[Y]=1/3\).

Using independence of \(X\) and \(Z\), and symmetry of \(X\),

$$ E[XY]=E[X^3]+E[XZ]=0+E[X]E[Z]=0. $$

It follows that \(\operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y]=0-(0)(1/3)=0\). Yet the variables are dependent. In fact, \(E[Y\mid X]=X^2\), which is not constant: for example, \(X^2<1/9\) and \(X^2>4/9\) each occur with positive probability. If \(X\) and \(Y\) were independent, then \(E[Y\mid X]=E[Y]=1/3\) almost surely. The conditional expectation is not constant, so independence fails.

Here the added noise \(Z\) does not remove the dependence. The conditional mean of \(Y\) still changes with \(X\), even though the covariance is zero.

What Zero Cross-Covariance Does and Does Not Say

For random vectors, a zero cross-covariance matrix means that every component of one vector has zero covariance with every component of the other. It is stronger than checking just one selected pair of components, but it is still a statement about second-order relationships. It does not, in general, imply that the vectors are independent. The discrete and continuous scalar examples already establish this: scalar variables are a special case of random vectors with one component.

A related source of confusion is the difference between covariance and the uncentered product moment \(E[XY]\). The identity \(E[XY]=\operatorname{Cov}(X,Y)+E[X]E[Y]\) shows why a zero covariance need not make the product moment zero. Conversely, a zero product moment does not by itself imply zero covariance if the means are nonzero.

If both variances are positive, correlation is the standardized covariance, \(\operatorname{Corr}(X,Y)=\operatorname{Cov}(X,Y)/\sqrt{\operatorname{Var}(X)\operatorname{Var}(Y)}\). In that case, zero correlation and zero covariance are equivalent. This equivalence still does not turn either condition into independence. If a variance is zero, that variable is constant almost surely and the correlation denominator is zero, so correlation is not defined by this formula.

Common Mistake and Practical Interpretation

Wrong reasoning: “The covariance is zero, so the variables are independent.” This reverses the implication proved above. The discrete example has zero covariance but fails the defining probability factorization; the continuous example is also dependent despite zero covariance.
Correction: Conclude only that the covariance is zero, or that the variables are uncorrelated. Independence requires the joint distribution to factor into the marginal distributions.
Wrong reasoning: “The variables are independent, so \(E[XY]=0\).” Independence gives \(E[XY]=E[X]E[Y]\) when the relevant expectations exist; this product is generally nonzero when both means are nonzero.
Correction: Center the variables when calculating covariance. The means are subtracted in the definition, which is why independent variables can have nonzero means and still have zero covariance.

In analysis, zero covariance is useful information, but its interpretation should match its scope. It describes a second-moment property; a claim of independence is a claim about the full joint distribution and needs additional justification. Keeping those conclusions distinct prevents a narrow summary from being mistaken for a complete dependence assessment.

Check Your Understanding

Use the definitions and calculations above to answer these questions.

State the moment condition needed for the theorem that independence implies zero covariance.
For independent \(X\) and \(Y\) with means 2 and 3, what is \(E[XY]\), assuming the expectations exist?
In the discrete example, give a pair of events whose joint probability is not the product of their marginal probabilities.
Why does \(E[Y\mid X]=X^2\) show dependence in the continuous example?
Does a zero cross-covariance matrix generally establish independence of two random vectors? Explain briefly.