Why Zero Covariance Does Not Prove Independence

Why a Covariance Calculation Is Not an Independence Test

For two real random variables with finite second moments, independence implies zero covariance. The converse does not generally hold, and central Gaussian quadratic forms provide direct examples of the distinction. Their covariance has a compact matrix formula, but that formula is only one numerical summary of the joint distribution. A zero value can conceal dependence in higher moments or in conditional distributions.

Let \(X\) be a Gaussian vector with mean zero and covariance matrix \(\Sigma\), and let \(A\) and \(B\) be symmetric matrices of matching dimensions. The forms \(X^{\mathsf T}AX\) and \(X^{\mathsf T}BX\) are central Gaussian quadratic forms. The covariance formula applies even if \(\Sigma\) is singular, an important point for projected or linearly constrained Gaussian vectors.

Definition: For a mean-zero Gaussian vector \(X\in\mathbb{R}^p\) with covariance matrix \(\Sigma\), and symmetric \(p\times p\) matrices \(A\) and \(B\), define the central quadratic forms \(Q_A=X^{\mathsf T}AX\) and \(Q_B=X^{\mathsf T}BX\). Their covariance is \[ \operatorname{Cov}(Q_A,Q_B) =E[(Q_A-E[Q_A])(Q_B-E[Q_B])]. \]

Symmetry is a convenient convention: for any real matrix \(M\), \(x^{\mathsf T}Mx=x^{\mathsf T}(M+M^{\mathsf T})x/2\). Thus a quadratic form depends only on the symmetric part of its matrix. We use symmetric matrices throughout so that the trace expressions below have their stated form.

The Covariance Formula

Theorem: If \(X\sim N_p(0,\Sigma)\), where \(\Sigma\) is positive semidefinite, and \(A,B\) are symmetric \(p\times p\) matrices, then \[ E[X^{\mathsf T}AX]=\operatorname{tr}(A\Sigma) \] and \[ \operatorname{Cov}(X^{\mathsf T}AX,X^{\mathsf T}BX) =2\operatorname{tr}(A\Sigma B\Sigma). \] No nonsingularity assumption on \(\Sigma\) is required.

Here is a derivation that also explains why singular covariance matrices cause no difficulty. A positive semidefinite matrix has a symmetric square root \(\Sigma^{1/2}\). Even when \(\Sigma\) is singular, \(X\) has the same distribution as \(\Sigma^{1/2}Z\), where \(Z\sim N_p(0,I_p)\). Set \(C=\Sigma^{1/2}A\Sigma^{1/2}\) and \(D=\Sigma^{1/2}B\Sigma^{1/2}\). Both are symmetric, and the two forms have the same joint distribution as \(Z^{\mathsf T}CZ\) and \(Z^{\mathsf T}DZ\).

Moment identity: For independent standard normal coordinates \(Z_1,\ldots,Z_p\), \[ E[Z_iZ_jZ_kZ_l] =\delta_{ij}\delta_{kl}+\delta_{ik}\delta_{jl}+\delta_{il}\delta_{jk}, \] where \(\delta_{ij}=1\) if \(i=j\) and \(0\) otherwise.

To verify the identity, independence and symmetry make the expectation zero whenever some coordinate occurs an odd number of times. If the four indices form two distinct pairs, the expectation is \(E[Z_r^2]E[Z_s^2]=1\), and exactly one term on the right is 1. If all four indices agree, the expectation is \(E[Z_r^4]=3\), and all three terms on the right are 1. These cases exhaust the possibilities with nonzero expectation, proving the identity.

Expand the product of the two forms and apply that identity:

E[(Z^{\mathsf T}CZ)(Z^{\mathsf T}DZ)] =\sum_{i,j,k,l} C_{ij}D_{kl}E[Z_iZ_jZ_kZ_l] =\operatorname{tr}(C)\operatorname{tr}(D)+2\operatorname{tr}(CD).

The first pairing, \(\delta_{ij}\delta_{kl}\), contributes \(\operatorname{tr}(C)\operatorname{tr}(D)\). The second, \(\delta_{ik}\delta_{jl}\), contributes \(\sum_{i,j}C_{ij}D_{ij}=\operatorname{tr}(CD)\), since \(D\) is symmetric. The third contributes \(\sum_{i,j}C_{ij}D_{ji}=\operatorname{tr}(CD)\). Also, \(E[Z^{\mathsf T}CZ]=\operatorname{tr}(C)\) and \(E[Z^{\mathsf T}DZ]=\operatorname{tr}(D)\), by expanding and using \(E[Z_iZ_j]=\delta_{ij}\). Subtracting the product of the means gives covariance \(2\operatorname{tr}(CD)\). Finally, the cyclic property of the trace gives \[ \operatorname{tr}(CD) =\operatorname{tr}(\Sigma^{1/2}A\Sigma B\Sigma^{1/2}) =\operatorname{tr}(A\Sigma B\Sigma). \] Also \(\operatorname{tr}(C)=\operatorname{tr}(A\Sigma)\). This proves both formulas, including in the singular case. \(\square\)

Corollary: The variance of a central Gaussian quadratic form is \[ \operatorname{Var}(X^{\mathsf T}AX)=2\operatorname{tr}(A\Sigma A\Sigma). \] Two such forms have zero covariance exactly when \(\operatorname{tr}(A\Sigma B\Sigma)=0\).

The corollary follows by setting \(B=A\) in the theorem. The zero-covariance condition is likewise just the covariance formula with its factor of 2 removed. It is a matrix trace condition; it does not say that the joint distribution factors into the product of the marginal distributions.

Worked Examples

Worked Example: Zero Covariance and Independence

Let \(X=(X_1,X_2)^{\mathsf T}\sim N_2(0,I_2)\), and set \(Q_1=X_1^2\), \(Q_2=X_2^2\). Take \[ A=\begin{pmatrix}1&0\\0&0\end{pmatrix}, \qquad B=\begin{pmatrix}0&0\\0&1\end{pmatrix}. \] Then \(Q_1=X^{\mathsf T}AX\), \(Q_2=X^{\mathsf T}BX\), and \(\Sigma=I_2\).

The covariance formula gives \[ \operatorname{Cov}(Q_1,Q_2)=2\operatorname{tr}(AB)=0, \] because \(AB\) is the zero matrix. In this example the forms really are independent: \(Q_1\) is a function of \(X_1\), \(Q_2\) is a function of \(X_2\), and the two standard normal coordinates are independent. Each form has a chi-square distribution with one degree of freedom. The independence here comes from the independence of the coordinates, not merely from the zero covariance.

Worked Example: Zero Covariance but Dependence Through a Conditional Second Moment

Let \(X=(X_1,X_2)^{\mathsf T}\sim N_2(0,I_2)\), and define \[ Q_1=X_1^2,\qquad Q_2=2X_1X_2. \] The corresponding symmetric matrices are \[ A=\begin{pmatrix}1&0\\0&0\end{pmatrix}, \qquad B=\begin{pmatrix}0&1\\1&0\end{pmatrix}, \] since \(X^{\mathsf T}BX=2X_1X_2\). Their product has trace zero, so \[ \operatorname{Cov}(Q_1,Q_2)=2\operatorname{tr}(AB)=0. \]

To test independence, consider the conditional second moment of \(Q_2\). Since \(X_2\) is independent of \(Q_1=X_1^2\), and \(E[X_2^2]=1\), \[ E[Q_2^2\mid Q_1] =E[4X_1^2X_2^2\mid Q_1] =4Q_1. \] Its unconditional value is \(E[Q_2^2]=4E[X_1^2]E[X_2^2]=4\). If \(Q_1\) and \(Q_2\) were independent, the integrable variable \(Q_2^2\) would satisfy \(E[Q_2^2\mid Q_1]=E[Q_2^2]=4\) almost surely. Instead, the conditional expectation is \(4Q_1\), and \(Q_1\) is not almost surely constant: it has mean 1 and variance 2. Therefore \(Q_1\) and \(Q_2\) are dependent despite having zero covariance.

Worked Example: A Second Dependent Pair With Zero Covariance

Let \(X=(X_1,X_2,X_3)^{\mathsf T}\sim N_3(0,I_3)\), and define \(Q_1=X_1^2+X_2^2\), \(Q_2=2X_1X_3\). Use \[ A=\begin{pmatrix}1&0&0\\0&1&0\\0&0&0\end{pmatrix}, \qquad B=\begin{pmatrix}0&0&1\\0&0&0\\1&0&0\end{pmatrix}. \] Then \(Q_1=X^{\mathsf T}AX\), \(Q_2=X^{\mathsf T}BX\), and \(\operatorname{tr}(AB)=0\). Consequently, \(\operatorname{Cov}(Q_1,Q_2)=0\).

But \(Q_2^2=4X_1^2X_3^2\). Given \(Q_1\), exchangeability of \(X_1\) and \(X_2\), together with \(X_1^2+X_2^2=Q_1\), gives \(E[X_1^2\mid Q_1]=Q_1/2\). Also \(X_3\) is independent of \((X_1,X_2)\), so \(E[X_3^2\mid Q_1]=1\). Therefore \[ E[Q_2^2\mid Q_1] =4E[X_1^2\mid Q_1]E[X_3^2\mid Q_1] =2Q_1. \] Unconditionally, \(E[Q_2^2]=4E[X_1^2]E[X_3^2]=4\). Independence would require the conditional expectation to equal 4 almost surely. Since \(Q_1\) is nonconstant, \(2Q_1\) is not almost surely 4, so the forms are dependent.

Interpreting the Matrix Condition

For a standard normal vector, \(\Sigma=I_p\), and the covariance reduces to \(2\operatorname{tr}(AB)\). For a general covariance matrix, it is \(2\operatorname{tr}(A\Sigma B\Sigma)\). Either way, zero covariance means that a particular trace is zero. Different matrices can produce a zero trace through cancellation, even when the forms still share random variation. The conditional-moment calculations above expose dependence that the covariance alone cannot detect.

Independence is a statement about the full joint distribution: for every pair of suitable sets, the probability that the two variables fall in those sets must factor into the product of the marginal probabilities. A single covariance cannot verify all of those requirements. For Gaussian quadratic forms, Craig's criterion gives a stronger matrix criterion for independence; a zero trace is not a substitute for that criterion.

Common Mistake

Wrong reasoning: “The covariance formula gives zero, so the two forms are independent.” This reasoning confuses a necessary implication with a sufficient condition. Independence implies zero covariance when the second moments exist, but the converse is false. The correction is to treat zero covariance as a moment statement only. To establish independence, use an applicable independence criterion or prove that the joint distribution factors; to disprove it, a conditional distribution or conditional moment that depends on the other form is enough.

The examples also show why checking only conditional means may miss dependence. In the second example, symmetry gives \(E[Q_2\mid Q_1]=0\), the same as \(E[Q_2]\), but \(E[Q_2^2\mid Q_1]=4Q_1\) varies with \(Q_1\). Examining a conditional second moment reveals structure that the covariance does not register.

Key takeaway: For central Gaussian quadratic forms, covariance is \(2\operatorname{tr}(A\Sigma B\Sigma)\). A zero trace establishes uncorrelatedness, not independence; additional distributional or matrix structure is needed to establish independence.

Check Your Understanding

Use the covariance formula and, where needed, conditional moments to answer these questions.

For \(X\sim N_p(0,\Sigma)\), what matrix trace must vanish for \(X^{\mathsf T}AX\) and \(X^{\mathsf T}BX\) to have zero covariance?
Why does the covariance formula remain valid when \(\Sigma\) is singular?
For independent standard normal \(X_1,X_2\), explain why \(X_1^2\) and \(X_2^2\) are independent, not merely uncorrelated.
In the pair \(Q_1=X_1^2\), \(Q_2=2X_1X_2\), what conditional moment shows that the forms are dependent?
Why is zero covariance a necessary but not sufficient condition for independence when the variables have finite second moments?