Zero Cross-Covariance and Gaussian Block Independence

Why a Covariance Block Can Decide Independence

For general random variables, zero covariance is only a statement about a second moment; it does not usually rule out dependence. Joint Gaussian models are special. Within this family, the entire joint distribution is determined by its mean vector and covariance matrix, so the cross-covariance block can determine whether two subvectors are independent.

This criterion is useful when a Gaussian vector is partitioned into groups of coordinates. It lets us decide independence by inspecting a block of the covariance matrix, without trying to factor a complicated joint density. The result also applies when that covariance matrix is singular, where a density with respect to ordinary volume may not exist.

Definition: Let $X\in\mathbb{R}^p$ and $Y\in\mathbb{R}^q$ have finite second moments. Their cross-covariance matrix is $$ \operatorname{Cov}(X,Y) = E\big[(X-E[X])(Y-E[Y])^{\mathsf T}\big], $$ a $p\times q$ matrix whose $(i,j)$ entry is $\operatorname{Cov}(X_i,Y_j)$. The vectors $X$ and $Y$ are jointly Gaussian if the concatenated vector $(X^{\mathsf T},Y^{\mathsf T})^{\mathsf T}$ has a multivariate Gaussian distribution; singular Gaussian distributions are included.

Write the mean and covariance of the concatenated vector in block form:

\begin{pmatrix}X\\Y\end{pmatrix} \sim N\left( \begin{pmatrix}\mu_X\\\mu_Y\end{pmatrix}, \begin{pmatrix} \Sigma_{XX} & \Sigma_{XY}\\ \Sigma_{YX} & \Sigma_{YY} \end{pmatrix} \right), \qquad \Sigma_{XY}=\operatorname{Cov}(X,Y).

The matrix $\Sigma_{YX}$ is $\Sigma_{XY}^{\mathsf T}$. The question is whether the two subvectors are independent, not merely whether a particular pair of coordinates has zero covariance.

The Gaussian Block Independence Criterion

Theorem: If $X$ and $Y$ are jointly Gaussian, then $$ X\text{ and }Y\text{ are independent} \quad\Longleftrightarrow\quad \operatorname{Cov}(X,Y)=0. $$ No nonsingularity assumption on the joint covariance matrix is required.

To prove the theorem, use the characteristic function of a Gaussian vector. For a Gaussian vector $Z$ with mean $\mu$ and covariance $\Sigma$, its characteristic function at $u$ is $\exp\{i u^{\mathsf T}\mu-\tfrac12u^{\mathsf T}\Sigma u\}$. This formula holds for singular as well as nonsingular Gaussian distributions.

First suppose $X$ and $Y$ are independent. Each coordinate has a finite second moment because Gaussian distributions have finite second moments. For every $i,j$, independence gives

\operatorname{Cov}(X_i,Y_j) =E[(X_i-E[X_i])(Y_j-E[Y_j])] =E[X_i-E[X_i]]\,E[Y_j-E[Y_j]] =0.

Every entry of $\operatorname{Cov}(X,Y)$ is therefore zero, so the whole block is zero.

For the reverse direction, suppose $\Sigma_{XY}=0$. Let $s\in\mathbb{R}^p$ and $t\in\mathbb{R}^q$. The joint characteristic function of $(X,Y)$, evaluated at $(s,t)$, is

\begin{aligned} \phi_{X,Y}(s,t) &=\exp\left\{i(s^{\mathsf T}\mu_X+t^{\mathsf T}\mu_Y) -\frac12\left(s^{\mathsf T}\Sigma_{XX}s +2s^{\mathsf T}\Sigma_{XY}t+t^{\mathsf T}\Sigma_{YY}t\right)\right\}\\ &=\exp\left\{i s^{\mathsf T}\mu_X-\frac12s^{\mathsf T}\Sigma_{XX}s\right\} \exp\left\{i t^{\mathsf T}\mu_Y-\frac12t^{\mathsf T}\Sigma_{YY}t\right\}\\ &=\phi_X(s)\phi_Y(t). \end{aligned}

The product $\phi_X(s)\phi_Y(t)$ is the characteristic function of the product of the marginal distributions of $X$ and $Y$. Since characteristic functions uniquely determine probability distributions, the joint distribution equals that product distribution. This is precisely independence. The proof uses no inverse covariance matrix and no density, so it remains valid in the singular case. $\square$

Worked Example: A Zero Cross-Covariance Block

Suppose $(X,Y)$ is jointly Gaussian with mean $(1,-2)^{\mathsf T}$ and covariance matrix $$ \begin{pmatrix}4&0\\0&9\end{pmatrix}. $$ Determine whether $X$ and $Y$ are independent, and verify the conclusion using the density.

The cross-covariance block is the off-diagonal entry, which is $0$. The theorem therefore gives independence. The marginal distributions are $X\sim N(1,4)$ and $Y\sim N(-2,9)$. Their densities are $$ f_X(x)=\frac{1}{2\sqrt{2\pi}}\exp\left(-\frac{(x-1)^2}{8}\right), \qquad f_Y(y)=\frac{1}{3\sqrt{2\pi}}\exp\left(-\frac{(y+2)^2}{18}\right). $$

The determinant of the covariance matrix is $4\cdot9=36$, so the joint density is $$ f_{X,Y}(x,y)=\frac{1}{2\pi\sqrt{36}} \exp\left(-\frac{(x-1)^2}{8}-\frac{(y+2)^2}{18}\right). $$ Since $\sqrt{36}=6$, its leading constant is $1/(12\pi)$. The product of the marginal leading constants is also $$ \frac{1}{2\sqrt{2\pi}}\frac{1}{3\sqrt{2\pi}}=\frac{1}{12\pi}, $$ and the exponents add. Thus $f_{X,Y}(x,y)=f_X(x)f_Y(y)$, confirming independence directly.

Applying the Criterion to Partitioned Models

For subvectors, the check is matrix-valued: every entry in the cross block must be zero. A single nonzero entry is enough to show that the blocks are not independent. It is important to calculate the block from the stated covariance matrix rather than to infer independence from a selected pair of coordinates or a visual impression of the model.

Worked Example: A Nonzero Cross Block Rules Out Independence

Let $(X_1,X_2,Y)$ be jointly Gaussian with mean zero and covariance $$ \operatorname{Cov}(X_1,X_2,Y)= \begin{pmatrix} 2&1&1\\ 1&2&-2\\ 1&-2&5 \end{pmatrix}. $$ Take $X=(X_1,X_2)^{\mathsf T}$ as one block and $Y$ as the other. Are these blocks independent?

The cross-covariance block is read from the first two rows and the last column: $$ \operatorname{Cov}(X,Y)= \begin{pmatrix}1\\-2\end{pmatrix}. $$ It is not the zero matrix, so the Gaussian criterion says that $X$ and $Y$ are not independent. The covariance matrix is a valid positive-definite covariance matrix: its leading principal minors are $2$, $2\cdot2-1\cdot1=3$, and $$ \det\begin{pmatrix}2&1&1\\1&2&-2\\1&-2&5\end{pmatrix} =2(10-4)-(5+2)+(-2-2)=12-7-4=1, $$ all of which are positive.

A direct check using a linear combination gives the same warning. For $T=X_1-X_2$, $$ \operatorname{Cov}(T,Y)=\operatorname{Cov}(X_1,Y)-\operatorname{Cov}(X_2,Y)=1-(-2)=3. $$ Also, $\operatorname{Var}(T)=2+2-2(1)=2$. In particular, $T$ and $Y$ cannot be independent, and the blocks containing them cannot be independent. The calculation uses the $(2,3)$ covariance entry $-2$ with its sign intact.

Worked Example: Independence with Singular Gaussian Blocks

Let $Z$ and $W$ be independent $N(0,1)$ random variables, and define $X=(Z,2Z)^{\mathsf T}$ and $Y=(W,-W)^{\mathsf T}$. Check the cross-covariance criterion and explain why the singularity of the blocks causes no difficulty.

The covariance matrices within the two blocks are $$ \operatorname{Cov}(X)= \begin{pmatrix}1&2\\2&4\end{pmatrix}, \qquad \operatorname{Cov}(Y)= \begin{pmatrix}1&-1\\-1&1\end{pmatrix}. $$ For example, $\operatorname{Cov}(Z,2Z)=2\operatorname{Var}(Z)=2$, and $\operatorname{Cov}(W,-W)=-\operatorname{Var}(W)=-1$. Independence of $Z$ and $W$ makes each cross-covariance zero: $$ \operatorname{Cov}(X,Y)= \begin{pmatrix} \operatorname{Cov}(Z,W)&\operatorname{Cov}(Z,-W)\\ \operatorname{Cov}(2Z,W)&\operatorname{Cov}(2Z,-W) \end{pmatrix} = \begin{pmatrix}0&0\\0&0\end{pmatrix}. $$ The concatenated vector is jointly Gaussian because it is a linear transformation of the Gaussian vector $(Z,W)^{\mathsf T}$. The theorem therefore implies that $X$ and $Y$ are independent. Each block covariance is singular: for $X$, the second coordinate is twice the first, and for $Y$, the second coordinate is the negative of the first. Their singularity prevents an ordinary full-dimensional density, but does not affect the characteristic-function argument or the criterion.

More Than Two Blocks

The same idea applies to a partition into several Gaussian subvectors. Mutual independence means that the joint distribution factors into the product of all the block marginal distributions. For jointly Gaussian blocks, this has a simple covariance characterization.

Corollary: Let $X_1,\ldots,X_k$ be subvectors of one jointly Gaussian vector. They are mutually independent if and only if $\operatorname{Cov}(X_a,X_b)=0$ for every distinct pair $a,b$.

If the blocks are mutually independent, every pair is independent, and the first direction of the theorem gives $\operatorname{Cov}(X_a,X_b)=0$ for all distinct $a,b$. Conversely, suppose all these blocks are zero. In the joint Gaussian characteristic function, the quadratic form in its exponent has no cross terms between distinct blocks. It therefore splits into a sum of one quadratic form for each block. Exponentiating turns that sum into a product of the $k$ marginal characteristic functions. Uniqueness of characteristic functions identifies the joint distribution with the product of the marginals, which is mutual independence. $\square$

Thus, for a jointly Gaussian vector partitioned into $k$ blocks, pairwise zero cross-covariance blocks imply mutual independence of all $k$ blocks. This implication relies on joint Gaussianity; it is not a general rule for arbitrary collections of random vectors.

Common Mistake: Extending the Criterion Beyond Gaussian Models

A frequent error is to treat zero covariance as a universal synonym for independence. The theorem only makes that equivalence under joint Gaussianity. For a concrete counterexample, let $X\sim N(0,1)$, let $S$ be independent of $X$ with $P(S=1)=P(S=-1)=1/2$, and set $Y=SX$.

Both $X$ and $Y$ have mean zero, and $$ \operatorname{Cov}(X,Y)=E[XY]=E[SX^2]=E[S]E[X^2]=0\cdot1=0. $$ Nevertheless, $|Y|=|X|$ with probability one, so $X$ and $Y$ are dependent. Indeed, if they were independent standard normal variables, the event $\{|X|=|Y|\}$ would have probability zero: the pair has a joint density, and the two lines $y=x$ and $y=-x$ have two-dimensional probability zero. The pair $(X,Y)$ is not jointly Gaussian, which is exactly why the Gaussian criterion cannot be applied.

Key takeaway: For jointly Gaussian subvectors, inspect the full cross-covariance block: it is zero exactly when the subvectors are independent. This remains true for singular Gaussian distributions. Outside the jointly Gaussian setting, zero covariance alone does not establish independence.

Check Your Understanding

Use the joint Gaussian criterion where its hypothesis holds, and distinguish a cross block from the covariance within a block.

A jointly Gaussian vector is partitioned as $X=(X_1,X_2)^{\mathsf T}$ and $Y=(Y_1,Y_2)^{\mathsf T}$. What must be true of the four cross-covariances for $X$ and $Y$ to be independent?
If one entry of $\operatorname{Cov}(X,Y)$ is nonzero and $X,Y$ are jointly Gaussian, what can you conclude?
Why does the characteristic-function proof still work when the joint covariance matrix is singular?
For jointly Gaussian blocks $X_1,X_2,X_3$, is it enough for mutual independence that each pair has zero cross-covariance? Explain.
In the construction $Y=SX$, with $X$ standard normal and $S$ an independent random sign, why does zero covariance fail to imply independence?