Zero Cross-Covariance and Gaussian Independence

Why Zero Cross-Covariance Is Special for Gaussian Blocks

Covariance measures linear association. In general, two random variables can have zero covariance and still be dependent: covariance does not detect every possible relationship. The multivariate Gaussian family has a stronger property. For jointly Gaussian random vectors, a zero cross-covariance matrix does not merely say that the blocks are uncorrelated; it guarantees that they are independent.

The qualification jointly Gaussian matters. It is not enough that each block separately has a Gaussian distribution. The conclusion depends on the distribution of the two blocks taken together. This tutorial establishes the precise criterion, including the case of singular covariance matrices, and then checks both the Gaussian result and its limitation.

Definition and Independence Criterion

Definition: Let $X\in\mathbb{R}^p$ and $Y\in\mathbb{R}^q$ be random vectors with finite second moments. Their cross-covariance matrix is $$ \operatorname{Cov}(X,Y)=E\big[(X-E[X])(Y-E[Y])^\mathsf{T}\big], $$ a $p\times q$ matrix. Its $(i,j)$ entry is $\operatorname{Cov}(X_i,Y_j)$. The vectors are jointly Gaussian if the concatenated vector $(X^\mathsf{T},Y^\mathsf{T})^\mathsf{T}$ has a multivariate Gaussian distribution, allowing a singular covariance matrix.

Write the mean and covariance of the concatenated vector in block form as

E\begin{bmatrix}X\\Y\end{bmatrix} = \begin{bmatrix}\mu_X\\\mu_Y\end{bmatrix}, \qquad \operatorname{Cov}\begin{pmatrix}X\\Y\end{pmatrix} = \begin{bmatrix} \Sigma_X & \Sigma_{XY}\\ \Sigma_{YX} & \Sigma_Y \end{bmatrix}, \qquad \Sigma_{XY}=\operatorname{Cov}(X,Y).

Thus $\Sigma_{XY}=0$ means that every component of $X$ has zero covariance with every component of $Y$. It does not require the components within either block to be uncorrelated.

Theorem: If $X$ and $Y$ are jointly Gaussian, then $X$ and $Y$ are independent if and only if $\operatorname{Cov}(X,Y)=0$. This statement includes degenerate Gaussian distributions. More generally, for any independent $X$ and $Y$ with finite second moments, $\operatorname{Cov}(X,Y)=0$; the Gaussian assumption is needed for the reverse implication.

Proof by Characteristic Functions

For a Gaussian vector with mean $\mu$ and covariance matrix $\Sigma$, its characteristic function at $t$ is $\exp\{i t^\mathsf{T}\mu-\tfrac12 t^\mathsf{T}\Sigma t\}$. This formula also holds when $\Sigma$ is singular. Apply it to the concatenated vector and partition the argument into $t\in\mathbb{R}^p$ and $s\in\mathbb{R}^q$.

Proof: Suppose first that $\operatorname{Cov}(X,Y)=\Sigma_{XY}=0$. The characteristic function of the concatenated vector is $$ \varphi_{X,Y}(t,s) = \exp\left\{ i t^\mathsf{T}\mu_X+i s^\mathsf{T}\mu_Y -\frac12\left( t^\mathsf{T}\Sigma_Xt +t^\mathsf{T}\Sigma_{XY}s +s^\mathsf{T}\Sigma_{YX}t +s^\mathsf{T}\Sigma_Ys \right) \right\}. $$ Because $\Sigma_{XY}=0$ and $\Sigma_{YX}=\Sigma_{XY}^\mathsf{T}=0$, this becomes $$ \varphi_{X,Y}(t,s) = \exp\left\{i t^\mathsf{T}\mu_X-\frac12t^\mathsf{T}\Sigma_Xt\right\} \exp\left\{i s^\mathsf{T}\mu_Y-\frac12s^\mathsf{T}\Sigma_Ys\right\} = \varphi_X(t)\varphi_Y(s). $$ The product $\varphi_X(t)\varphi_Y(s)$ is the characteristic function of the product of the marginal distributions of $X$ and $Y$. Characteristic functions uniquely determine probability distributions, so the joint distribution equals that product distribution. Hence $X$ and $Y$ are independent.

Conversely, suppose $X$ and $Y$ are independent. For each component pair $X_i,Y_j$, independence and finite second moments give $$ E[(X_i-E[X_i])(Y_j-E[Y_j])] = E[X_i-E[X_i]]\,E[Y_j-E[Y_j]]=0. $$ Every entry of $\operatorname{Cov}(X,Y)$ is therefore zero. Gaussian vectors have finite second moments, so this argument applies in the stated theorem. $\square$

The factorization is the key step: when the cross-covariance blocks vanish, the Gaussian characteristic function separates into a function of $t$ times a function of $s$. No density or matrix inverse is required, which is why the proof also covers singular Gaussian laws.

Worked Checks

Worked Example: Reading Independence From a Block Covariance

Let $Z_1,Z_2,Z_3,Z_4$ be independent $N(0,1)$ random variables, and define $X=(2Z_1,\ Z_1+Z_2)^\mathsf{T}$ and $Y=(3Z_3,\ Z_3+2Z_4)^\mathsf{T}$. Find their cross-covariance and determine whether the blocks are independent.

Both blocks are linear transformations of a Gaussian vector, so they are jointly Gaussian. Since the $Z_1,Z_2$ variables are independent of $Z_3,Z_4$, every component of $X$ has zero covariance with every component of $Y$. In particular,

\operatorname{Cov}(X,Y) = \begin{bmatrix} \operatorname{Cov}(2Z_1,3Z_3) & \operatorname{Cov}(2Z_1,Z_3+2Z_4)\\ \operatorname{Cov}(Z_1+Z_2,3Z_3) & \operatorname{Cov}(Z_1+Z_2,Z_3+2Z_4) \end{bmatrix} = \begin{bmatrix}0&0\\0&0\end{bmatrix}.

For a check of the within-block covariance matrices, $\operatorname{Var}(X_1)=4$, $\operatorname{Var}(X_2)=2$, and $\operatorname{Cov}(X_1,X_2)=2$, giving $\Sigma_X=\begin{bmatrix}4&2\\2&2\end{bmatrix}$. Likewise, $\Sigma_Y=\begin{bmatrix}9&3\\3&5\end{bmatrix}$. The within-block covariances are not zero, but that is not required. The joint Gaussian criterion applies to the zero cross-block, so $X$ and $Y$ are independent.

Worked Example: Orthogonal Gaussian Linear Combinations

Let $Z_1,Z_2$ be independent $N(0,1)$ variables, and set $U=Z_1+2Z_2$ and $V=2Z_1-Z_2$. Check whether $U$ and $V$ are independent.

The pair $(U,V)$ is jointly Gaussian because it is a linear transformation of $(Z_1,Z_2)$. Its means are both zero. Using independence and unit variances,

\operatorname{Var}(U)=1+4=5,\qquad \operatorname{Var}(V)=4+1=5,

and

\operatorname{Cov}(U,V) = \operatorname{Cov}(Z_1+2Z_2,2Z_1-Z_2) = 2\operatorname{Var}(Z_1)-2\operatorname{Var}(Z_2) =2-2=0.

Thus the covariance matrix of $(U,V)$ is $\begin{bmatrix}5&0\\0&5\end{bmatrix}$. The theorem gives independence. In fact, each marginal distribution is $N(0,5)$, so $U$ and $V$ are independent $N(0,5)$ variables. The coefficient vectors $(1,2)$ and $(2,-1)$ are orthogonal, matching the zero covariance calculation.

Worked Example: Zero Covariance Without Independence

Let $Z\sim N(0,1)$, and define $W=Z^2-1$. Verify that $Z$ and $W$ have zero covariance, then show they are dependent.

Since $E[Z]=0$ and $E[W]=E[Z^2]-1=0$,

\operatorname{Cov}(Z,W) =E[Z(Z^2-1)]-E[Z]E[W] =E[Z^3]-E[Z] =0.

Here $E[Z^3]=0$ by symmetry of the standard normal distribution. Nevertheless, $W$ is determined exactly by $Z$, so the two variables are not independent. More explicitly, let $A=\{|Z|<1\}$ and $B=\{W<0\}$. Since $W<0$ exactly when $|Z|<1$, the events $A$ and $B$ are the same. Their probability $p=P(|Z|<1)$ satisfies $0<p<1$, so $P(A\cap B)=p$, whereas independence would require $P(A)P(B)=p^2$. Since $p\ne p^2$, independence fails. The pair is not jointly Gaussian, so the Gaussian criterion does not apply.

Degeneracy and Scope of the Criterion

The theorem does not require $\Sigma_X$, $\Sigma_Y$, or the full covariance matrix to be invertible. For example, if $X$ is any Gaussian vector and $Y$ is the constant zero vector, then $\operatorname{Cov}(X,Y)=0$. A constant random vector is independent of every random vector, and the joint Gaussian law is degenerate. The characteristic-function proof remains valid without modification.

It is also important to distinguish jointly Gaussian from “each block is Gaussian.” Marginal Gaussian distributions alone do not guarantee a jointly Gaussian pair. The counterexample above has $Z$ Gaussian, but $W=Z^2-1$ is not Gaussian and the pair is dependent despite zero covariance. For a multivariate Gaussian vector, every linear combination of its coordinates is Gaussian; a nonlinear transformation such as squaring need not preserve joint Gaussianity.

Common Mistake

Wrong reasoning: “The cross-covariance is zero, so the blocks are independent.” This conclusion does not follow for arbitrary random vectors; it confuses lack of linear association with independence.
Correction: First establish joint Gaussianity of the concatenated vector. Then zero cross-covariance implies independence. Without joint Gaussianity, use a separate argument about the joint distribution; the example $W=Z^2-1$ shows that zero covariance alone is insufficient.
Another distinction: Independence does imply zero cross-covariance when the second moments exist, whether or not the variables are Gaussian. The special Gaussian result is the converse direction.

Key takeaway: For jointly Gaussian blocks, $\operatorname{Cov}(X,Y)=0$ is exactly the independence criterion. The Gaussian assumption turns a second-moment condition into a statement about the full joint distribution.

Check Your Understanding

Use the distinction between joint Gaussianity, covariance, and independence in your answers.

For jointly Gaussian vectors $X$ and $Y$, what does the equation $\operatorname{Cov}(X,Y)=0$ say about the entries of the cross-covariance matrix?
Why does the characteristic function factor when the cross-covariance block is zero?
Does the Gaussian independence criterion require nonsingular covariance matrices? Explain briefly.
Give the covariance calculation showing that $Z$ and $Z^2-1$ are uncorrelated when $Z\sim N(0,1)$.
State which direction of the equivalence between independence and zero cross-covariance requires joint Gaussianity.