Block Marginals and Independence in Gaussian Vectors

From a Joint Gaussian Model to Its Blocks

A multivariate Gaussian model can describe several measurements together, but many questions concern only a subset of them. For instance, a researcher may record several outcomes but analyze just two. The corresponding subvector still has a Gaussian distribution; its mean and covariance are obtained by retaining the matching entries and covariance rows and columns. Dependence between two such blocks is captured by the cross-covariance block.

The key feature of Gaussian vectors is that their covariance structure does more than summarize linear association: it determines the full joint distribution. Consequently, for two disjoint coordinate subvectors, zero cross-covariance is not merely evidence of weak association. It is exactly the condition for independence.

Definition: A random vector \(X\in\mathbb{R}^p\) is jointly Gaussian if every linear combination \(a'X\), for \(a\in\mathbb{R}^p\), has a univariate normal distribution, allowing a degenerate normal distribution with variance zero. Write \(X\sim N_p(\mu,\Sigma)\), where \(\mu=E[X]\) and \(\Sigma=\operatorname{Cov}(X)\). The matrix \(\Sigma\) is symmetric and positive semidefinite.

Partition the coordinates into two disjoint groups, with sizes \(r\) and \(s\), and write \(X=(X_A',X_B')'\). The corresponding partitions of the mean and covariance are

\mu= \begin{pmatrix} \mu_A\\ \mu_B \end{pmatrix}, \qquad \Sigma= \begin{pmatrix} \Sigma_{AA}&\Sigma_{AB}\\ \Sigma_{BA}&\Sigma_{BB} \end{pmatrix}.

Here \(\Sigma_{AA}=\operatorname{Cov}(X_A)\), \(\Sigma_{BB}=\operatorname{Cov}(X_B)\), and \(\Sigma_{AB}=\operatorname{Cov}(X_A,X_B)=E[(X_A-\mu_A)(X_B-\mu_B)']\). In particular, \(\Sigma_{BA}=\Sigma_{AB}'\). A subvector formed by selecting coordinates has a covariance matrix given by the corresponding principal submatrix of \(\Sigma\).

Worked Example: Finding a Two-Coordinate Marginal

Suppose \(X\sim N_3(\mu,\Sigma)\), where

\mu= \begin{pmatrix}1\\-2\\3\end{pmatrix}, \qquad \Sigma= \begin{pmatrix} 4&1&2\\ 1&9&-1\\ 2&-1&5 \end{pmatrix}.

First check that the proposed covariance matrix is valid. Its leading principal minors are \(4\), \(4(9)-1=35\), and

\det(\Sigma) =4(45-1)-1(5-(-2))+2((-1)-18) =176-7-38 =131.

All three are positive, so Sylvester's criterion shows that \(\Sigma\) is positive definite. Select coordinates 1 and 3, so \(X_A=(X_1,X_3)'\). Retaining their entries from the mean and their rows and columns from the covariance gives

X_A\sim N_2\left( \begin{pmatrix}1\\3\end{pmatrix}, \begin{pmatrix}4&2\\2&5\end{pmatrix} \right).

For a check on the covariance entries, \(\operatorname{Var}(X_1)=4\), \(\operatorname{Var}(X_3)=5\), and \(\operatorname{Cov}(X_1,X_3)=2\). Thus \(\operatorname{Var}(X_1-X_3)=4+5-2(2)=5\), as the displayed marginal covariance matrix also implies.

Why Coordinate Marginals Remain Gaussian

The marginal result follows directly from the definition of a jointly Gaussian vector. A linear combination of the selected coordinates is also a linear combination of all the original coordinates, with zero coefficients assigned to the coordinates that were not selected. This argument covers both nonsingular and singular Gaussian vectors.

Theorem: If \(X\sim N_p(\mu,\Sigma)\) and \(X_A\) selects \(r\) coordinates of \(X\), then \(X_A\sim N_r(\mu_A,\Sigma_{AA})\), where \(\mu_A\) and \(\Sigma_{AA}\) are the corresponding entries and principal submatrix.

Proof. Let \(C\) be the \(r\times p\) coordinate-selection matrix, so \(X_A=CX\). For any \(t\in\mathbb{R}^r\), the scalar \(t'X_A=(C't)'X\) is normal by joint Gaussianity. Its expectation is \(t'C\mu\) and its variance is \(t'C\Sigma C't\). Therefore \(X_A\) is jointly Gaussian, with mean \(C\mu=\mu_A\) and covariance \(C\Sigma C'=\Sigma_{AA}\). This proves the stated marginal law. \(\square\)

Equivalently, the characteristic function of \(X\) is

\varphi_X(t)=E[e^{it'X}] =\exp\left(it'\mu-\frac12t'\Sigma t\right), \qquad t\in\mathbb{R}^p.

Setting the coordinates of \(t\) outside \(A\) to zero gives the characteristic function of \(X_A\), with mean \(\mu_A\) and covariance \(\Sigma_{AA}\). Characteristic functions determine distributions, including in the singular case.

Zero Cross-Covariance and Independence

For arbitrary random vectors, independence implies zero cross-covariance whenever the relevant second moments exist. The reverse implication generally fails. For jointly Gaussian subvectors, however, the joint characteristic function separates into one factor for each block exactly when the cross-covariance block is zero.

Theorem: Let \(X=(X_A',X_B')'\) be jointly Gaussian, with finite covariance matrix partitioned as above. Then \(X_A\) and \(X_B\) are independent if and only if \(\Sigma_{AB}=0\).

Proof. For \(u\in\mathbb{R}^r\) and \(v\in\mathbb{R}^s\), the joint characteristic function is

\varphi_{X_A,X_B}(u,v) = \exp\left\{ i u'\mu_A+i v'\mu_B -\frac12\left( u'\Sigma_{AA}u+2u'\Sigma_{AB}v+v'\Sigma_{BB}v \right) \right\}.

If \(\Sigma_{AB}=0\), the term \(2u'\Sigma_{AB}v\) vanishes. The resulting expression is the product of the marginal characteristic functions:

\varphi_{X_A,X_B}(u,v) = \exp\left(iu'\mu_A-\tfrac12u'\Sigma_{AA}u\right) \exp\left(iv'\mu_B-\tfrac12v'\Sigma_{BB}v\right) = \varphi_{X_A}(u)\varphi_{X_B}(v).

The characteristic-function criterion for independence says that a joint characteristic function factors into the marginal characteristic functions for all arguments if and only if the random vectors are independent. Hence \(X_A\) and \(X_B\) are independent.

Conversely, if \(X_A\) and \(X_B\) are independent, then for every \(u,v\), their joint characteristic function factors into the marginal functions. Comparing this product with the displayed joint Gaussian characteristic function gives

\exp(-u'\Sigma_{AB}v)=1 \qquad\text{for all }u,v.

The exponent is real, so this equality requires \(u'\Sigma_{AB}v=0\) for every \(u,v\). Taking \(u\) and \(v\) to be coordinate unit vectors shows that every entry of \(\Sigma_{AB}\) is zero. Thus \(\Sigma_{AB}=0\), proving the converse. \(\square\)

This proof does not require \(\Sigma\) to be invertible. That matters because a Gaussian vector can be concentrated on a lower-dimensional set, in which case it has no ordinary density with respect to full-dimensional Lebesgue measure. The characteristic-function argument still applies.

Worked Example: Verifying Independence of Two Gaussian Blocks

Let \(X_A=(X_1,X_2)'\) and \(X_B=(X_3,X_4)'\) be the two blocks of a jointly Gaussian vector with mean and covariance

\mu= \begin{pmatrix}0\\1\\-1\\2\end{pmatrix}, \qquad \Sigma= \begin{pmatrix} 1&0.6&0&0\\ 0.6&2&0&0\\ 0&0&3&-1\\ 0&0&-1&2 \end{pmatrix}.

The cross-covariance block is the upper-right \(2\times2\) block, which is the zero matrix. To verify that this is a valid, nondegenerate Gaussian covariance, its two diagonal blocks have leading principal minors \(1, 2-0.6^2=1.64\) and \(3, 6-1=5\), respectively. Both blocks are positive definite, so their block-diagonal combination is positive definite. The theorem therefore gives \(X_A\perp X_B\). Their respective marginal laws are

X_A\sim N_2\left( \begin{pmatrix}0\\1\end{pmatrix}, \begin{pmatrix}1&0.6\\0.6&2\end{pmatrix} \right), \qquad X_B\sim N_2\left( \begin{pmatrix}-1\\2\end{pmatrix}, \begin{pmatrix}3&-1\\-1&2\end{pmatrix} \right).

Each block can have dependence within itself: here \(\operatorname{Cov}(X_1,X_2)=0.6\) and \(\operatorname{Cov}(X_3,X_4)=-1\). Independence concerns the relationship between the two blocks, not the relationships among coordinates within either block.

Worked Example: Nonzero Cross-Covariance Rules Out Independence

Suppose \((X_1,X_2)'\) is jointly Gaussian with mean \((2,-1)'\) and covariance

\Sigma= \begin{pmatrix}1&0.5\\0.5&1\end{pmatrix}.

The determinant is \(1-0.5^2=0.75>0\), and the first diagonal entry is positive, so \(\Sigma\) is positive definite. The cross-covariance between the two scalar blocks is \(0.5\), which is not zero. Therefore \(X_1\) and \(X_2\) are not independent. This conclusion also follows from the general fact that independent variables with finite second moments have covariance zero.

Why the Gaussian Assumption Matters

The equivalence in the theorem depends on joint Gaussianity, not just on each coordinate having a Gaussian marginal distribution. In general, zero covariance only says that the variables are uncorrelated; it does not rule out nonlinear dependence.

Worked Example: Zero Covariance Without Independence

Let \(X\) be uniformly distributed on \((-1,1)\), and set \(Y=X^2\). By symmetry, \(E[X]=0\) and \(E[X^3]=0\), while \(E[X^2]=1/3\). Hence

\operatorname{Cov}(X,Y) =E[X^3]-E[X]E[X^2] =0-0\left(\frac13\right) =0.

Nevertheless, \(Y\) is determined by \(X\), and the pair is not independent. For a direct check, set \(a=1/\sqrt{2}\), \(A=\{X>a\}\), and \(B=\{Y>1/2\}\). Since \(Y=X^2\), \(A\) is contained in \(B\), and

P(A)=\frac{1-a}{2},\qquad P(B)=1-a,\qquad P(A\cap B)=\frac{1-a}{2}.

Because \(0<1-a<1\), \(P(A\cap B)\ne P(A)P(B)\), so \(X\) and \(Y\) are dependent. There is no contradiction with the Gaussian theorem: this pair is not jointly Gaussian.

Common Mistake

Wrong reasoning: “The cross-covariance is zero, so the blocks must be independent.” This inference is not valid for arbitrary distributions; the example \(Y=X^2\) shows why.
Correction: First establish that the subvectors are jointly Gaussian. Then zero cross-covariance is equivalent to independence. Also check the entire cross-covariance block: a single zero entry does not establish independence if other entries in the block are nonzero.
Another error: Treating the covariance submatrix for a marginal as an inverse or as a conditional covariance. A marginal covariance is obtained by selecting the relevant rows and columns. Conditional distributions are a separate question.

Key takeaway: A coordinate subvector of \(N_p(\mu,\Sigma)\) is Gaussian with the corresponding entries of \(\mu\) and principal submatrix of \(\Sigma\). For two disjoint subvectors of a jointly Gaussian vector, independence holds exactly when their entire cross-covariance block is zero.

Check Your Understanding

Try these questions using the definitions and results above.

If \(X\sim N_4(\mu,\Sigma)\), what parts of \(\mu\) and \(\Sigma\) determine the distribution of \((X_2,X_4)'\)?
Why does the zero-cross-covariance criterion apply even when the joint Gaussian covariance matrix is singular?
For jointly Gaussian blocks \(X_A\) and \(X_B\), what does independence imply about every entry of \(\Sigma_{AB}\)?
Can two individually Gaussian random variables with zero covariance be assumed independent without knowing that they are jointly Gaussian? Explain.
In the four-dimensional worked example, are \(X_1\) and \(X_2\) independent? What covariance entry supports your answer?