Precision-Matrix Blocks and Conditional Independence

Why Precision-Matrix Blocks Matter

A covariance matrix describes how coordinates vary together before conditioning. But after some coordinates are observed, associations can change: two subvectors may be dependent marginally yet independent given the rest. For a nonsingular Gaussian vector, the precision matrix provides a direct way to identify this conditional structure. The key is to inspect the block connecting the two subvectors, not the corresponding block of the covariance matrix.

Definition: Let $X$ be a $p$-dimensional random vector with mean $\mu$ and positive definite covariance matrix $\Sigma$. Its precision matrix is $\Omega=\Sigma^{-1}$. For disjoint coordinate sets $A$, $B$, and $C$ that together contain all coordinates, $X_A$ and $X_B$ are conditionally independent given $X_C$, written $X_A\perp X_B\mid X_C$, if their conditional joint distribution factors into the product of their conditional marginal distributions for almost every value of $X_C$.

Here $X_A$ denotes the subvector whose indices belong to $A$, and likewise for $B$ and $C$. The sets $A$ and $B$ may each contain several coordinates. Write the precision matrix in the corresponding coordinate order as

\Omega= \begin{pmatrix} \Omega_{AA} & \Omega_{AB} & \Omega_{AC}\\ \Omega_{BA} & \Omega_{BB} & \Omega_{BC}\\ \Omega_{CA} & \Omega_{CB} & \Omega_{CC} \end{pmatrix}.

The block $\Omega_{AB}$ has one row for each coordinate in $A$ and one column for each coordinate in $B$. Saying this block is zero means that every entry in it is zero. The central result is an equivalence: for a nonsingular Gaussian vector, this happens exactly when the two subvectors are conditionally independent given the remaining coordinates.

Deriving the Conditional Density Kernel

Let $X\sim N(\mu,\Sigma)$, with $\Sigma$ positive definite, and set $Z=(X_A^T,X_B^T)^T$. The joint Gaussian density is proportional to

\exp\left\{-\frac12(x-\mu)^T\Omega(x-\mu)\right\}.

To obtain the conditional density of $Z$ for a fixed $X_C=c$, hold $x_C=c$ constant and collect the terms involving $z$. Put $y_Z=z-\mu_Z$, $y_C=c-\mu_C$, and let $Q=\Omega_{ZZ}$, the principal block indexed by $A\cup B$. The exponent terms that depend on $z$ are

-\frac12 y_Z^TQy_Z-y_Z^T\Omega_{ZC}y_C.

The omitted terms depend only on $c$, so they become part of the conditional normalizing constant. Because $\Omega$ is positive definite, its principal block $Q$ is positive definite too. Completing the square therefore gives a Gaussian conditional distribution with covariance $Q^{-1}$ and mean $\mu_Z-Q^{-1}\Omega_{ZC}(c-\mu_C)$. In particular, its precision matrix is $Q$. This derivation also shows why the relevant precision block is the $A\cup B$ principal block of the joint precision matrix.

The Zero-Block Characterization

Theorem: Suppose $X\sim N(\mu,\Sigma)$ with $\Sigma$ positive definite, and partition its coordinates into disjoint blocks $A$, $B$, and $C$. Then $$ X_A\perp X_B\mid X_C \quad\Longleftrightarrow\quad \Omega_{AB}=0, $$ where $\Omega=\Sigma^{-1}$. The statement includes the case that $C$ is empty, in which case it reduces to ordinary independence of $X_A$ and $X_B$.

Proof. Use the conditional Gaussian distribution derived above. Its covariance for $Z=(X_A^T,X_B^T)^T$, given $X_C=c$, is $Q^{-1}$, where

Q= \begin{pmatrix} \Omega_{AA} & \Omega_{AB}\\ \Omega_{BA} & \Omega_{BB} \end{pmatrix}.

This conditional covariance does not depend on $c$. If $\Omega_{AB}=0$, symmetry gives $\Omega_{BA}=0$, so $Q$ is block diagonal. Its inverse $Q^{-1}$ is then block diagonal as well. Thus the conditional cross-covariance of $X_A$ and $X_B$ is zero. The conditional distribution is jointly Gaussian, and a jointly Gaussian vector with zero cross-covariance between two blocks has independent blocks. Hence $X_A\perp X_B\mid X_C$.

For the converse, suppose $X_A\perp X_B\mid X_C$. Conditional independence implies zero conditional cross-covariance: the conditional expectation of the product of centered components factors into the product of their conditional expectations, which are both zero. All these moments exist because Gaussian distributions have finite second moments. Consequently, the off-diagonal block of $Q^{-1}$, the conditional covariance, is zero. Since $Q^{-1}$ is symmetric, it is block diagonal; its inverse $Q$ is also block diagonal. Therefore $\Omega_{AB}=0$, as required. The conditional distributions and covariances used here are valid for almost every $c$, which is sufficient for conditional independence. $\square$

The theorem is specifically about the precision matrix. A zero block in the covariance matrix instead implies marginal independence for Gaussian subvectors, without conditioning. The two statements answer different questions.

Worked Example: Conditional Independence Despite Marginal Dependence

Let $X=(X_1,X_2,X_3)^T\sim N(0,\Sigma)$, where its precision matrix is

\Omega= \begin{pmatrix} 2&0&0.5\\ 0&3&-0.4\\ 0.5&-0.4&2 \end{pmatrix}.

First check that this specifies a nonsingular Gaussian distribution. The leading principal minors are $2$, $6$, and \[ \det(\Omega)=2(6-0.16)+0.5(0-1.5)=11.68-0.75=10.93. \] All are positive, so Sylvester's criterion gives positive definiteness. Take $A=\{1\}$, $B=\{2\}$, and $C=\{3\}$. The precision block between $A$ and $B$ is $\Omega_{12}=0$; therefore $X_1\perp X_2\mid X_3$.

This does not mean $X_1$ and $X_2$ are marginally independent. The $(1,2)$ entry of $\Sigma=\Omega^{-1}$ is the $(2,1)$ cofactor of $\Omega$, divided by its determinant. That cofactor is \[ -\det\begin{pmatrix}0&0.5\\-0.4&2\end{pmatrix} =-0.2, \] so $\operatorname{Cov}(X_1,X_2)=-0.2/10.93\ne 0$. The coordinates are conditionally independent given $X_3$, but marginally dependent.

Worked Example: A Nonzero Block Means Conditional Dependence

Suppose $X=(X_1,X_2,X_3)^T$ is zero-mean Gaussian with precision matrix

\Omega= \begin{pmatrix} 2&-1&0\\ -1&2&0\\ 0&0&1 \end{pmatrix}.

The leading principal minors are $2$, $3$, and $3$, so $\Omega$ is positive definite. For $A=\{1\}$, $B=\{2\}$, and $C=\{3\}$, the conditional precision of $(X_1,X_2)^T$ given $X_3$ is

Q= \begin{pmatrix}2&-1\\-1&2\end{pmatrix}, \qquad Q^{-1}=\frac13\begin{pmatrix}2&1\\1&2\end{pmatrix}.

The conditional covariance is therefore $1/3$, not zero. Equivalently, $\Omega_{12}=-1\ne0$. The theorem says $X_1$ and $X_2$ are not conditionally independent given $X_3$. Here $X_3$ is in fact independent of the pair, but conditioning on it does not remove the dependence within the pair.

Worked Example: A Multivariate Block Check

Let $A=\{1,2\}$, $B=\{3\}$, and $C=\{4\}$, and suppose $X\sim N(0,\Sigma)$ has precision matrix

\Omega= \begin{pmatrix} 2&0&0&0.2\\ 0&2&0&-0.1\\ 0&0&3&0.3\\ 0.2&-0.1&0.3&2 \end{pmatrix}.

This symmetric matrix is strictly diagonally dominant with positive diagonal entries: in each row the diagonal entry exceeds the sum of the absolute off-diagonal entries. Its eigenvalues are positive, so it is positive definite. The block connecting $A$ to $B$ is the two-by-one zero matrix:

\Omega_{AB}=\begin{pmatrix}0\\0\end{pmatrix}.

The conditional precision of $(X_1,X_2,X_3)^T$ given $X_4$ is $\operatorname{diag}(2,2,3)$, so the conditional covariance is $\operatorname{diag}(1/2,1/2,1/3)$. For a fixed $X_4=c$, the conditional mean is \[ -\operatorname{diag}(2,2,3)^{-1} \begin{pmatrix}0.2\\-0.1\\0.3\end{pmatrix}c = \begin{pmatrix}-0.1c\\0.05c\\-0.1c\end{pmatrix}. \] Thus $X_1$ and $X_2$ may each depend conditionally on $X_4$, and $X_3$ may also depend on it, while the block criterion still gives $X_A\perp X_B\mid X_4$. The result applies to whole subvectors, not only pairs of scalar coordinates.

Reading the Criterion Correctly

In applications, the precision matrix is often useful because its block pattern encodes conditional relationships compactly. For any chosen pair of coordinate groups $A$ and $B$, the test is: extract their block from $\Omega$; if every entry is zero, then the groups are conditionally independent given all other coordinates. The theorem does not require the blocks $\Omega_{AC}$ or $\Omega_{BC}$ to vanish.

A common mistake is to look at $\Sigma_{AB}$ and use it to decide conditional independence. That block describes marginal covariance, not conditional covariance. For a Gaussian vector, $\Sigma_{AB}=0$ implies marginal independence of $X_A$ and $X_B$, but conditional independence given $X_C$ is characterized by $\Omega_{AB}=0$. Example 1 shows why the distinction matters: conditioning can remove dependence that is present marginally.

Another mistake is to check only one entry of $\Omega_{AB}$ when either group has multiple coordinates. A single zero does not establish the result; the entire rectangular block must vanish. Conversely, a zero block is a statement about the conditional distribution of the two groups together, not a claim that either group is independent of the conditioning coordinates.

Key takeaway: For a nonsingular jointly Gaussian vector, a zero off-diagonal block in the precision matrix is equivalent to conditional independence between the corresponding coordinate subvectors given every remaining coordinate.

Check Your Understanding

Use the zero-block criterion and the assumptions stated above to answer these questions.

For a nonsingular Gaussian vector partitioned into $A$, $B$, and $C$, which precision-matrix block determines whether $X_A\perp X_B\mid X_C$?
Why does a block-diagonal conditional precision matrix imply a block-diagonal conditional covariance matrix?
Can $X_A$ and $X_B$ be conditionally independent given $X_C$ even when their marginal covariance block is nonzero? Explain.
If $A$ contains two coordinates and $B$ contains three, what must be true of $\Omega_{AB}$ for conditional independence?
In the second worked example, what feature of the conditional covariance shows that $X_1$ and $X_2$ remain conditionally dependent?

Precision-Matrix Blocks and Conditional Independence

What You'll Learn