Precision-Matrix Zeros And Gaussian Conditional Independence

Why Precision Zeros Describe Conditional Relationships

In a multivariate Gaussian model, the covariance matrix describes marginal variation and association. The precision matrix—the inverse covariance matrix—has a different interpretation: its off-diagonal entries describe the association between two variables after the other variables have been held fixed. This distinction is useful when the scientific question concerns relationships that remain after adjustment for all other measured variables.

The precise statement depends on the Gaussian assumption and on nonsingularity. Under those conditions, an off-diagonal precision entry is zero exactly when the corresponding pair is conditionally independent given the rest. The proof comes from the conditional Gaussian density, not from treating a zero covariance as a general test for independence.

Definition: Let $X=(X_1,\ldots,X_p)^T$ have a nonsingular multivariate Gaussian distribution, with mean vector $\mu$, covariance matrix $\Sigma$, and precision matrix $\Omega=\Sigma^{-1}$. For distinct indices $i,j$, write $X_{-(i,j)}$ for the vector of all coordinates except $X_i,X_j$. The variables $X_i$ and $X_j$ are conditionally independent given the remaining variables if, for almost every value $z$ of $X_{-(i,j)}$, the conditional joint distribution of $(X_i,X_j)$ given $X_{-(i,j)}=z$ is the product of its two conditional marginal distributions.

When $p=2$, there are no remaining variables, so this definition reduces to ordinary independence. For $p>2$, the conditioning set is specifically all the other coordinates—not an arbitrary subset. Since $\Sigma$ is nonsingular, the Gaussian distribution has a density and its conditional distributions are well-defined Gaussian distributions.

The Conditional Gaussian Distribution in Precision Form

Fix the pair $A=(i,j)$, and let $B$ denote the indices of the remaining coordinates. Partition vectors and matrices according to $A$ and $B$. In particular, write $\Omega_{AA}$ for the $2\times2$ principal block of the precision matrix and $\Omega_{AB}$ for the block connecting the pair to the remaining coordinates.

Conditional distribution result: For a nonsingular multivariate Gaussian vector, the conditional distribution of $X_A$ given $X_B=b$ is Gaussian with mean and covariance $$ E[X_A\mid X_B=b] =\mu_A-\Omega_{AA}^{-1}\Omega_{AB}(b-\mu_B), \qquad \operatorname{Cov}(X_A\mid X_B=b)=\Omega_{AA}^{-1}. $$ If $B$ is empty, the conditional distribution here means the marginal distribution of $X_A$.

To derive the result when $B$ is nonempty, use the Gaussian density in precision form. For a fixed $b$, its terms that depend on $x_A$ are proportional to $$ \exp\!\left\{-\frac12 \begin{pmatrix}x_A-\mu_A\\ b-\mu_B\end{pmatrix}^{T} \begin{pmatrix}\Omega_{AA}&\Omega_{AB}\\ \Omega_{BA}&\Omega_{BB}\end{pmatrix} \begin{pmatrix}x_A-\mu_A\\ b-\mu_B\end{pmatrix}\right\}. $$ As a function of $x_A$, the exponent’s quadratic and linear terms are $$ -\frac12(x_A-\mu_A)^T\Omega_{AA}(x_A-\mu_A) -(x_A-\mu_A)^T\Omega_{AB}(b-\mu_B). $$ The matrix $\Omega_{AA}$ is positive definite: for any nonzero two-vector $v$, extend $v$ with zeros in the coordinates in $B$; positive definiteness of $\Omega$ then gives $v^T\Omega_{AA}v>0$. Thus it is invertible, and completing the square gives $$ -\frac12\left(x_A-\mu_A+\Omega_{AA}^{-1}\Omega_{AB}(b-\mu_B)\right)^T \Omega_{AA} \left(x_A-\mu_A+\Omega_{AA}^{-1}\Omega_{AB}(b-\mu_B)\right) +\text{a term independent of }x_A. $$ After normalization as a density in $x_A$, this is a Gaussian density with the stated mean and covariance $\Omega_{AA}^{-1}$. The same conclusion for empty $B$ follows directly by taking the marginal pair $X_A$; its precision matrix is then $\Omega_{AA}=\Omega$. $\square$

The Zero-Entry Equivalence

Theorem: If $X$ has a nonsingular multivariate Gaussian distribution with precision matrix $\Omega$, then, for every distinct pair $i,j$, $$ X_i\ \text{and}\ X_j\ \text{are conditionally independent given }X_{-(i,j)} \quad\Longleftrightarrow\quad \Omega_{ij}=0. $$

Proof. Set $A=(i,j)$, and write the conditional precision block as $$ \Omega_{AA}= \begin{pmatrix} \omega_{ii}&\omega_{ij}\\ \omega_{ij}&\omega_{jj} \end{pmatrix}. $$ It is positive definite by the argument above, so its determinant $\Delta=\omega_{ii}\omega_{jj}-\omega_{ij}^2$ is positive. The conditional covariance matrix of the pair is its inverse: $$ \operatorname{Cov}(X_A\mid X_B) =\Omega_{AA}^{-1} =\frac{1}{\Delta} \begin{pmatrix} \omega_{jj}&-\omega_{ij}\\ -\omega_{ij}&\omega_{ii} \end{pmatrix}. $$ The conditional covariance between the two variables is therefore $-\omega_{ij}/\Delta$, which is zero exactly when $\omega_{ij}=0$.

For a bivariate Gaussian distribution, independence is equivalent to zero covariance. Indeed, independence and finite second moments imply zero covariance. Conversely, if the covariance is zero, its covariance matrix is diagonal, and the bivariate Gaussian density factors into the product of the two univariate Gaussian densities. Apply this fact to the conditional bivariate Gaussian distribution for each fixed value of $X_B$. Its covariance does not depend on that value, so the conditional variables are independent for almost every conditioning value exactly when $-\omega_{ij}/\Delta=0$, or equivalently $\Omega_{ij}=0$. If $B$ is empty, the same argument applies to the marginal bivariate Gaussian. $\square$

The calculation also gives the conditional correlation: $$ \operatorname{Corr}(X_i,X_j\mid X_{-(i,j)}) =-\frac{\omega_{ij}}{\sqrt{\omega_{ii}\omega_{jj}}}. $$ To see this, divide the conditional covariance $-\omega_{ij}/\Delta$ by the square root of the two conditional variances, $\sqrt{(\omega_{jj}/\Delta)(\omega_{ii}/\Delta)}$. The positive determinant cancels. Thus the precision entry determines both whether the conditional association is zero and, after scaling by the diagonal entries, its conditional correlation.

Worked Examples

Worked Example: Zero Precision Entry but Nonzero Marginal Association

Let $X\sim N(0,\Sigma)$ be a three-dimensional Gaussian vector with precision matrix $$ \Omega= \begin{pmatrix} 2&0&-1\\ 0&2&-1\\ -1&-1&3 \end{pmatrix}. $$ The leading principal minors are $2$, $4$, and $8$, all positive, so $\Omega$ is positive definite and defines a nonsingular Gaussian distribution.

For the pair $(X_1,X_2)$ given $X_3$, the relevant blocks are $$ \Omega_{AA}= \begin{pmatrix}2&0\\0&2\end{pmatrix}, \qquad \Omega_{AB}= \begin{pmatrix}-1\\-1\end{pmatrix}. $$ Consequently, the conditional covariance is $$ \Omega_{AA}^{-1}= \begin{pmatrix}1/2&0\\0&1/2\end{pmatrix}, $$ and the conditional mean given $X_3=x_3$ is $$ -\Omega_{AA}^{-1}\Omega_{AB}x_3 = -\begin{pmatrix}1/2&0\\0&1/2\end{pmatrix} \begin{pmatrix}-1\\-1\end{pmatrix}x_3 = \begin{pmatrix}x_3/2\\x_3/2\end{pmatrix}. $$ The conditional covariance is diagonal, so $X_1$ and $X_2$ are conditionally independent given $X_3$.

They are nevertheless marginally correlated. The determinant of $\Omega$ is $8$, and the $(1,2)$ entry of $\Omega^{-1}$ is the cofactor $C_{2,1}$ divided by that determinant. The minor for $C_{2,1}$ is $$ \begin{vmatrix}0&-1\\-1&3\end{vmatrix}=-1, $$ and its cofactor sign is $(-1)^{2+1}=-1$, so $C_{2,1}=1$. Therefore $\operatorname{Cov}(X_1,X_2)=\Sigma_{12}=1/8$, which is nonzero. Conditioning on $X_3$ removes the association that is present marginally.

Worked Example: Nonzero Precision Entry and Conditional Dependence

Consider $X\sim N(0,\Sigma)$ in three dimensions with $$ \Omega= \begin{pmatrix} 2&1&0\\ 1&2&0\\ 0&0&1 \end{pmatrix}. $$ This matrix is positive definite: its eigenvalues are $1,1,3$. For $X_1,X_2$ given $X_3$, the relevant precision block is $$ \Omega_{AA}= \begin{pmatrix}2&1\\1&2\end{pmatrix}. $$ Its determinant is $4-1=3$, so the conditional covariance is $$ \Omega_{AA}^{-1} =\frac13\begin{pmatrix}2&-1\\-1&2\end{pmatrix}. $$ The conditional correlation is $$ \frac{-1/3}{\sqrt{(2/3)(2/3)}}=-\frac12. $$ It is nonzero, so $X_1$ and $X_2$ are conditionally dependent given $X_3$. This agrees with $\Omega_{12}=1\ne0$. In this example, $X_3$ is independent of the pair, but conditioning on it does not change the pair’s dependence.

Worked Example: Conditioning on More Than One Remaining Variable

Let $X\sim N(0,\Sigma)$ have four-dimensional precision matrix $$ \Omega= \begin{pmatrix} 2&0&-1&0\\ 0&2&0&-1\\ -1&0&2&0\\ 0&-1&0&2 \end{pmatrix}. $$ After reordering coordinates as $(1,3,2,4)$, this consists of two blocks $\begin{pmatrix}2&-1\\-1&2\end{pmatrix}$, each with eigenvalues $1$ and $3$. Hence $\Omega$ is positive definite. Consider $X_1,X_2$ conditional on both $X_3=x_3$ and $X_4=x_4$. The blocks are $$ \Omega_{AA}=\begin{pmatrix}2&0\\0&2\end{pmatrix}, \qquad \Omega_{AB}=\begin{pmatrix}-1&0\\0&-1\end{pmatrix}. $$ Thus the conditional mean is $$ -\Omega_{AA}^{-1}\Omega_{AB} \begin{pmatrix}x_3\\x_4\end{pmatrix} = \begin{pmatrix}x_3/2\\x_4/2\end{pmatrix}, $$ and the conditional covariance is $\operatorname{diag}(1/2,1/2)$. The conditional joint distribution is therefore the product of $N(x_3/2,1/2)$ and $N(x_4/2,1/2)$. The calculation conditions on every coordinate outside the pair, and $\Omega_{12}=0$ confirms conditional independence.

Scope, Interpretation, and a Common Pitfall

The equivalence is an exact population statement. It requires a jointly Gaussian distribution with a nonsingular covariance matrix. It does not say that a small estimated precision entry proves conditional independence in a finite sample; estimation uncertainty and any regularization used to obtain an estimate are separate issues. Nor does the result automatically extend to non-Gaussian distributions: zero partial covariance alone need not imply conditional independence there.

Common mistake: reading a precision zero as marginal independence. In the first example, $\Omega_{12}=0$, but $\Sigma_{12}=1/8$. The correction is to read this zero as conditional independence given the remaining variables, not as a claim about the marginal pair.
Common mistake: conditioning on only some of the other variables. The theorem concerns conditioning on all coordinates other than $i,j$. A zero entry does not, by itself, establish independence after conditioning on a different subset.
Common mistake: using the result without nonsingularity. The inverse covariance matrix and the conditional-density argument used here require $\Sigma$ to be positive definite. If $\Sigma$ is singular, this theorem does not apply in the stated form.

The practical value of the result is that a matrix of conditional relationships can be studied through the sparsity pattern of $\Omega$: zeros encode pairwise conditional independences under the Gaussian model. The interpretation is precise, but its scope is equally precise: Gaussianity, nonsingularity, and conditioning on the full complement of the pair are essential.

Check Your Understanding

Use the conditional Gaussian distribution and the zero-entry equivalence to answer these questions.

For a pair $X_i,X_j$, which block of the precision matrix determines their conditional covariance given all other coordinates?
Why does a zero conditional covariance imply conditional independence in this setting?
In the first worked example, how can $\Omega_{12}=0$ coexist with $\Sigma_{12}\ne0$?
What is the conditional correlation in terms of $\omega_{ij},\omega_{ii},\omega_{jj}$?
Does the theorem establish conditional independence given an arbitrary subset of the remaining variables? Explain.