First-Order Partial Correlations from the Precision Matrix

Why the Precision Matrix Encodes Partial Correlation

A pair of variables can be associated because they share variation with other measured variables. Partial correlation isolates the association that remains after linearly adjusting both variables for the same set of controls. When the controls are all the other variables in a multivariate system, a single entry of the inverse covariance matrix records the direction and strength of that remaining association.

The key distinction is that this is a statement about linear residuals. For jointly Gaussian variables, their residual correlation also equals their conditional correlation given the controls. Without joint normality, those quantities need not be the same, and a zero partial correlation does not by itself establish conditional independence.

Definition: Let $X=(X_1,\ldots,X_p)^\mathsf{T}$ have finite second moments and positive-definite covariance matrix $\Sigma$. For two distinct indices $i,j$, let $X_{-ij}$ denote the vector of all components other than $X_i,X_j$. Linearly regress $X_i$ and $X_j$ on $X_{-ij}$, including an intercept if the variables are not centered. The partial correlation of $X_i$ and $X_j$ given all the remaining variables, denoted $\rho_{ij\cdot -ij}$, is the correlation between the two regression residuals. If there are no remaining variables, it is the ordinary correlation.

Write the precision matrix as $\Omega=\Sigma^{-1}$, and let $\omega_{ij}$ denote its $(i,j)$ entry. The name “first-order partial correlation” is sometimes used for this pairwise quantity adjusted for all the other variables. The formula below applies to that quantity: it uses the two diagonal precision entries and the off-diagonal entry for the selected pair.

Deriving the Formula

For the derivation, reorder the variables so the pair of interest is $X_A=(X_i,X_j)^\mathsf{T}$, and write $X_B=X_{-ij}$ for the controls. Partition the covariance matrix conformably:

\Sigma = \begin{pmatrix} \Sigma_{AA} & \Sigma_{AB}\\ \Sigma_{BA} & \Sigma_{BB} \end{pmatrix}.

Positive definiteness of $\Sigma$ implies that $\Sigma_{BB}$ is invertible when controls are present. The population linear projection residual vector for the pair is $R_A=X_A-E[X_A]-\Sigma_{AB}\Sigma_{BB}^{-1}(X_B-E[X_B])$. Its covariance is the Schur complement $S=\Sigma_{AA}-\Sigma_{AB}\Sigma_{BB}^{-1}\Sigma_{BA}$. The earlier material on residual covariance from block matrix identities gives this residual-covariance form; the block calculation also shows precisely why the relevant precision entries are its inverse.

Theorem: If $\Sigma$ is positive definite and $\Omega=\Sigma^{-1}$, the partial correlation of $X_i$ and $X_j$ given all the other components is $$ \rho_{ij\cdot -ij} = -\frac{\omega_{ij}}{\sqrt{\omega_{ii}\omega_{jj}}}. $$ The formula is valid for variables with finite second moments; joint normality is not required for the residual-correlation interpretation.

Proof. In the partition above, denote the blocks of $\Omega$ by $\Omega_{AA},\Omega_{AB},\Omega_{BA},\Omega_{BB}$. The first block row of $\Sigma\Omega=I$ gives $\Sigma_{AA}\Omega_{AA}+\Sigma_{AB}\Omega_{BA}=I$. The second block row gives $\Sigma_{BA}\Omega_{AA}+\Sigma_{BB}\Omega_{BA}=0$, so $\Omega_{BA}=-\Sigma_{BB}^{-1}\Sigma_{BA}\Omega_{AA}$. Substituting this expression into the first block-row equation yields

\left(\Sigma_{AA}-\Sigma_{AB}\Sigma_{BB}^{-1}\Sigma_{BA}\right) \Omega_{AA}=I.

Thus $S\Omega_{AA}=I$, or $\Omega_{AA}=S^{-1}$. Because $S$ is a positive-definite $2\times2$ covariance matrix, write its inverse block as $\Omega_{AA}=\begin{pmatrix}a&b\\b&d\end{pmatrix}$, where $a=\omega_{ii}$, $b=\omega_{ij}$, and $d=\omega_{jj}$. Inverting this $2\times2$ matrix gives

S=\frac{1}{ad-b^2} \begin{pmatrix} d&-b\\ -b&a \end{pmatrix}.

The correlation of the two residuals is their covariance divided by the square root of the product of their variances. Substituting the entries of $S$ gives $(-b/(ad-b^2))/\sqrt{(d/(ad-b^2))(a/(ad-b^2))}=-b/\sqrt{ad}$. The determinant $ad-b^2$ is positive, so the cancellation is valid. Replacing $a,b,d$ by their precision entries proves the formula. If there are no controls, the same calculation uses $S=\Sigma$ and $\Omega_{AA}=\Sigma^{-1}$. $\square$

Worked Calculations

Worked Example: Three Variables and One Control

Suppose $X=(X_1,X_2,X_3)^\mathsf{T}$ has covariance matrix

\Sigma=\frac{1}{56} \begin{pmatrix} 15&-4&1\\ -4&16&-4\\ 1&-4&15 \end{pmatrix}, \qquad \Omega= \begin{pmatrix} 4&1&0\\ 1&4&1\\ 0&1&4 \end{pmatrix}.

The displayed matrices are inverses: multiplying the integer matrix in $\Sigma$ by $\Omega$ gives $56I$. For $X_1$ and $X_2$ adjusted for $X_3$, the required precision entries are $\omega_{12}=1$, $\omega_{11}=4$, and $\omega_{22}=4$. Therefore

\rho_{12\cdot 3} =-\frac{1}{\sqrt{4\cdot4}} =-\frac14=-0.25.

For a direct residual check, the covariance of the two regression residuals is the inverse of the selected precision block: $\begin{pmatrix}4&1\\1&4\end{pmatrix}^{-1} =\frac1{15}\begin{pmatrix}4&-1\\-1&4\end{pmatrix}$. Its diagonal entries are both $4/15$, and its off-diagonal entry is $-1/15$. Hence its correlation is $(-1/15)/(4/15)=-1/4$, as required.

Worked Example: Adjusting for Two Controls

Let $X$ have four components and precision matrix

\Omega= \begin{pmatrix} 3&0.6&0&0\\ 0.6&2&0.3&0\\ 0&0.3&2&0.4\\ 0&0&0.4&2 \end{pmatrix}.

This matrix is symmetric and strictly diagonally dominant with positive diagonal entries: the diagonal entries exceed the sums of the absolute off-diagonal entries in their rows. It is therefore positive definite, so it can be used as the precision matrix of a valid covariance matrix. For the pair $X_1,X_2$, the controls are $X_3,X_4$. The relevant entries are $\omega_{12}=0.6$, $\omega_{11}=3$, and $\omega_{22}=2$, giving

\rho_{12\cdot 34} =-\frac{0.6}{\sqrt{3\cdot2}} =-\frac{0.6}{\sqrt6} \approx -0.245.

The residual covariance block can also be checked. The selected precision block is $\begin{pmatrix}3&0.6\\0.6&2\end{pmatrix}$, whose determinant is $6-0.36=5.64$. Its inverse is $\frac1{5.64}\begin{pmatrix}2&-0.6\\-0.6&3\end{pmatrix}$. Thus the residual variances are $2/5.64$ and $3/5.64$, while the residual covariance is $-0.6/5.64$. Their correlation is $(-0.6/5.64)/\sqrt{(2/5.64)(3/5.64)}=-0.6/\sqrt6$, confirming the calculation.

Worked Example: A Zero Partial Correlation Despite Marginal Association

Let $Z,\varepsilon_1,\varepsilon_2$ be independent $N(0,1)$ variables, and set $X_1=Z+\varepsilon_1$ and $X_2=Z+\varepsilon_2$. For the vector $(X_1,X_2,Z)^\mathsf{T}$, direct covariance calculations give

\Sigma= \begin{pmatrix} 2&1&1\\ 1&2&1\\ 1&1&1 \end{pmatrix}, \qquad \Omega=\Sigma^{-1}= \begin{pmatrix} 1&0&-1\\ 0&1&-1\\ -1&-1&3 \end{pmatrix}.

Multiplying the two displayed matrices gives the identity, verifying the inverse. The marginal correlation of $X_1$ and $X_2$ is $1/\sqrt{2\cdot2}=0.5$. After adjusting for $Z$, however, the precision entry $\omega_{12}$ is zero, so $\rho_{12\cdot 3}=-0/\sqrt{1\cdot1}=0$.

The residual interpretation is transparent in the construction: regressing each $X_k$ on $Z$ leaves $\varepsilon_k$, and the two errors are independent. Since the variables are jointly Gaussian, $X_1$ and $X_2$ are also conditionally independent given $Z$ in this example. That stronger conclusion relies on joint normality; zero residual correlation alone does not imply conditional independence for an arbitrary distribution.

What the Formula Does and Does Not Say

Positive definiteness ensures $\omega_{ii}>0$ and $\omega_{jj}>0$, so the denominator is real and positive. It also ensures the selected $2\times2$ precision block is positive definite, which means $\omega_{ii}\omega_{jj}-\omega_{ij}^2>0$. Consequently, the formula always gives a partial correlation strictly between $-1$ and $1$.

It follows directly that the sign of the partial correlation is the opposite of the sign of $\omega_{ij}$, and that $\rho_{ij\cdot -ij}=0$ exactly when $\omega_{ij}=0$. These are statements about linear adjustment. In a multivariate normal model, the zero entry additionally encodes conditional independence of that pair given the remaining variables. For non-Gaussian variables, that independence interpretation is not generally justified.

A practical advantage is that one inverse covariance matrix provides the adjusted association for every pair: each pair uses its own off-diagonal entry and the two corresponding diagonal entries. This can be especially useful when there are many variables, but the formula does not remove the need for a reliable covariance estimate. If the covariance matrix is singular, its ordinary inverse does not exist; if it is nearly singular, estimated precision entries and resulting partial correlations can be unstable.

Common pitfall: Using $+\omega_{ij}/\sqrt{\omega_{ii}\omega_{jj}}$ reverses the sign. The minus sign comes from the off-diagonal entry of the inverse of the $2\times2$ residual covariance matrix. Another error is to treat $\omega_{ii}$ as a residual variance: it is a diagonal entry of the inverse residual covariance block. The residual variances are the diagonal entries of that block’s inverse.

Check Your Understanding

Use the theorem and its assumptions to answer the following questions.

For a given pair, which three entries of the precision matrix determine its partial correlation after adjusting for all other variables?
If $\omega_{ij}$ is positive, what is the sign of the corresponding partial correlation? Why?
For a selected precision block $\begin{pmatrix}9&-3\\-3&4\end{pmatrix}$, calculate the partial correlation and show the arithmetic.
What positive-definiteness condition ensures that the formula produces a partial correlation whose absolute value is less than one?
Why does a zero precision off-diagonal entry not always establish conditional independence?

First-Order Partial Correlations from the Precision Matrix

What You'll Learn