Precision Matrices and Partial Correlations

Why Precision Matrices Encode Conditional Association

In a multi-omics data set, two features can be correlated because each is associated with other measured features. Their marginal correlation alone does not distinguish a direct conditional association from one that disappears after accounting for the remaining features. Under a multivariate Gaussian model, the precision matrix provides a compact way to express these conditional relationships.

The earlier material on Gaussian Graphical Models for Continuous Omics introduces the Gaussian model for continuous measurements. Here the focus is the precise link between its precision matrix and partial correlations: how to derive the formula, calculate it, and interpret it without confusing conditional association with causation.

Definition: Let \(X=(X_1,\ldots,X_p)^T\) be a random vector with a nonsingular covariance matrix \(\Sigma\). Its precision matrix is \(\Omega=\Sigma^{-1}\). For distinct features \(i\) and \(j\), the partial correlation \(\rho_{ij\mid -\{i,j\}}\) is the correlation between \(X_i\) and \(X_j\) conditional on all the other features \(X_{-\{i,j\}}\). In a nondegenerate multivariate Gaussian model, \(\rho_{ij\mid -\{i,j\}}=-\Omega_{ij}/\sqrt{\Omega_{ii}\Omega_{jj}}\).

The formula concerns population quantities. It assumes \(X\sim N_p(\mu,\Sigma)\), where \(\Sigma\) is positive definite, so that \(\Omega\) exists and its diagonal entries are positive. The conditioning set is all measured features other than \(i\) and \(j\). With only two features, that set is empty and the partial correlation is the ordinary correlation. The formula is not, by itself, a method for establishing statistical significance or causal effects.

Deriving the Formula

Let \(A=\{i,j\}\) and let \(B\) index the remaining features. Partition \(X\) into \(X_A\) and \(X_B\), and partition \(\Omega\) into matching blocks. In the Gaussian density, the part of the quadratic form involving \(X_A\), conditional on a fixed value of \(X_B\), has quadratic term \(X_A^T\Omega_{AA}X_A\). Terms involving the fixed \(X_B\) affect the conditional mean but not this quadratic coefficient. Therefore the conditional distribution of \(X_A\) given \(X_B\) has precision matrix \(\Omega_{AA}\), and conditional covariance matrix \(\Omega_{AA}^{-1}\).

\Omega_{AA}= \begin{pmatrix} \Omega_{ii}&\Omega_{ij}\\ \Omega_{ij}&\Omega_{jj} \end{pmatrix}, \qquad \Omega_{AA}^{-1} = \frac{1}{\Omega_{ii}\Omega_{jj}-\Omega_{ij}^2} \begin{pmatrix} \Omega_{jj}&-\Omega_{ij}\\ -\Omega_{ij}&\Omega_{ii} \end{pmatrix}.

The covariance in this conditional distribution is the off-diagonal entry of \(\Omega_{AA}^{-1}\); its two variances are the diagonal entries. Since \(\Omega_{AA}\) is positive definite, its determinant is positive. Substituting those three entries into the definition of correlation cancels the common determinant factor:

\rho_{ij\mid -\{i,j\}} = \frac{-\Omega_{ij}/D} {\sqrt{(\Omega_{jj}/D)(\Omega_{ii}/D)}} = -\frac{\Omega_{ij}}{\sqrt{\Omega_{ii}\Omega_{jj}}}, \qquad D=\Omega_{ii}\Omega_{jj}-\Omega_{ij}^2>0.

This derivation also explains a frequent source of mistakes: to obtain the conditional covariance of the pair, invert the \(2\times2\) principal block \(\Omega_{AA}\). Do not invert the corresponding block of \(\Sigma\) and treat that as the conditional covariance. In general, the conditional covariance is not the marginal covariance of \(X_A\).

Interpretation: The sign of \(\rho_{ij\mid -\{i,j\}}\) is the opposite of the sign of \(\Omega_{ij}\). A positive value describes a positive linear association between the two features after conditioning on the others; a negative value describes a negative conditional linear association. Its magnitude is between 0 and 1 and measures the strength of that conditional correlation, not a change in either feature per unit of the other.

Calculating and Interpreting Partial Correlations

Worked Example: Reading a Partial Correlation From a Precision Matrix

Suppose a hypothetical Gaussian model for three standardized features—RNA feature \(X_1\), protein feature \(X_2\), and metabolite feature \(X_3\)—has precision matrix

\Omega= \begin{pmatrix} 2.00&-0.60&0.20\\ -0.60&1.50&-0.30\\ 0.20&-0.30&1.00 \end{pmatrix}.

The matrix is symmetric, and it is strictly diagonally dominant with positive diagonal entries, which ensures positive definiteness here. For \(X_1\) and \(X_2\), condition on \(X_3\). The required entries are \(\Omega_{12}=-0.60\), \(\Omega_{11}=2.00\), and \(\Omega_{22}=1.50\). Thus

\rho_{12\mid 3} = -\frac{-0.60}{\sqrt{(2.00)(1.50)}} = \frac{0.60}{\sqrt{3.00}} \approx 0.3464.

In this hypothetical model, the RNA and protein features have a positive conditional correlation of about 0.346 after accounting for the metabolite feature. This is a dimensionless correlation, not a slope and not evidence that either feature causes the other.

The diagonal entries are needed for normalization. An off-diagonal precision entry alone gives the direction of the partial correlation, but not its magnitude. The normalization also makes the result invariant to changing measurement units: rescaling a feature changes entries of \(\Omega\), but the ratio in the formula remains the same.

Worked Example: From Covariance to Conditional Correlation

Consider hypothetical standardized features with covariance matrix

\Sigma= \begin{pmatrix} 1&0.60&0.60\\ 0.60&1&0.20\\ 0.60&0.20&1 \end{pmatrix}.

Its determinant is \(1+2(0.60)(0.60)(0.20)-0.60^2-0.60^2-0.20^2=0.384\), which is positive; its leading principal minors are \(1\) and \(1-0.60^2=0.64\), also positive. Thus this symmetric matrix is positive definite. For the conditional correlation of \(X_1\) and \(X_2\) given \(X_3\), the conditional covariance and variances can be calculated directly:

\operatorname{Cov}(X_1,X_2\mid X_3)=0.60-(0.60)(0.20)=0.48, $$ $$ \operatorname{Var}(X_1\mid X_3)=1-0.60^2=0.64, \qquad \operatorname{Var}(X_2\mid X_3)=1-0.20^2=0.96.

Therefore

\rho_{12\mid 3} = \frac{0.48}{\sqrt{(0.64)(0.96)}} = \frac{0.48}{\sqrt{0.6144}} \approx 0.6124.

The same result follows from the precision matrix. The inverse is \(\Omega=\Sigma^{-1}\), with \(\Omega_{12}=(0.12-0.60)/0.384=-1.25\), \(\Omega_{11}=0.96/0.384=2.50\), and \(\Omega_{22}=0.64/0.384\approx1.667\). Hence

-\frac{\Omega_{12}}{\sqrt{\Omega_{11}\Omega_{22}}} = \frac{1.25}{\sqrt{(2.50)(1.667)}} \approx 0.6124.

The agreement checks both the conditional-covariance calculation and the precision-matrix formula. The marginal correlation of \(X_1\) and \(X_2\) is \(0.60\); conditioning on \(X_3\) changes it to about \(0.6124\). Conditioning need not always weaken an association.

Marginal and Conditional Relationships Can Differ

Worked Example: Zero Partial Correlation With Nonzero Marginal Correlation

Suppose a hypothetical three-feature Gaussian model has precision matrix

\Omega= \begin{pmatrix} 1&-0.50&0\\ -0.50&1&-0.50\\ 0&-0.50&1 \end{pmatrix}.

The leading principal minors are \(1\), \(0.75\), and \(0.50\), so this symmetric matrix is positive definite. Its inverse is

\Sigma= \begin{pmatrix} 1.50&1.00&0.50\\ 1.00&2.00&1.00\\ 0.50&1.00&1.50 \end{pmatrix}.

The marginal correlation between \(X_1\) and \(X_3\) is \(0.50/\sqrt{(1.50)(1.50)}=0.3333\). But \(\Omega_{13}=0\), so

\rho_{13\mid 2} = -\frac{0}{\sqrt{(1)(1)}}=0.

Thus these features are marginally correlated but have zero partial correlation given \(X_2\). In a Gaussian model, zero partial correlation is equivalent to conditional independence given the other features. It does not mean the features are uncorrelated in the unconditioned data.

Assumptions, Checks, and Alternatives

The exact interpretation above relies on the joint Gaussian assumption. It also requires a positive-definite covariance matrix: without it, the ordinary inverse \(\Sigma^{-1}\) may not exist, or the conditional variances needed for a correlation may not be positive. In applied work, a sample-based estimate of the precision matrix can be unstable or ill-conditioned, especially when the number of features is large relative to the number of samples. The formula still defines the partial correlation for a valid estimated precision matrix, but the estimate may be highly uncertain.

Before interpreting computed values, check that the matrix is symmetric and positive definite, that its diagonal entries are positive, and that the resulting correlations are in \([-1,1]\). Values outside that range indicate an invalid matrix or a calculation error. Also check feature definitions, preprocessing, and missing-data handling: the fitted Gaussian model must represent the measurements being interpreted. Strong skewness, outliers, or nonlinear dependence can make a Gaussian partial correlation an incomplete summary, even if the numerical calculation is correct.

Partial correlation is one option among several summaries. A marginal correlation describes the unadjusted linear relationship and can be useful when the unconditioned association is the target. A regression coefficient describes the expected change in one outcome per unit change in a predictor under a specified regression model; it is not itself a correlation, and its value depends on units and model specification. A partial correlation is a unitless, symmetric measure of conditional linear association for a pair after accounting for the chosen other variables. These summaries answer different questions; none alone establishes causation.

The model conditions on the measured features included in \(X\), not on every possible biological or technical factor. Unmeasured variables, measurement error, and model misspecification can affect the observed conditional relationships. A near-zero partial correlation is therefore a statement about the specified model and conditioning set, not a universal claim that two biological processes have no relationship.

Common Pitfall

Common mistake: “The marginal correlation is \(0.60\), so the partial correlation must be smaller because adjusting for other features removes association.” This reasoning is wrong: conditioning can strengthen, weaken, or reverse an association. Calculate the conditional covariance and variances, or use the precision formula, and interpret the result for its specified conditioning set.

Confirm the target.
Identify the two features and state which other features are being conditioned on.

Check the model quantity.
Use a valid precision matrix from the stated Gaussian model; confirm symmetry and positive definiteness.

Compute and interpret.
Evaluate \(-\Omega_{ij}/\sqrt{\Omega_{ii}\Omega_{jj}}\), then describe its sign and magnitude as a conditional correlation in context.

Check Your Understanding

Answer each question, then compare with the solutions below.

If \(\Omega_{ij}=0.40\), \(\Omega_{ii}=1.60\), and \(\Omega_{jj}=2.50\), what is the partial correlation, and what is its sign?
Why is \(\Omega_{AA}^{-1}\), rather than the corresponding block of \(\Sigma\), used for the conditional covariance of a pair of Gaussian features?
In a Gaussian model, what can be concluded if the partial correlation for two features given all others is zero? What cannot be concluded from that fact alone?
A researcher says a partial correlation of \(-0.70\) means that one feature causes the other to decrease. Identify the error and give a correct interpretation.

Solutions. (1) \(-0.40/\sqrt{(1.60)(2.50)}=-0.40/2.00=-0.2000\), a negative conditional correlation. (2) The quadratic term in the Gaussian density for the pair, with the other features fixed, has precision block \(\Omega_{AA}\); its inverse is the conditional covariance. The covariance block of \(\Sigma\) describes the pair marginally instead. (3) The two features are conditionally independent given all the other modeled features under the joint Gaussian assumption. This does not imply marginal independence or absence of relationships involving unmeasured variables. (4) A partial correlation is not a causal effect. The appropriate interpretation is a negative conditional linear association of magnitude \(0.70\), given the other modeled features; causal direction is not identified by this quantity.