Partial Correlations in Gaussian Conditioning

Why Measure Association After Conditioning?

Two Gaussian variables can be correlated because they share variation with a third variable. If the goal is to describe how much dependence remains after that third variable, their marginal correlation is not the relevant quantity. The partial correlation measures the association between the remaining Gaussian variation in the two variables after conditioning on the specified coordinates.

For Gaussian vectors, conditioning changes covariance according to a Schur complement. Partial correlation standardizes the resulting conditional covariance by the two conditional standard deviations. The result is a dimensionless measure that ranges from $-1$ to $1$, just like an ordinary correlation, but it refers to dependence after conditioning.

Definition From Conditional Covariance

Let $X$ and $Y$ be scalar jointly Gaussian random variables, and let $Z$ be a $q$-dimensional Gaussian vector. Assume $\operatorname{Var}(Z)$ is positive definite and the conditional variances of $X$ and $Y$ given $Z$ are positive. Write the covariance blocks as $\Sigma_{XZ}=\operatorname{Cov}(X,Z)$, $\Sigma_{ZY}=\operatorname{Cov}(Z,Y)$, and $\Sigma_{ZZ}=\operatorname{Var}(Z)$. The conditional covariance and variances do not depend on the value $z$ in a Gaussian model, so they define one partial correlation.

Definition: The partial correlation of $X$ and $Y$ given $Z$, denoted $\rho_{XY\cdot Z}$, is the correlation under the conditional Gaussian distribution: $$ \rho_{XY\cdot Z} = \frac{\operatorname{Cov}(X,Y\mid Z)} {\sqrt{\operatorname{Var}(X\mid Z)\operatorname{Var}(Y\mid Z)}}. $$ With covariance blocks, its numerator is $$ \operatorname{Cov}(X,Y\mid Z) = \Sigma_{XY}-\Sigma_{XZ}\Sigma_{ZZ}^{-1}\Sigma_{ZY}, $$ and the two conditional variances are $$ \operatorname{Var}(X\mid Z) = \Sigma_{XX}-\Sigma_{XZ}\Sigma_{ZZ}^{-1}\Sigma_{ZX}, \qquad \operatorname{Var}(Y\mid Z) = \Sigma_{YY}-\Sigma_{YZ}\Sigma_{ZZ}^{-1}\Sigma_{ZY}. $$

For two coordinates $X_i$ and $X_j$ of a larger Gaussian vector, conditioning on a selected vector $Z$ uses exactly the same definition. More generally, if a Gaussian vector is partitioned into target coordinates $X$ and conditioning coordinates $Z$, its conditional covariance matrix is $$ \Sigma_{XX\mid Z} = \Sigma_{XX}-\Sigma_{XZ}\Sigma_{ZZ}^{-1}\Sigma_{ZX}. $$ A partial correlation between two target coordinates is the corresponding off-diagonal entry divided by the square root of the product of their diagonal entries. In this tutorial, conditioning covariance blocks are assumed positive definite, and the conditional variances used in correlations are positive.

Why the Block Formula Gives the Conditional Covariance

The block formula can be understood through Gaussian regression residuals. Let $X$ now be a vector of target coordinates and $Z$ a conditioning vector, and suppose $(X,Z)$ is jointly Gaussian with $\Sigma_{ZZ}$ positive definite. Define the residual after subtracting the conditional mean: $$ R_X=X-\mu_X-\Sigma_{XZ}\Sigma_{ZZ}^{-1}(Z-\mu_Z). $$ The conditional Gaussian law has mean $\mu_X+\Sigma_{XZ}\Sigma_{ZZ}^{-1}(z-\mu_Z)$. The following derivation shows that the covariance of the residual is the covariance remaining after conditioning.

Theorem: Under the assumptions above, $R_X$ is independent of $Z$, and $$ \operatorname{Var}(R_X) = \Sigma_{XX}-\Sigma_{XZ}\Sigma_{ZZ}^{-1}\Sigma_{ZX}. $$ Consequently, this matrix is the conditional covariance $\operatorname{Var}(X\mid Z)$.

Proof. Set $A=\Sigma_{XZ}\Sigma_{ZZ}^{-1}$. By bilinearity of covariance, $$ \operatorname{Cov}(R_X,Z) = \Sigma_{XZ}-A\Sigma_{ZZ} = \Sigma_{XZ}-\Sigma_{XZ}\Sigma_{ZZ}^{-1}\Sigma_{ZZ} = 0. $$ The pair $(R_X,Z)$ is jointly Gaussian because it is an affine transformation of the jointly Gaussian vector $(X,Z)$. Jointly Gaussian vectors with zero cross-covariance are independent. Thus $R_X$ is independent of $Z$.

For its covariance, expand the covariance of $X-A(Z-\mu_Z)$: $$ \begin{aligned} \operatorname{Var}(R_X) &=\Sigma_{XX}-A\Sigma_{ZX}-\Sigma_{XZ}A^{\mathsf T} +A\Sigma_{ZZ}A^{\mathsf T}\\ &=\Sigma_{XX}-\Sigma_{XZ}\Sigma_{ZZ}^{-1}\Sigma_{ZX} -\Sigma_{XZ}\Sigma_{ZZ}^{-1}\Sigma_{ZX} +\Sigma_{XZ}\Sigma_{ZZ}^{-1}\Sigma_{ZX}\\ &=\Sigma_{XX}-\Sigma_{XZ}\Sigma_{ZZ}^{-1}\Sigma_{ZX}. \end{aligned} $$ Here $\Sigma_{ZZ}$ is symmetric, so its inverse is symmetric. Since $X$ is the sum of a function of $Z$ and $R_X$, and $R_X$ is independent of $Z$, the conditional covariance of $X$ given $Z$ is the covariance of $R_X$. This proves the result. $\square$

For scalar $X$ and $Y$, apply the same argument jointly to the target vector $(X,Y)$. Its conditional covariance matrix has the two conditional variances on the diagonal and the conditional covariance off the diagonal. Standardizing that off-diagonal entry gives $\rho_{XY\cdot Z}$. Equivalently, the partial correlation is the ordinary correlation between the two residuals obtained by subtracting their respective conditional means given $Z$.

Worked Computations

Worked Example: Conditioning on One Gaussian Variable

Suppose $(X,Y,Z)$ is jointly Gaussian with $\operatorname{Var}(X)=4$, $\operatorname{Var}(Y)=9$, $\operatorname{Var}(Z)=2$, $\operatorname{Cov}(X,Y)=1$, $\operatorname{Cov}(X,Z)=1$, and $\operatorname{Cov}(Y,Z)=1.5$. Find $\rho_{XY\cdot Z}$.

First calculate the conditional variances and covariance. Since $Z$ is scalar, the block correction for a covariance involving $Z$ is the product of the two covariances with $Z$, divided by $\operatorname{Var}(Z)=2$: $$ \operatorname{Var}(X\mid Z)=4-\frac{1^2}{2}=3.5, \qquad \operatorname{Var}(Y\mid Z)=9-\frac{1.5^2}{2}=9-1.125=7.875. $$ The conditional covariance is $$ \operatorname{Cov}(X,Y\mid Z) =1-\frac{(1)(1.5)}{2} =0.25. $$ Therefore $$ \rho_{XY\cdot Z} = \frac{0.25}{\sqrt{(3.5)(7.875)}} = \frac{0.25}{5.25} = \frac{1}{21} \approx 0.0476. $$ The conditional association is positive but small. The arithmetic is internally consistent: the product of the conditional variances is $27.5625$, whose square root is $5.25$.

Worked Example: Conditioning on a Two-Dimensional Vector

Let $Z=(Z_1,Z_2)^{\mathsf T}\sim N(0,I_2)$. Suppose $$ X=Z_1+Z_2+\varepsilon_X, \qquad Y=Z_1-Z_2+\varepsilon_Y, $$ where $(\varepsilon_X,\varepsilon_Y)$ is independent of $Z$, is jointly Gaussian, and has covariance matrix $$ \begin{pmatrix}3&0.5\\0.5&3\end{pmatrix}. $$ Find the marginal and partial correlations of $X$ and $Y$, where the partial correlation conditions on both coordinates of $Z$.

The noise covariance is positive definite: its diagonal entries are positive and its determinant is $9-0.25=8.75>0$. The conditional means are $E[X\mid Z]=Z_1+Z_2$ and $E[Y\mid Z]=Z_1-Z_2$. After conditioning, the remaining variables are $\varepsilon_X$ and $\varepsilon_Y$. Hence $$ \operatorname{Var}(X\mid Z)=3,\qquad \operatorname{Var}(Y\mid Z)=3,\qquad \operatorname{Cov}(X,Y\mid Z)=0.5. $$ It follows that $$ \rho_{XY\cdot Z}=\frac{0.5}{\sqrt{3\cdot3}}=\frac{1}{6}. $$ Marginally, the shared $Z$-variation contributes covariance $\operatorname{Cov}(Z_1+Z_2,Z_1-Z_2)=1-1=0$. Thus $\operatorname{Cov}(X,Y)=0.5$. The marginal variances are $1+1+3=5$ for each variable, so $$ \operatorname{Corr}(X,Y)=\frac{0.5}{\sqrt{5\cdot5}}=0.1. $$ The partial correlation is larger than the marginal correlation here because conditioning removes variation that inflated both variances without contributing covariance.

Worked Example: Partial Correlations Among Three Targets

Let $Z\sim N(0,I_2)$, and let $X=(X_1,X_2,X_3)^{\mathsf T}=BZ+\varepsilon$, where $$ B= \begin{pmatrix} 1&0\\ 0&2\\ 1&-1 \end{pmatrix}, \qquad \varepsilon\sim N\left(0, \begin{pmatrix} 4&1&-2\\ 1&9&3\\ -2&3&16 \end{pmatrix}\right), $$ with $\varepsilon$ independent of $Z$. Compute the three pairwise partial correlations given $Z$.

The displayed residual covariance matrix is positive definite: its first leading principal minor is $4>0$, its second is $4\cdot9-1=35>0$, and its determinant is $$ 4(9\cdot16-3\cdot3)-1(1\cdot16-3(-2))-2(1\cdot3-9(-2)) =540-22-42=476>0. $$ Thus this construction defines a valid nondegenerate Gaussian residual vector. Given $Z$, the vector $BZ$ is fixed, so the conditional covariance of $X$ is exactly the covariance matrix of $\varepsilon$. In particular, the conditional variances are $4,9,16$, and the conditional covariances are $1,-2,3$ for the pairs $(1,2),(1,3),(2,3)$, respectively. Therefore $$ \rho_{X_1X_2\cdot Z}=\frac{1}{\sqrt{4\cdot9}}=\frac{1}{6}, \qquad \rho_{X_1X_3\cdot Z}=\frac{-2}{\sqrt{4\cdot16}}=-\frac{1}{4}, \qquad \rho_{X_2X_3\cdot Z}=\frac{3}{\sqrt{9\cdot16}}=\frac{1}{4}. $$ Each calculation conditions on $Z$, not on the other target coordinate. The three values summarize residual pairwise associations under that same conditioning set.

Zero Partial Correlation and Conditional Independence

For scalar Gaussian coordinates with positive conditional variances, a zero partial correlation has a stronger interpretation than merely saying that the residuals are uncorrelated.

Theorem: If $(X,Y,Z)$ is jointly Gaussian, $\operatorname{Var}(Z)$ is positive definite, and $\operatorname{Var}(X\mid Z)$ and $\operatorname{Var}(Y\mid Z)$ are positive, then $$ \rho_{XY\cdot Z}=0 \quad\Longleftrightarrow\quad X\text{ and }Y\text{ are conditionally independent given }Z. $$

Proof. For almost every $z$, the conditional distribution of $(X,Y)$ given $Z=z$ is bivariate Gaussian. Its two marginal variances are positive, and its covariance is $\operatorname{Cov}(X,Y\mid Z)$, which is constant in $z$. Since the denominator defining the partial correlation is positive, $\rho_{XY\cdot Z}=0$ exactly when that conditional covariance is zero. A bivariate Gaussian distribution with zero covariance has independent coordinates. Thus zero partial correlation implies that the conditional distribution factors into its $X$ and $Y$ marginals for almost every $z$, which is conditional independence.

Conversely, conditional independence implies that for almost every $z$, the conditional covariance is zero: the conditional expectation of the product of the centered variables factors into the product of their conditional expectations, both of which are zero. The conditional covariance is constant in $z$ for a jointly Gaussian vector, so it is zero, and therefore the partial correlation is zero. $\square$

Interpretation and a Common Pitfall

A partial correlation is a conditional, standardized measure of linear association. In a Gaussian model, it also detects conditional independence when it is zero. A value near $1$ or $-1$ indicates that, after conditioning, the two coordinates retain a strong positive or negative linear association. The value need not be close to the marginal correlation: removing shared variation can weaken, strengthen, or even reverse the association.

Pitfall: using the marginal covariance in the numerator. The incorrect reasoning is to take $\operatorname{Cov}(X,Y)$ and divide it by the conditional standard deviations. That mixes a marginal covariance with conditional variances. The correction is to compute all three entries from the same conditional covariance matrix, including the subtraction $\Sigma_{XY}-\Sigma_{XZ}\Sigma_{ZZ}^{-1}\Sigma_{ZY}$ in the numerator.
Pitfall: interpreting zero partial correlation as universal conditional independence. Outside the joint Gaussian setting, zero conditional covariance or correlation does not in general imply conditional independence. The implication above uses the Gaussian conditional distribution.
Pitfall: changing the conditioning set without noticing. A partial correlation is always relative to its stated set $Z$. Conditioning on additional variables generally changes the conditional covariance matrix and can change the value or sign.

The practical calculation is therefore systematic: specify the conditioning coordinates, form their conditional covariance matrix for the target coordinates, and standardize each desired off-diagonal entry by the corresponding conditional standard deviations. This makes clear both what the number measures and which variation has been removed.

Check Your Understanding

Use the definitions and conditional covariance calculations above to answer the following questions.

For scalar $X,Y$ and vector $Z$, write the block formula for $\operatorname{Cov}(X,Y\mid Z)$.
In the two-dimensional conditioning example, why does conditioning on $Z$ leave the covariance $0.5$ between $X$ and $Y$?
For the three-target example, which two quantities are used to standardize $\operatorname{Cov}(X_1,X_3\mid Z)$?
Under the stated Gaussian assumptions, what does $\rho_{XY\cdot Z}=0$ imply, and which Gaussian property justifies the implication?
Why is it incorrect to use $\operatorname{Cov}(X,Y)$ with conditional variances when calculating a partial correlation?