Partial Covariance Through Linear Residualization

Why Residualize Before Measuring Covariance?

Two variables can covary because they are associated with one another, because both vary with a third block of variables, or through both mechanisms. To separate these contributions in a linear Gaussian model, predict each target from the same control block and examine the covariance between what remains. This adjusted covariance is called a partial covariance.

The calculation is a multivariate version of subtracting a fitted linear trend before studying the remaining variation. The control block may contain several variables, and each target may itself be a vector. The central result is a block-matrix expression: subtract from the original cross-covariance the part explained through the controls.

Definition Through Linear Residualization

Let $X\in\mathbb{R}^p$, $Y\in\mathbb{R}^q$, and $Z\in\mathbb{R}^r$ be jointly Gaussian random vectors with finite second moments. Write their means as $\mu_X,\mu_Y,\mu_Z$, and partition their covariance matrix as

\operatorname{Cov} \begin{pmatrix} X\\Y\\Z \end{pmatrix} = \begin{pmatrix} \Sigma_{XX}&\Sigma_{XY}&\Sigma_{XZ}\\ \Sigma_{YX}&\Sigma_{YY}&\Sigma_{YZ}\\ \Sigma_{ZX}&\Sigma_{ZY}&\Sigma_{ZZ} \end{pmatrix}.

Assume $\Sigma_{ZZ}$ is positive definite, so the linear regression coefficients for predicting $X$ and $Y$ from $Z$ are well defined using its inverse. The fitted linear components, including intercepts, are $\mu_X+\Sigma_{XZ}\Sigma_{ZZ}^{-1}(Z-\mu_Z)$ and $\mu_Y+\Sigma_{YZ}\Sigma_{ZZ}^{-1}(Z-\mu_Z)$. Subtract these fitted components to obtain the residual vectors.

Definition: The partial covariance of $X$ and $Y$ given $Z$, denoted $\Sigma_{XY\cdot Z}$, is the cross-covariance of their linear residuals $$ R_X=X-\mu_X-\Sigma_{XZ}\Sigma_{ZZ}^{-1}(Z-\mu_Z),\qquad R_Y=Y-\mu_Y-\Sigma_{YZ}\Sigma_{ZZ}^{-1}(Z-\mu_Z). $$ Thus $\Sigma_{XY\cdot Z}=\operatorname{Cov}(R_X,R_Y)$, a $p\times q$ matrix.

The control block is not restricted to a single scalar. For example, if $Z$ has three coordinates and $X$ has two, the fitted component for $X$ uses a $2\times3$ coefficient matrix. The partial covariance then records the adjusted covariance for every coordinate of $X$ with every coordinate of $Y$.

Deriving the Block-Matrix Formula

Set $A=\Sigma_{XZ}\Sigma_{ZZ}^{-1}$ and $B=\Sigma_{YZ}\Sigma_{ZZ}^{-1}$. By bilinearity of covariance and the identity $\operatorname{Cov}(U,MW)=\operatorname{Cov}(U,W)M^{\mathsf T}$ for a fixed matrix $M$,

\begin{aligned} \operatorname{Cov}(R_X,R_Y) &=\operatorname{Cov}\bigl(X-\mu_X-A(Z-\mu_Z),\, Y-\mu_Y-B(Z-\mu_Z)\bigr)\\ &=\Sigma_{XY}-\Sigma_{XZ}B^{\mathsf T} -A\Sigma_{ZY}+A\Sigma_{ZZ}B^{\mathsf T}. \end{aligned}

Because $\Sigma_{ZZ}$ is symmetric, its inverse is symmetric. Consequently $B^{\mathsf T}=\Sigma_{ZZ}^{-1}\Sigma_{ZY}$. Substituting the definitions of $A$ and $B$, each of the three adjustment terms involving $Z$ is the same matrix:

\Sigma_{XZ}\Sigma_{ZZ}^{-1}\Sigma_{ZY}.

The first two such terms are subtracted and the last is added. Therefore the partial covariance has the following formula.

Theorem: If $(X,Y,Z)$ is jointly Gaussian and $\Sigma_{ZZ}$ is positive definite, then $$ \Sigma_{XY\cdot Z} =\operatorname{Cov}(R_X,R_Y) =\Sigma_{XY}-\Sigma_{XZ}\Sigma_{ZZ}^{-1}\Sigma_{ZY}. $$ The formula does not depend on the means. It is the $X,Y$ cross-block of the Schur complement obtained by adjusting the joint covariance of $(X,Y)$ for $Z$.

The same calculation also gives the residual covariance matrix for the two target blocks together:

\operatorname{Cov} \begin{pmatrix}R_X\\R_Y\end{pmatrix} = \begin{pmatrix} \Sigma_{XX}&\Sigma_{XY}\\ \Sigma_{YX}&\Sigma_{YY} \end{pmatrix} - \begin{pmatrix}\Sigma_{XZ}\\\Sigma_{YZ}\end{pmatrix} \Sigma_{ZZ}^{-1} \begin{pmatrix}\Sigma_{ZX}&\Sigma_{ZY}\end{pmatrix}.

Its off-diagonal block is $\Sigma_{XY\cdot Z}$. Its diagonal blocks are the residual covariance matrices for $X$ and $Y$. Since the displayed matrix is a covariance matrix, it is positive semidefinite; in particular, the adjusted blocks must jointly satisfy the usual covariance constraints.

Why Gaussianity Connects Residuals to Conditioning

Residualization is defined through a linear prediction operation. Conditioning is a distributional operation. For general distributions, these need not yield the same covariance. In the jointly Gaussian model, however, the linear residuals are independent of the controls, which makes the two quantities agree.

Theorem: Under the joint Gaussian and positive-definiteness assumptions above, for every $z$ in the support of $Z$, $$ \operatorname{Cov}(X,Y\mid Z=z) =\operatorname{Cov}(R_X,R_Y) =\Sigma_{XY\cdot Z}. $$ In particular, the conditional cross-covariance is constant in $z$.

To prove the result, first compute the covariance of each residual with the controls:

\operatorname{Cov}(R_X,Z) =\Sigma_{XZ}-\Sigma_{XZ}\Sigma_{ZZ}^{-1}\Sigma_{ZZ}=0, \qquad \operatorname{Cov}(R_Y,Z) =\Sigma_{YZ}-\Sigma_{YZ}\Sigma_{ZZ}^{-1}\Sigma_{ZZ}=0.

The vectors $(R_X,R_Y)$ and $Z$ are jointly Gaussian, since they are affine transformations of a jointly Gaussian vector. The Gaussian block-independence result says that jointly Gaussian blocks with zero cross-covariance are independent. Hence $(R_X,R_Y)$ is independent of $Z$. By the definitions of the residuals, the identities

X=\mu_X+\Sigma_{XZ}\Sigma_{ZZ}^{-1}(Z-\mu_Z)+R_X,\qquad Y=\mu_Y+\Sigma_{YZ}\Sigma_{ZZ}^{-1}(Z-\mu_Z)+R_Y

express $X$ and $Y$, conditional on $Z=z$, as fixed shifts of $R_X$ and $R_Y$. Their conditional cross-covariance is therefore the cross-covariance of those residuals. The formula derived above identifies it as $\Sigma_{XY\cdot Z}$, proving the claim. $\square$

Worked Calculations

Worked Example: One Scalar Control

Suppose $X,Y,Z$ are jointly Gaussian with zero means and the following covariances: $\operatorname{Var}(Z)=4$, $\operatorname{Cov}(X,Z)=2$, $\operatorname{Cov}(Y,Z)=3$, and $\operatorname{Cov}(X,Y)=3.5$. Calculate the partial covariance of $X$ and $Y$ given $Z$.

For a scalar control, the inverse of its variance is $1/4$, so the adjustment is

\operatorname{Cov}(X,Z)\operatorname{Var}(Z)^{-1}\operatorname{Cov}(Z,Y) =2\left(\frac14\right)3=1.5.

Subtract this explained component from the original covariance:

\operatorname{Cov}(X,Y\mid Z)=\Sigma_{XY\cdot Z} =3.5-1.5=2.

For a consistency check, take $\operatorname{Var}(X)=6$ and $\operatorname{Var}(Y)=8$. The residual variances are $6-2(1/4)2=5$ and $8-3(1/4)3=5.75$. The residual covariance matrix has determinant $5(5.75)-2^2=24.75>0$, so it is positive definite; these values are compatible with a nondegenerate Gaussian covariance matrix.

Worked Example: Two Control Variables

Let $Z=(Z_1,Z_2)^{\mathsf T}$, and suppose the relevant covariance blocks are

\Sigma_{ZZ}= \begin{pmatrix}2&0\\0&1\end{pmatrix},\qquad \Sigma_{XZ}=\begin{pmatrix}1&1\end{pmatrix},\qquad \Sigma_{YZ}=\begin{pmatrix}2&1\end{pmatrix},\qquad \Sigma_{XY}=5.

Here $X$ and $Y$ are scalars. The inverse control covariance matrix is $\Sigma_{ZZ}^{-1}=\begin{pmatrix}1/2&0\\0&1\end{pmatrix}$. Multiplying in the required order gives

\Sigma_{XZ}\Sigma_{ZZ}^{-1}\Sigma_{ZY} = \begin{pmatrix}1&1\end{pmatrix} \begin{pmatrix}1/2&0\\0&1\end{pmatrix} \begin{pmatrix}2\\1\end{pmatrix} = \begin{pmatrix}1/2&1\end{pmatrix} \begin{pmatrix}2\\1\end{pmatrix} =2.

Therefore $\Sigma_{XY\cdot Z}=5-2=3$. The subtraction accounts jointly for both controls and their variances; it is not obtained by subtracting the two raw covariances with the controls.

These blocks can be part of a valid joint Gaussian covariance matrix. For example, set $\operatorname{Var}(X)=\operatorname{Var}(Y)=10$. The residual variances are $10-(1/2+1)=8.5$ and $10-(2+1)=7$, and their covariance is $3$. Their covariance matrix has determinant $8.5(7)-3^2=50.5>0$. Together with the positive definite $\Sigma_{ZZ}$, this gives a positive definite full covariance matrix.

Worked Example: Covariance Explained Entirely by a Common Control

Let $Z\sim N(0,1)$, and let $\varepsilon_X\sim N(0,1)$ and $\varepsilon_Y\sim N(0,2)$ be independent of each other and of $Z$. Define $X=Z+\varepsilon_X$ and $Y=2Z+\varepsilon_Y$. Find their partial covariance given $Z$.

Independence and zero means give $\operatorname{Cov}(X,Y)=2$, $\operatorname{Cov}(X,Z)=1$, $\operatorname{Cov}(Y,Z)=2$, and $\operatorname{Var}(Z)=1$. Thus

\Sigma_{XY\cdot Z} =2-(1)(1)^{-1}(2)=0.

The residuals confirm the calculation directly: the regression coefficients are $1$ for $X$ and $2$ for $Y$, so $R_X=X-Z=\varepsilon_X$ and $R_Y=Y-2Z=\varepsilon_Y$. These errors are independent and have zero covariance. The marginal covariance of $2$ came entirely from the shared variation in $Z$; after residualizing both variables, no covariance remains.

Interpretation and a Common Pitfall

Each entry of $\Sigma_{XY\cdot Z}$ measures covariance between a coordinate of $X$ and a coordinate of $Y$ after their best linear predictions from $Z$ have been removed. The block formula shows that the adjustment is mediated by the covariance structure of the controls: $\Sigma_{ZZ}^{-1}$ scales and combines the relationships with $Z$ before the explained cross-covariance is subtracted.

Common mistake: treating partial covariance as the marginal covariance, or subtracting $\Sigma_{XZ}$ and $\Sigma_{ZY}$ directly. Those blocks may have different dimensions and units, and they do not by themselves quantify the covariance transmitted through $Z$. The correct adjustment is the dimensionally compatible product $\Sigma_{XZ}\Sigma_{ZZ}^{-1}\Sigma_{ZY}$.

A second important qualification is that residual covariance and conditional covariance are not interchangeable in every model. The equality proved here relies on joint Gaussianity: it makes the linear residuals independent of the controls. Outside that setting, the residual covariance still has a well-defined linear-adjustment interpretation, but it need not equal $\operatorname{Cov}(X,Y\mid Z=z)$, which may vary with $z$. Also, if $\Sigma_{ZZ}$ is singular, the inverse-based definition used here is unavailable; one must specify a different regression convention rather than silently applying this formula.

Key takeaway: For jointly Gaussian subvectors and a control block with positive definite covariance, partial covariance is the covariance of the two regression residuals: $$ \Sigma_{XY\cdot Z}=\Sigma_{XY}-\Sigma_{XZ}\Sigma_{ZZ}^{-1}\Sigma_{ZY}. $$ In this setting, it also equals the conditional cross-covariance given $Z$.

Check Your Understanding

Use the residualization definition and the block formula to answer the following.

What assumption on $\Sigma_{ZZ}$ allows the inverse-based residual coefficients to be defined?
If $X$ and $Y$ are scalar and $\operatorname{Var}(Z)=5$, $\operatorname{Cov}(X,Z)=2$, $\operatorname{Cov}(Y,Z)=4$, and $\operatorname{Cov}(X,Y)=6$, what is $\Sigma_{XY\cdot Z}$?
For vector-valued controls, what are the dimensions of $\Sigma_{XZ}\Sigma_{ZZ}^{-1}\Sigma_{ZY}$ when $X\in\mathbb{R}^p$ and $Y\in\mathbb{R}^q$?
Why does joint Gaussianity imply that the residual cross-covariance equals the conditional cross-covariance?
Can the residual covariance be assumed to equal conditional covariance for an arbitrary non-Gaussian model? Explain briefly.