Higher-Order Partial Correlations from Precision Submatrices

Why the Control Set Matters

A partial correlation describes the linear association between two variables after removing their linear associations with a chosen set of controls. With several controls, the calculation is not based on the two target variables alone: the covariance structure of the targets and all selected controls must be considered together. A useful matrix calculation is to invert the covariance submatrix for exactly those variables and read the partial correlation from the resulting precision matrix.

The choice of control set is part of the quantity being calculated. The same pair of target variables can have different partial correlations for different control sets. Here the goal is to calculate the value for a specified set, not to select controls or conduct a test.

Definition: Let \(X\) and \(Y\) be variables with finite, positive variances, and let \(Z=(Z_1,\ldots,Z_k)^T\) be a specified vector of control variables. Regress \(X\) linearly on an intercept and \(Z\), and denote the residual by \(R_X\); define \(R_Y\) in the same way. If both residual variances are positive, the partial correlation of \(X\) and \(Y\) given \(Z\) is \(\operatorname{Corr}(R_X,R_Y)=\operatorname{Cov}(R_X,R_Y)/\sqrt{\operatorname{Var}(R_X)\operatorname{Var}(R_Y)}\). It measures the correlation left after linear adjustment for the specified controls.

Assume throughout that the covariance matrix of the selected variables \((X,Y,Z_1,\ldots,Z_k)\) is positive definite. This ensures that the required inverses exist and that the residual variances are positive. Normality is not needed: this matrix calculation gives the linear partial correlation from second moments.

The Selected Covariance Matrix and Its Inverse

Put the target variables first and the controls after them. Write the covariance matrix of this selected vector in block form:

\Sigma_{(X,Y,Z)} = \begin{pmatrix} A & B\\ B^T & C \end{pmatrix}, \qquad A=\operatorname{Cov}\begin{pmatrix}X\\Y\end{pmatrix}, \quad C=\operatorname{Cov}(Z), \quad B=\operatorname{Cov}\left(\begin{pmatrix}X\\Y\end{pmatrix},Z\right).

The covariance matrix of the two residuals after linear projection on the controls is \(\Omega=A-BC^{-1}B^T\). This is the Schur complement of \(C\) in the selected covariance matrix. Inverting the full selected covariance matrix gives a block whose entries are precisely the inverse of \(\Omega\).

Theorem: Suppose \(\Sigma_{(X,Y,Z)}\) is positive definite and the variables are ordered with \(X,Y\) first. Let \(Q=\Sigma_{(X,Y,Z)}^{-1}\), and let \(q_{11},q_{12},q_{22}\) be the entries of its leading \(2\)-by-\(2\) block. Then the partial correlation of \(X\) and \(Y\) given \(Z\) is \(\displaystyle \rho_{XY\cdot Z}=-\frac{q_{12}}{\sqrt{q_{11}q_{22}}}\).

Proof. Because the selected covariance matrix is positive definite, \(C\) is invertible. The linear projection coefficients for the two targets on \(Z\) are \(BC^{-1}\). Subtracting the fitted linear parts therefore gives residual covariance \(\Omega=A-BC^{-1}B^T\). Positive definiteness also implies that \(\Omega\) is positive definite and invertible. The block inverse identity gives

\begin{pmatrix} A & B\\ B^T & C \end{pmatrix}^{-1} = \begin{pmatrix} \Omega^{-1} & -\Omega^{-1}BC^{-1}\\ -C^{-1}B^T\Omega^{-1} & C^{-1}+C^{-1}B^T\Omega^{-1}BC^{-1} \end{pmatrix}.

For verification, multiply the proposed inverse on the left by the covariance matrix. The upper-left block is \(A\Omega^{-1}-BC^{-1}B^T\Omega^{-1}=\Omega\Omega^{-1}=I\), and the upper-right block is \(-A\Omega^{-1}BC^{-1}+BC^{-1}B^T\Omega^{-1}BC^{-1}+BC^{-1}=0\). The lower-left block is \(B^T\Omega^{-1}-B^T\Omega^{-1}=0\); the lower-right block is \(-B^T\Omega^{-1}BC^{-1}+I+B^T\Omega^{-1}BC^{-1}=I\). Thus the identity holds, and the leading block of \(Q\) is \(\Omega^{-1}\).

Write the residual covariance as \(\Omega=\begin{pmatrix}u&w\\w&v\end{pmatrix}\). Its inverse is \(\frac{1}{uv-w^2}\begin{pmatrix}v&-w\\-w&u\end{pmatrix}\). Hence \(q_{11}=v/(uv-w^2)\), \(q_{12}=-w/(uv-w^2)\), and \(q_{22}=u/(uv-w^2)\). Substitution into the stated formula gives \(w/\sqrt{uv}\), which is the correlation of the two residuals. \(\square\)

Calculation rule: Select the covariance submatrix for \(X,Y\) and every control in \(Z\). Invert that selected matrix. If \(X,Y\) are first in the ordering, take the negative of the off-diagonal entry in the leading \(2\)-by-\(2\) block and divide by the square root of the product of its diagonal entries.

Worked Calculations

Worked Example: Two Independent Controls

Suppose the selected variables are ordered \(X,Y,Z_1,Z_2\), with covariance matrix

\Sigma= \begin{pmatrix} 4&0.6&1&1\\ 0.6&5&1&-1\\ 1&1&1&0\\ 1&-1&0&1 \end{pmatrix}.

We seek the partial correlation of \(X\) and \(Y\) controlling for both \(Z_1\) and \(Z_2\). The control covariance block is \(C=I_2\), and the cross-covariance block is \(B=\begin{pmatrix}1&1\\1&-1\end{pmatrix}\). Thus \(BB^T=\begin{pmatrix}2&0\\0&2\end{pmatrix}\). The residual covariance is

\Omega= \begin{pmatrix}4&0.6\\0.6&5\end{pmatrix} - \begin{pmatrix}2&0\\0&2\end{pmatrix} = \begin{pmatrix}2&0.6\\0.6&3\end{pmatrix}.

Its determinant is \(2(3)-0.6^2=5.64\), so \(\Omega^{-1}=\frac{1}{5.64}\begin{pmatrix}3&-0.6\\-0.6&2\end{pmatrix}\). By the block inverse identity, these are the leading entries of \(\Sigma^{-1}\). Therefore \(q_{11}=3/5.64\), \(q_{12}=-0.6/5.64\), and \(q_{22}=2/5.64\). The partial correlation is

-\frac{q_{12}}{\sqrt{q_{11}q_{22}}} = \frac{0.6/5.64}{\sqrt{(3/5.64)(2/5.64)}} = \frac{0.6}{\sqrt{6}} \approx 0.245.

After linear adjustment for both controls, the residual association is positive and smaller than the raw covariance might suggest; the calculation reports a residual correlation, not a residual covariance.

Worked Example: Correlated Controls

Consider the selected covariance matrix

\Sigma= \begin{pmatrix} 10/3&-5/3&1&0\\ -5/3&16/3&0&1\\ 1&0&1&1/2\\ 0&1&1/2&1 \end{pmatrix},

for \(X,Y,Z_1,Z_2\), in that order. The controls are correlated, so their covariance block is \(C=\begin{pmatrix}1&1/2\\1/2&1\end{pmatrix}\), with \(C^{-1}=\begin{pmatrix}4/3&-2/3\\-2/3&4/3\end{pmatrix}\). Here \(B=I_2\), so the residual covariance is \(A-C^{-1}\):

\Omega= \begin{pmatrix}10/3&-5/3\\-5/3&16/3\end{pmatrix} - \begin{pmatrix}4/3&-2/3\\-2/3&4/3\end{pmatrix} = \begin{pmatrix}2&-1\\-1&4\end{pmatrix}.

The determinant is \(2(4)-(-1)^2=7\), giving \(\Omega^{-1}=\frac{1}{7}\begin{pmatrix}4&1\\1&2\end{pmatrix}\). Thus \(q_{11}=4/7\), \(q_{12}=1/7\), and \(q_{22}=2/7\). The requested partial correlation is

-\frac{1/7}{\sqrt{(4/7)(2/7)}}=-\frac{1}{\sqrt{8}}\approx -0.354.

The negative sign comes from the positive off-diagonal precision entry. A positive precision off-diagonal entry corresponds to a negative residual covariance in this two-target calculation.

Worked Example: Three Controls

Let \(X,Y\) be the targets and \(Z_1,Z_2,Z_3\) the controls, in that order, with covariance matrix

\Sigma= \begin{pmatrix} 1.25&0.27&0.3&0.4&0\\ 0.27&1.8&0.2&-0.1&0.5\\ 0.3&0.2&1&0&0\\ 0.4&-0.1&0&1&0\\ 0&0.5&0&0&1 \end{pmatrix}.

The controls have covariance \(C=I_3\), so \(C^{-1}=I_3\). The cross-covariance block has rows \((0.3,0.4,0)\) and \((0.2,-0.1,0.5)\). Their squared lengths are \(0.3^2+0.4^2=0.25\) and \(0.2^2+(-0.1)^2+0.5^2=0.30\); their inner product is \(0.3(0.2)+0.4(-0.1)+0(0.5)=0.02\). Consequently,

\Omega=A-BB^T = \begin{pmatrix}1.25&0.27\\0.27&1.8\end{pmatrix} - \begin{pmatrix}0.25&0.02\\0.02&0.30\end{pmatrix} = \begin{pmatrix}1&0.25\\0.25&1.5\end{pmatrix}.

The determinant is \(1(1.5)-0.25^2=1.4375=23/16\), hence \(\Omega^{-1}=\begin{pmatrix}24/23&-4/23\\-4/23&16/23\end{pmatrix}\). The partial correlation is

-\frac{-4/23}{\sqrt{(24/23)(16/23)}} = \frac{4}{\sqrt{384}} = \frac{1}{\sqrt{24}} \approx 0.204.

The same calculation applies with three controls: the leading \(2\)-by-\(2\) block of the selected precision matrix supplies the needed entries, regardless of how many controls are included.

Ordering and What Is Being Inverted

Result: Reordering the controls does not change the partial correlation, provided the covariance matrix is reordered consistently. The targets should be placed first only to make the relevant precision entries easy to identify.

Proof. A reordering of controls multiplies their vector by a permutation matrix \(P\). The control covariance and target-control covariance become \(PCP^T\) and \(BP^T\), respectively, if the controls are represented as a column vector under the new ordering. Since \((PCP^T)^{-1}=PC^{-1}P^T\), the Schur complement remains \(A-(BP^T)(PC^{-1}P^T)(PB^T)=A-BC^{-1}B^T=\Omega\). Its inverse and the resulting target partial correlation are unchanged. The control order is bookkeeping, not a change to the control set. \(\square\)

Be precise about the matrix being inverted. If a larger vector of variables is available but only some variables are designated as controls, first extract the covariance submatrix for the two targets and those controls, and then invert it. In general, taking the corresponding entries from the inverse of the covariance matrix for all available variables gives a different quantity: that full inverse accounts for the other variables as well. Its entries describe adjustment for a larger conditioning set, not automatically the selected set in the question.

Common Mistakes and Interpretation

Using the wrong sign. The partial correlation is the negative normalized off-diagonal precision entry. Omitting the minus sign reverses the result.
Using the wrong diagonal entries. Normalize \(q_{12}\) by \(\sqrt{q_{11}q_{22}}\) from the same leading target block, not by diagonal entries of the original covariance matrix.
Inverting the full matrix when only a subset is controlled. Select the target and control variables first. The inverse of their covariance submatrix matches the specified control set.
Calling this automatically a conditional correlation. It is a correlation of linear-projection residuals. Without additional assumptions, it need not equal a conditional correlation that varies with the control values. Under joint multivariate normality, the residual covariance also describes the conditional covariance, but normality is not required for the linear partial correlation formula.

This calculation is useful whenever a multivariate covariance matrix is available and the question specifies which variables to adjust for. It makes the control set explicit, works for any number of controls, and separates the two-target residual association from the rest of the covariance structure. A numerical answer should always be reported with its control set, since a partial correlation is not fully specified by the target pair alone.

Check Your Understanding

Use the definitions and matrix identity above to answer these questions.

Which covariance submatrix should be inverted to calculate the partial correlation of \(X\) and \(Y\) given \(Z_1,Z_2\), when additional variables are also recorded?
Why does the formula use the negative of the off-diagonal precision entry?
For residual covariance \(\begin{pmatrix}5&-1\\-1&2\end{pmatrix}\), calculate the partial correlation.
Does the linear partial correlation formula require the variables to be multivariate normal? Explain.
What does a reordering of the controls change, and what does it leave unchanged?