Partial Correlation as Correlation of Residuals

What Partial Correlation Measures

Two variables can be correlated because they both vary with a third variable. Partial correlation asks a more focused question: after accounting for a chosen set of controls in a linear way, how strongly are the remaining parts of the two variables associated? The answer is a correlation between regression residuals.

For example, researchers might study the association between a health outcome and an exposure while controlling for age and smoking status. They regress the outcome on the controls and separately regress the exposure on those same controls. The residuals record, observation by observation, how far each value lies above or below its fitted value. Their correlation summarizes whether unusually high values of one variable tend to accompany unusually high or low values of the other, relative to those linear fits.

Definition: Let $X$ and $Y$ be two variables measured on the same $n$ observations, and let $Z_1,\ldots,Z_k$ be a specified set of controls. Regress $X$ on an intercept and the controls by least squares, and call the residual vector $e_X$. Regress $Y$ on the same intercept and controls, and call the residual vector $e_Y$. If both residual vectors have positive sums of squares, the sample partial correlation of $X$ and $Y$ given the controls is the sample correlation of $e_X$ and $e_Y$: $$ r_{XY\cdot Z_1,\ldots,Z_k} = \frac{\sum_{i=1}^n (e_{Xi}-\bar e_X)(e_{Yi}-\bar e_Y)} {\sqrt{\sum_{i=1}^n(e_{Xi}-\bar e_X)^2 \sum_{i=1}^n(e_{Yi}-\bar e_Y)^2}}. $$

The regressions must use the same observations and the same controls. We assume the design matrix, whose columns are the intercept and the controls, has full column rank, so its least-squares coefficients are uniquely determined. If a residual vector is identically zero, its correlation with another vector is undefined. Including an intercept is the usual convention; it also makes both residual vectors have mean zero.

Calculating the Residual Correlation

The regression step removes the part of each variable represented by a linear combination of the controls. The final step is an ordinary correlation calculation using the two residual vectors. In the examples below, we display residuals explicitly so that the partial correlation is not confused with a correlation between fitted values or with the original variables.

Worked Example: One Control and Four Observations

Suppose a study records a control $Z$ and two variables $X$ and $Y$ on four observations:

Observation	$Z$	$X$	$Y$
1	0	1	2
2	1	1	-5
3	2	3	0
4	3	7	-3

First regress $X$ on an intercept and $Z$. Here $\bar Z=1.5$ and $\bar X=3$. The centered cross-product is $ \sum_i (Z_i-\bar Z)(X_i-\bar X)=10 $, and $ \sum_i (Z_i-\bar Z)^2=5 $. Thus the slope is $10/5=2$, and the intercept is $3-2(1.5)=0$. The fitted values are $(0,2,4,6)$, giving residuals $e_X=(1,-1,-1,1)$.

For $Y$, $\bar Y=-1.5$. The centered cross-product with $Z$ is $-5$, so the slope is $-5/5=-1$, and the intercept is $-1.5-(-1)(1.5)=0$. The fitted values are $(0,-1,-2,-3)$, giving $e_Y=(2,-4,2,0)$.

Both residual vectors have mean zero. Their cross-product is $1(2)+(-1)(-4)+(-1)(2)+1(0)=4$. Their sums of squares are $1+1+1+1=4$ and $4+16+4+0=24$. Therefore

r_{XY\cdot Z}=\frac{4}{\sqrt{4\cdot24}} =\frac{1}{\sqrt{6}}\approx 0.408.

After removing the linear association with $Z$, the residuals have a moderate positive correlation. This does not say that the original variables have correlation $0.408$; it is the correlation of the residuals from the two specified regressions.

Worked Example: Two Controls

Consider five observations with controls $Z_1=(-2,-1,0,1,2)$ and $Z_2=(1,-2,2,-2,1)$. Let the observed variables be $X=(1,-7,4,-1,3)$ and $Y=(-0.5,-1,7,1,-6.5)$. The following residual vectors are useful: $u=(1,-2,0,2,-1)$ and $w=(-4,1,6,1,-4)$.

Each of $Z_1,Z_2,u,w$ has entries summing to zero. Their relevant cross-products are $Z_1\cdot Z_2=0$, $Z_1\cdot u=0$, $Z_2\cdot u=0$, $Z_1\cdot w=0$, and $Z_2\cdot w=0$. For example, $Z_2\cdot w=-4-2+12-2-4=0$. The observed variables can be written as

X=Z_1+2Z_2+u,\qquad Y=-Z_1+\tfrac12 Z_2+(u+w).

Consequently, the least-squares regression of $X$ on an intercept, $Z_1$, and $Z_2$ has intercept $0$, coefficients $1$ and $2$, and residual vector $e_X=u$. The corresponding regression of $Y$ has intercept $0$, coefficients $-1$ and $1/2$, and residual vector $e_Y=u+w=(-3,-1,6,3,-5)$. These fits follow directly because both residual vectors are orthogonal to the intercept and both control columns.

The residual cross-product is $u\cdot(u+w)=u\cdot u+u\cdot w=10+0=10$. The sums of squares are $u\cdot u=10$ and $(u+w)\cdot(u+w)=10+70=80$. Hence

r_{XY\cdot Z_1,Z_2}=\frac{10}{\sqrt{10\cdot80}} =\frac{1}{\sqrt{8}}\approx 0.354.

This example shows that “controlling for” several variables means using the same collection of predictors in both regressions, not just adjusting one variable and correlating it with the other unadjusted variable.

Why the Residual Formula Is a Correlation

With an intercept in each regression, the residuals are centered. This means the partial correlation can be calculated using residual cross-products directly, without separately subtracting residual means. The following result also states exactly when the calculation is defined.

Result: In ordinary least squares regressions that include an intercept, the residuals in each regression sum to zero. If neither residual vector is zero, their sample correlation is $$ r_{XY\cdot Z_1,\ldots,Z_k} = \frac{e_X^\mathsf{T}e_Y} {\sqrt{(e_X^\mathsf{T}e_X)(e_Y^\mathsf{T}e_Y)}}. $$ In particular, this value lies between $-1$ and $1$.

Proof. In the regression of $X$, write the observed vector as $x$, the design matrix (including its intercept column) as $D$, and the least-squares residual vector as $e_X=x-D\hat\beta_X$. Least squares minimizes $e_X^\mathsf{T}e_X$. Differentiating this quadratic objective with respect to the coefficient vector gives the normal equations $D^\mathsf{T}e_X=0$. The intercept column of $D$ is the vector of ones, so $\mathbf{1}^\mathsf{T}e_X=0$, or $\bar e_X=0$. The same argument gives $\bar e_Y=0$. Substituting these zero means into the sample correlation formula cancels the common factor $1/(n-1)$ from the covariance and the two sample variances, giving the displayed expression. Finally, the Cauchy–Schwarz inequality gives $|e_X^\mathsf{T}e_Y|\leq\sqrt{(e_X^\mathsf{T}e_X)(e_Y^\mathsf{T}e_Y)}$. The denominator is positive by assumption, so the ratio is defined and lies in $[-1,1]$. $\square$

The formula gives a precise interpretation of a zero partial correlation: the two residual vectors have zero inner product, so their sample linear association is zero. It does not establish that the variables are independent, nor does it rule out a nonlinear pattern in the residuals.

Result: If $e_X$ and $e_Y$ are nonzero, then the partial correlation is zero exactly when the regression of $e_Y$ on an intercept and $e_X$ has slope zero. Its sign is the sign of that slope.

Proof. Both residual vectors have mean zero. In the regression of $e_Y$ on an intercept and $e_X$, the slope is therefore $\hat b=(e_X^\mathsf{T}e_Y)/(e_X^\mathsf{T}e_X)$. The denominator is positive. Thus $\hat b=0$ exactly when $e_X^\mathsf{T}e_Y=0$, which, by the residual correlation formula, is exactly when the partial correlation is zero. The denominator in the expression for $\hat b$ is positive, so its sign agrees with the sign of $e_X^\mathsf{T}e_Y$, and hence with the partial correlation. $\square$

Raw Association Can Differ From Partial Association

A residual correlation can differ substantially from the ordinary correlation because the latter includes variation associated with the controls. The next example makes this distinction explicit. It also illustrates why a partial correlation should be reported with its controls identified: the value answers a question defined by the chosen adjustment set.

Worked Example: Strong Raw Association, Zero Partial Correlation

Take five observations with $Z=(-2,-1,0,1,2)$. Define $u=(1,-2,0,2,-1)$, $v=(1,0,-2,0,1)$, and set $X=2Z+u$, $Y=-Z+v$. The observed values are $X=(-3,-4,0,4,3)$ and $Y=(3,1,-2,-1,-1)$.

Both $u$ and $v$ sum to zero and are orthogonal to $Z$: $Z\cdot u=-2+2+0+2-2=0$ and $Z\cdot v=-2+0+0+0+2=0$. Thus regressions on an intercept and $Z$ leave residuals $u$ and $v$. Their cross-product is $u\cdot v=1+0+0+0-1=0$, so their partial correlation is $0$.

The raw variables also have mean zero. Their cross-product is $X\cdot Y=-9-4+0-4-3=-20$, while $X\cdot X=50$ and $Y\cdot Y=16$. Therefore their ordinary correlation is

r_{XY}=\frac{-20}{\sqrt{50\cdot16}} =-\frac{1}{\sqrt{2}}\approx -0.707.

Here the raw association is strongly negative, but after the linear trends with $Z$ are removed, the residuals are uncorrelated. These statements are compatible: they describe different pairs of quantities.

Interpretation and Common Pitfalls

Partial correlation is a measure of linear association after linear adjustment. It is not generally the same as a conditional correlation within every value of the controls. Nor does adjusting for a variable by itself establish a causal interpretation; that requires a justified study design and assumptions beyond the residual calculation.

Correlating the wrong quantities. Correlating fitted values, or correlating one residual vector with the original other variable, does not follow the definition. Regress both variables on the same controls, then correlate the two residual vectors.
Forgetting the intercept. Without an intercept, residuals need not sum to zero, and the centered-correlation simplification above does not apply as stated.
Ignoring a zero residual vector. If the controls explain one variable perfectly in the sample, that variable has no residual variation. Its residual correlation is undefined, not zero.
Interpreting zero too broadly. A zero residual correlation means zero linear residual association in this sample. It does not prove independence or absence of nonlinear association.
Changing the control set without noticing. The partial correlation is defined relative to the specified controls. A different set of controls leads to different residuals and can produce a different result.

In reporting a result, name the two variables, the controls, and the value of the residual correlation. That makes clear what has been adjusted for and prevents the partial correlation from being mistaken for the raw correlation or a causal effect.

Check Your Understanding

Use the residual definition and the examples above to answer these questions.

What two regressions are required to calculate the sample partial correlation of $X$ and $Y$ given controls $Z_1,\ldots,Z_k$?
Why do least-squares residuals sum to zero when each regression includes an intercept?
What does a zero partial correlation imply about the residual vectors, and what does it not imply about independence?
If the residual sum of squares for $X$ is zero, is the partial correlation zero or undefined? Explain.
In the four-observation example, identify the two residual vectors and verify the cross-product used in the partial correlation.

Observation	\(Z\)	\(X\)	\(Y\)
1	0	1	2
2	1	1	-5
3	2	3	0
4	3	7	-3