Suppression and Sign Reversal in Partial Correlations

Why a Correlation Can Change After Control

A pair of variables can have a modest correlation even when their relationship after linear adjustment is stronger. The reason is that their shared linear association with a third variable can contribute to the unadjusted covariance. Removing that contribution may expose a larger residual association. If the removed contribution is large enough and points in the opposite direction, the residual association can even have the opposite sign from the original one.

These patterns are called suppression and sign reversal. The calculations below use residual covariances to show exactly what changes. They describe linear association, not a causal effect: controlling a variable does not, by itself, establish that the variable is a confounder or that adjustment is scientifically appropriate.

Residual Covariance and Partial Correlation

Let \(X,Y,Z\) be random variables with finite second moments and positive variances. For the moment, suppose they are centered, so their expectations are zero. Regressing \(X\) linearly on \(Z\) gives the population residual \(X^*=X-\beta_X Z\), where \(\beta_X=\operatorname{Cov}(X,Z)/\operatorname{Var}(Z)\). Define \(Y^*\) in the same way. These are linear-projection residuals; no normality assumption is required.

Definition: The partial correlation between \(X\) and \(Y\) controlling for \(Z\) is the correlation between their residuals after each has been linearly projected on \(Z\): \(\operatorname{Corr}(X^*,Y^*)\), provided both residual variances are positive. The zero-order correlation is \(\operatorname{Corr}(X,Y)\), before controlling for \(Z\).

Write \(r_{XY}=\operatorname{Corr}(X,Y)\), \(r_{XZ}=\operatorname{Corr}(X,Z)\), and \(r_{YZ}=\operatorname{Corr}(Y,Z)\). The key calculation is that adjustment subtracts the covariance carried through the linear association with \(Z\).

Result: If \(|r_{XZ}|<1\) and \(|r_{YZ}|<1\), then the residual covariance, residual variances, and partial correlation are \[ \operatorname{Cov}(X^*,Y^*)=\sigma_X\sigma_Y(r_{XY}-r_{XZ}r_{YZ}), \] \[ \operatorname{Var}(X^*)=\sigma_X^2(1-r_{XZ}^2),\qquad \operatorname{Var}(Y^*)=\sigma_Y^2(1-r_{YZ}^2), \] \[ r_{XY\cdot Z}= \frac{r_{XY}-r_{XZ}r_{YZ}} {\sqrt{(1-r_{XZ}^2)(1-r_{YZ}^2)}}. \]

Here \(\sigma_X^2=\operatorname{Var}(X)\), and the other standard deviations are defined correspondingly. The numerator is the residual covariance expressed in units of \(\sigma_X\sigma_Y\); the denominator converts it to a correlation. Thus an increase in residual covariance does not by itself determine the size of the partial correlation: residual variances also matter.

Worked Example: A Stronger Positive Partial Correlation

Suppose \(X,Y,Z\) are standardized, with \(r_{XY}=0.20\), \(r_{XZ}=0.60\), and \(r_{YZ}=-0.20\). Calculate the partial correlation between \(X\) and \(Y\) controlling for \(Z\), and compare it with the zero-order correlation.

The residual covariance is \(r_{XY}-r_{XZ}r_{YZ}=0.20-(0.60)(-0.20)=0.20+0.12=0.32\). The residual variances are \(\operatorname{Var}(X^*)=1-0.60^2=0.64\) and \(\operatorname{Var}(Y^*)=1-(-0.20)^2=0.96\). Therefore their variance product is \(0.64(0.96)=0.6144\), and the standardizing factor is \(\sqrt{0.6144}\approx0.7838\). Hence \(r_{XY\cdot Z}=0.32/0.7838\approx0.4082\).

The partial correlation is positive and larger in magnitude than \(0.20\). The residual covariance rose from \(0.20\) to \(0.32\) because the product \(r_{XZ}r_{YZ}=-0.12\) is negative: the part of the covariance associated with \(Z\) opposes the remaining association. The correlation is not \(0.32\), because \(0.32\) is the standardized-scale residual covariance, not the correlation of the residuals.

The proposed correlations are feasible. The determinant of their \(3\)-by-\(3\) correlation matrix is \(1+2(0.20)(0.60)(-0.20)-0.20^2-0.60^2-(-0.20)^2=0.512>0\). Together with the positive leading principal minors, this verifies positive definiteness.

When Does the Sign Reverse?

The denominator in the partial-correlation formula is positive whenever both residual variances are positive. Consequently, the sign of the partial correlation is exactly the sign of \(r_{XY}-r_{XZ}r_{YZ}\). This gives a direct criterion for sign reversal.

Theorem: Suppose \(|r_{XZ}|<1\), \(|r_{YZ}|<1\), and \(r_{XY}\ne0\). The partial correlation \(r_{XY\cdot Z}\) has the opposite sign from the zero-order correlation \(r_{XY}\) if and only if \[ r_{XY}(r_{XY}-r_{XZ}r_{YZ})<0. \] In particular, if \(r_{XY}>0\), reversal occurs exactly when \(r_{XZ}r_{YZ}>r_{XY}\). If \(r_{XY}<0\), it occurs exactly when \(r_{XZ}r_{YZ}<r_{XY}\).

To prove the criterion, the positive denominator means \(\operatorname{sign}(r_{XY\cdot Z})=\operatorname{sign}(r_{XY}-r_{XZ}r_{YZ})\). Two nonzero numbers have opposite signs exactly when their product is negative, giving the stated inequality. For \(r_{XY}>0\), the residual covariance must be negative, so \(r_{XY}-r_{XZ}r_{YZ}<0\), equivalent to \(r_{XZ}r_{YZ}>r_{XY}\). For \(r_{XY}<0\), it must be positive, so \(r_{XZ}r_{YZ}<r_{XY}\). This proves both cases. \(\square\)

Worked Example: Reversal From Positive to Negative

Let \(r_{XY}=0.15\), \(r_{XZ}=0.60\), and \(r_{YZ}=0.50\). The zero-order correlation is positive. Since \(r_{XZ}r_{YZ}=0.30>0.15\), the sign-reversal criterion predicts a negative partial correlation.

The residual covariance is \(0.15-(0.60)(0.50)=0.15-0.30=-0.15\). The residual variances are \(1-0.60^2=0.64\) and \(1-0.50^2=0.75\). Their product is \(0.48\), so \(r_{XY\cdot Z}=-0.15/\sqrt{0.48}\approx-0.2165\).

Thus the zero-order association is \(0.15\), while the residual association is negative. The adjustment term \(0.30\) is larger than the original covariance \(0.15\), leaving a negative residual covariance. The correlation matrix is valid: its determinant is \(1+2(0.15)(0.60)(0.50)-0.15^2-0.60^2-0.50^2=0.4575>0\), and its leading principal minors are positive.

The reversal is a statement about linear residual association. It does not imply that increasing \(X\) causes \(Y\) to decrease, nor that \(Z\) is necessarily a confounder. Those claims require a justified study design and additional assumptions.

When Does Control Increase the Magnitude?

A control can increase the absolute partial correlation without reversing its sign. To determine whether that happens, compare the squared partial correlation with the squared zero-order correlation. Squaring avoids separate cases for positive and negative correlations.

Result: Let \(a=r_{XY}\), \(b=r_{XZ}\), and \(c=r_{YZ}\), with \(|b|<1\) and \(|c|<1\). Then controlling for \(Z\) increases the absolute correlation exactly when \[ |r_{XY\cdot Z}|>|r_{XY}| \quad\Longleftrightarrow\quad (a-bc)^2>a^2(1-b^2)(1-c^2). \]

Indeed, both sides of the comparison of absolute correlations are nonnegative, and the denominator \(\sqrt{(1-b^2)(1-c^2)}\) is positive. Squaring \(|(a-bc)/\sqrt{(1-b^2)(1-c^2)}|>|a|\) and multiplying by the positive factor \((1-b^2)(1-c^2)\) gives precisely the stated inequality. Every operation is reversible, proving the equivalence. \(\square\)

This condition accounts for both the change in residual covariance and the change in residual variances. A negative product \(bc\) raises the adjusted numerator relative to \(a\) when \(a>0\), as in the first example, but the denominator may still affect the final comparison. Therefore, inspecting the covariance alone is not a complete check that the correlation magnitude has increased.

Worked Example: A Pure Suppressor Pattern

Suppose \(r_{XY}=0.30\), \(r_{XZ}=0.70\), and \(r_{YZ}=0\). The variable \(Z\) is uncorrelated with \(Y\) but associated with \(X\). Calculate the partial correlation and verify that its magnitude increases.

The residual covariance is \(0.30-(0.70)(0)=0.30\): it has not changed. The residual variances are \(1-0.70^2=0.51\) for \(X\) and \(1-0^2=1\) for \(Y\). Therefore \(r_{XY\cdot Z}=0.30/\sqrt{0.51}\approx0.4201\), which exceeds \(0.30\) in absolute value. Here the increase comes entirely from standardization: removing the part of \(X\) associated with \(Z\) reduces its residual variance, while \(Y\)'s residual variance is unchanged.

For a direct inequality check, \(a=0.30\), \(b=0.70\), and \(c=0\). The left side is \((a-bc)^2=0.30^2=0.09\); the right side is \(a^2(1-b^2)(1-c^2)=0.09(0.51)(1)=0.0459\). Since \(0.09>0.0459\), the magnitude increases. The correlation matrix determinant is \(1-0.30^2-0.70^2=0.42>0\), so these correlations are feasible.

Interpreting Suppression Carefully

In these examples, suppression describes a change in association after linear adjustment. The mechanism can differ: the residual covariance may increase because the removed covariance component opposes it, or the residual covariance may stay the same while a residual variance falls. A sign reversal is a particularly clear warning that the unadjusted and adjusted associations summarize different linear relationships.

A common mistake is to report the adjusted numerator as the partial correlation. In the first example, \(0.32\) is the residual covariance because the variables are standardized initially; the partial correlation is \(0.32/\sqrt{0.6144}\approx0.4082\). Another mistake is to assume that adjustment must reduce a correlation. Residual variance changes make that claim false, as the pure suppressor example demonstrates.

Finally, a large increase or sign reversal does not prove that the control variable should be included in a scientific model. The algebra describes what happens under linear projection. Choosing controls requires substantive reasoning about the design and the question being asked. The practical workflow is to calculate the residual covariance, calculate both residual variances, standardize only after those quantities are known, and interpret the result as an adjusted association rather than a causal conclusion.

Check Your Understanding

Use the residual-covariance formula and show the key arithmetic in each calculation.

If \(r_{XY}=0.25\), \(r_{XZ}=0.50\), and \(r_{YZ}=-0.20\), what is the residual covariance in standardized units? What are the residual variances?
State the condition for a positive zero-order correlation to become a negative partial correlation after controlling for \(Z\).
Can the partial correlation increase in magnitude when the residual covariance does not change? Explain using the residual variances.
For \(r_{XY}=-0.10\), \(r_{XZ}=0.40\), and \(r_{YZ}=0.50\), use the sign criterion to determine whether sign reversal occurs.
Why does a sign reversal in partial correlation, by itself, not establish a causal effect?