Comparing Partial Correlations with Shared Controls

Why Comparing Partial Correlations Requires Their Joint Covariance

Suppose a study asks whether \(X\) is more strongly associated with \(Y\) than with \(Z\), after adjusting both associations for the same covariates. The target is the difference between two population partial correlations, not the difference between two unrelated estimates. Because both estimates come from the same individuals and use the same controls, their sampling errors are generally dependent. A valid standard error for their difference must account for that dependence.

The method below treats the partial correlations as functions of a common sample covariance matrix. It derives their joint covariance using influence functions, then gives a simplified formula under joint normality when the two residual correlations share one variable. The same framework applies when the compared pairs do not share a variable.

Partial Correlations as Residual Correlations

This definition uses linear residuals; it does not require the conditional expectation of each variable given \(C\) to be linear. Under a multivariate normal model, linear residualization also gives the usual conditional correlation. For inference, assume independent and identically distributed observations, finite fourth moments, a nonsingular control covariance matrix, and positive residual variances for every variable used in the comparison.

Write \(r_i\) for the population residual and \(v_i=E[r_i^2]\) for its variance. The sample partial correlation is the ordinary sample correlation of the residuals from regressing the two sample variables on the same controls, including an intercept. Equivalently, it is a smooth function of the sample covariance matrix whenever the relevant control and residual covariance matrices are nonsingular.

Influence Functions and the Joint Covariance

An influence function describes, to first order, how an estimate changes when one observation is added to the population. For a partial correlation, it can be written using the population residuals alone. Define standardized residuals \(e_i=r_i/\sqrt{v_i}\), so that \(E[e_i]=0\), \(E[e_i^2]=1\), and \(E[e_i e_j]=\rho_{ij\cdot C}\).

It has mean zero because \(E[e_i e_j]=\rho_{ij\cdot C}\) and \(E[e_i^2]=E[e_j^2]=1\). The result accounts for estimating the regression coefficients for the controls as well as estimating the residual variances; it is not just the product of two fixed, known residuals.

The corresponding asymptotic variance of their difference is

To see why the influence function has the stated form, first consider the residual covariance \(V_{ij}=\operatorname{Cov}(R_i,R_j)\). Its first-order change under a perturbation of the joint covariance matrix is the change in the covariance of the residuals \(r_i,r_j\), namely \(r_i r_j-V_{ij}\). This follows by differentiating the residual covariance \(\sigma_{ij}-\sigma_{iC}\Sigma_{CC}^{-1}\sigma_{Cj}\): the changes in the two regression terms and the control covariance combine with the change in \(\sigma_{ij}\) to give the residual product. The same argument applies to each residual variance \(v_i\).

Now differentiate the correlation \(V_{ij}/\sqrt{v_i v_j}\). Its first-order change is the change in \(V_{ij}\) divided by \(\sqrt{v_i v_j}\), minus \(\rho_{ij\cdot C}/2\) times the relative changes in \(v_i\) and \(v_j\). Substituting the residual covariance and variance changes gives

For a sample, the multivariate central limit theorem applied to the influence-function averages gives the joint asymptotic normal distribution of the two estimates. In particular, their first-order errors are \(n^{-1}\sum_{m=1}^n\psi_{ij,m}\) and \(n^{-1}\sum_{m=1}^n\psi_{kl,m}\). The covariance of these averages is \(E[\psi_{ij}\psi_{kl}]/n\), which proves the joint covariance result. Finite fourth moments ensure the influence functions have finite variances. \(\square\)

A Gaussian Formula for Two Correlations Sharing a Variable

When the residual variables are jointly normal and the two partial correlations share one variable, the covariance has a compact expression. Suppose the two correlations are between residuals \(X,Y\) and \(X,Z\), after adjustment for the same controls. Let \(a=\rho_{XY\cdot C}\), \(b=\rho_{XZ\cdot C}\), and \(c=\rho_{YZ\cdot C}\). Joint normality of the residuals gives

For a derivation, use the influence functions \(\psi_{XY}=e_Xe_Y-a(e_X^2+e_Y^2)/2\) and \(\psi_{XZ}=e_Xe_Z-b(e_X^2+e_Z^2)/2\). Isserlis' formula for fourth moments of centered jointly normal variables gives

Expanding \(E[\psi_{XY}\psi_{XZ}]\) and substituting these moments gives the displayed covariance formula. In the same normal setting, the variance of an individual sample partial correlation has asymptotic coefficient \((1-a^2)^2\), so that \(\operatorname{Var}(\hat a)=(1-a^2)^2/n+o(n^{-1})\). For nonnormal residuals, use the influence-function expectation rather than the Gaussian formula.

Estimating the Covariance and Conducting the Test

In applications, replace population quantities in the influence functions by sample estimates. For each observation, form its control-adjusted residuals, standardize them using the estimated residual standard deviations, and compute the estimated influence value for each partial correlation. The empirical covariance of the two influence-value columns estimates \(\Gamma_{12}\); their empirical variances estimate \(\Gamma_{11}\) and \(\Gamma_{22}\). Dividing these estimates by \(n\) gives the estimated covariance matrix of the two partial correlations.

Equivalently, write \(\hat\Gamma_{ab}=n^{-1}\sum_{m=1}^n(\hat\psi_{a,m}-\bar{\hat\psi}_a)(\hat\psi_{b,m}-\bar{\hat\psi}_b)\). Then the estimated standard error for the difference is

Under \(H_0:\rho_1-\rho_2=0\), the Wald statistic \(Z=(\hat\rho_1-\hat\rho_2)/\widehat{\operatorname{SE}}(\hat\rho_1-\hat\rho_2)\) is asymptotically standard normal when the regularity conditions hold and the estimated variance is positive. For a two-sided test, compare \(|Z|\) with a standard normal critical value or calculate its two-sided tail probability. This is a large-sample test, not an exact finite-sample test.

Common Pitfall and Practical Interpretation

A tempting but generally incorrect calculation is to treat the two sample partial correlations as independent and use \(\{\widehat{\operatorname{Var}}(\hat\rho_1)+\widehat{\operatorname{Var}}(\hat\rho_2)\}^{1/2}\) as the standard error of their difference. The correct variance subtracts twice their covariance. If that covariance is positive, ignoring it overstates the variance; if it is negative, ignoring it understates the variance. Neither direction can be assumed in advance.

The Gaussian shared-variable formula is convenient, but it should not be applied merely because the variables are measured on a large sample. Its moment calculations require joint normality of the residual variables. With nonnormal data and finite fourth moments, the empirical influence-function covariance provides a sandwich estimate instead. In either case, the comparison concerns equality of linear partial correlations, not equality of causal effects or conditional associations of every possible form.