From a Correlation to a Defensible Analysis
A partial or multiple correlation is a summary, not a complete analysis. Its value depends on which variables were measured, which variables were conditioned on, and whether a linear summary is reasonable for the observed data. A disciplined workflow makes those choices visible: define the question, choose the conditioning set, calculate the summaries, inspect the data, and report uncertainty and limitations.
Suppose an observational study measures physical activity \(X\), systolic blood pressure \(Y\), age \(Z_1\), and baseline body-mass index \(Z_2\). The question might be whether activity and blood pressure are linearly associated among people with the same age and baseline body-mass index. That is a conditional association question. It does not, by itself, ask what would happen to blood pressure if activity were changed.
The control set should be chosen to define the scientific comparison, not selected merely because adding a variable makes the correlation larger, smaller, or statistically significant. State the controls in advance when possible and explain why they belong in the analysis. A partial correlation describes association after linear adjustment for that particular set; it is not a guarantee that all relevant differences have been removed.
Calculating Partial and Multiple Correlations
For a compact calculation, suppose the variables have been standardized and their sample correlations are as follows. Let \(Z_1\) and \(Z_2\) be uncorrelated in this sample.
| Pair | Sample correlation |
|---|---|
| \(X,Y\) | 0.55 |
| \(X,Z_1\) | 0.50 |
| \(X,Z_2\) | 0.30 |
| \(Y,Z_1\) | 0.40 |
| \(Y,Z_2\) | 0.20 |
| \(Z_1,Z_2\) | 0 |
Worked Example: Adjusting for Age and Baseline BMI
Assume the table describes \(n=80\) complete observations. Calculate the partial correlation between activity \(X\) and blood pressure \(Y\), controlling for age \(Z_1\) and baseline body-mass index \(Z_2\). Then calculate the multiple correlation of \(Y\) with all three predictors.
Partial correlation. Since the two controls are uncorrelated, the linear projection of standardized \(X\) on them has coefficients 0.50 and 0.30. Its residual variance is \(1-0.50^2-0.30^2=0.66\). For \(Y\), the corresponding residual variance is \(1-0.40^2-0.20^2=0.80\). The covariance of the two residuals is the original covariance minus the part accounted for by the controls:
Therefore the partial correlation is
The adjusted linear association is positive. Its magnitude is not the raw correlation of 0.55: it compares the parts of activity and blood pressure left after their separate linear relationships with the two controls have been removed.
Multiple correlation. Regression of \(Y\) on the two controls alone explains \(0.40^2+0.20^2=0.20\) of its sample variance, because the controls are uncorrelated. After removing their linear contributions, the residual variance of \(Y\) is 0.80. The residualized \(X\) has variance 0.66 and covariance 0.29 with that residualized \(Y\). Adding \(X\) therefore explains an additional fraction of the original \(Y\) variance equal to
Thus \(R^2_{Y\cdot X,Z_1,Z_2}=0.20+0.1274=0.3274\), and the multiple correlation is \(R_{Y\cdot X,Z_1,Z_2}=\sqrt{0.3274}\approx 0.572\). This is a nonnegative measure of the linear association of \(Y\) with the predictor set; unlike a partial correlation, it has no positive or negative sign.
The displayed correlations are internally possible: after projecting out \(Z_1,Z_2\), the residual covariance matrix for \(X,Y\) has diagonal entries 0.66 and 0.80 and determinant \((0.66)(0.80)-0.29^2=0.4439>0\). Together with the identity correlation matrix for the two controls, this gives a positive-definite correlation matrix.
Why Residualization Gives the Partial Correlation
The residual definition is also a practical identity: one can calculate a partial correlation by fitting two regressions and correlating their residuals. The algebra below makes the role of the conditioning variables explicit. Let the columns of \(Z\) include an intercept and the controls, and assume \(Z\) has full column rank. Let \(P_Z=Z(Z^\mathsf{T}Z)^{-1}Z^\mathsf{T}\) be the projection onto those columns and \(M_Z=I-P_Z\) the residual-maker matrix.
Proof. The least-squares residual vectors from the two regressions are \(e_x=M_Zx\) and \(e_y=M_Zy\). Because \(M_Z\) is symmetric and idempotent, \(e_x^\mathsf{T}e_y=x^\mathsf{T}M_Z^\mathsf{T}M_Zy=x^\mathsf{T}M_Zy\). Similarly, \(e_x^\mathsf{T}e_x=x^\mathsf{T}M_Zx\) and \(e_y^\mathsf{T}e_y=y^\mathsf{T}M_Zy\). The residuals are orthogonal to the intercept, so their sample means are zero. Their ordinary sample correlation is therefore their inner product divided by the square root of the product of their sums of squares, which gives the stated expression. Nonzero residual vectors ensure that the denominator is positive. \(\square\)
This formula also clarifies why a conditioning set matters: changing the columns of \(Z\) changes the projection, the residuals, and therefore the partial correlation. If a control is nearly a linear combination of other controls, the projection may be numerically unstable; in that case, report the issue rather than treating the calculated value as automatically reliable.
Check Linearity and Influence
A correlation summarizes linear association. Before interpreting it, inspect the relevant scatterplots and regression diagnostics. For a partial correlation, examine the relationship between the residualized \(X\) and residualized \(Y\), as well as residual-versus-fitted plots from the adjusted regression. For a multiple correlation, inspect the outcome against fitted values and examine residual plots for structure. These displays can reveal curvature, changing spread, clusters, or observations that have disproportionate influence.
Worked Example: A Residual Plot That Flags Curvature
Consider a simple fitted line with predictor values \(x=(-2,-1,0,1,2)\) and residuals \(e=(2,-1,-2,-1,2)\). The residuals sum to zero, and their inner product with the centered predictor is \((-2)(2)+(-1)(-1)+(0)(-2)+(1)(-1)+(2)(2)=0\). Thus these residuals are compatible with least-squares fitting of an intercept and a straight-line term.
Ordered by fitted value, the residuals are positive at both ends and negative in the middle. A residual-versus-fitted plot would display a U-shaped pattern rather than an unstructured cloud around zero. The diagnostic is not a reason to calculate a different correlation until it looks favorable; it is evidence that a straight-line summary may not describe the observed relationship adequately. The partial or multiple correlation can still be reported as a linear summary, but the pattern and its consequence for interpretation should be stated.
Worked Example: Sensitivity to One High-Leverage Observation
Suppose five residualized predictor-outcome pairs are \((-1,-1),(-1,1),(1,-1),(1,1),(4,4)\). Their sums are \(4\) for each coordinate, their sums of squares are \(20\) for each coordinate, and their cross-product sum is \(16\). The centered cross-product is \(16-(4)(4)/5=12.8\), and each centered sum of squares is \(20-4^2/5=16.8\). Hence their correlation is
Now omit the fifth pair and recalculate. The first four \(x\)-values and \(y\)-values each sum to zero, their cross-product sum is \((-1)(-1)+(-1)(1)+(1)(-1)+(1)(1)=0\), and each sum of squares is 4. Their correlation is therefore \(0/\sqrt{4\cdot4}=0\). The fifth point has substantially changed the result; its extreme predictor value gives it high leverage in a linear analysis.
This deletion calculation is a sensitivity check, not a rule that the observation should be discarded. Verify unusual values against the source data, explain any exclusion using a defensible criterion, and report how conclusions change. In an applied regression analysis, leverage and influence diagnostics can help locate observations that merit investigation; no single cutoff replaces examining the data and the scientific context.
Uncertainty and Interpretation
A sample partial correlation is an estimate, so report its uncertainty as well as its point value. Under independent sampling from a multivariate normal distribution, with \(q\) controls and a population partial correlation away from the boundary values, a commonly used Fisher-\(z\) approximation is \( \operatorname{atanh}(r_{XY\cdot Z})\) approximately normal with standard error \(1/\sqrt{n-q-3}\). A confidence interval on that transformed scale is converted back using the hyperbolic tangent. The approximation depends on the sampling and distributional assumptions; it does not account for a poor linear fit, influential observations, or selecting the controls after examining the results.
Worked Example: Reporting an Approximate Interval
Return to the partial correlation \(r=0.399\), with \(n=80\) observations and \(q=2\) controls. The approximate standard error on the Fisher-\(z\) scale is
The transformed estimate is \(\operatorname{atanh}(0.399)\approx0.423\). A 95% interval on the transformed scale is \(0.423\pm1.96(0.1155)\), or approximately \((0.197,0.649)\). Applying the increasing inverse transformation \(\tanh\) to each endpoint gives an approximate interval of \((0.194,0.571)\) for the population partial correlation.
A suitable report would say: “Among the 80 observed participants, the sample partial correlation between physical activity and systolic blood pressure, controlling linearly for age and baseline body-mass index, was 0.399; the approximate 95% confidence interval was 0.194 to 0.571.” This statement identifies the variables, controls, estimate, and uncertainty. It should also mention any substantial diagnostic concerns.
A conditional correlation is not a causal effect. In particular, adjusting for age and baseline body-mass index does not establish that all causes of activity and blood pressure have been controlled, nor does it establish the correct direction of influence. The result describes a linear association under the specified adjustment. A causal conclusion would require a separate design and assumptions that this correlation calculation does not supply.
A Practical Workflow and Common Pitfalls
Name the outcome, the variable of interest, and the target population. State whether the question is marginal or conditional.
Choose and justify the conditioning set based on the question and study context. Record the set because a different set defines a different partial correlation.
Compute the partial correlation from residuals and the multiple correlation from the fitted model. Check that the model is estimable and residual variation remains.
Review scatterplots, residual-versus-fitted plots, and influential-observation diagnostics. Investigate unusual cases and assess sensitivity without automatic deletion.
Give the estimate, controls, uncertainty, and important diagnostic findings. Interpret the result as an association, not a causal effect.
- Choosing controls to get a preferred answer. Adding or removing controls changes the quantity being summarized. Select them for the question, not for the resulting p-value or correlation.
- Reading multiple correlation as signed. Multiple correlation is nonnegative; it summarizes the joint linear association of an outcome with a predictor set. Use coefficients or partial correlations for signed directions.
- Assuming a large sample cures bad diagnostics. More observations do not make a curved relationship linear, or make one influential observation harmless. Examine the case-level data and report concerns.
- Calling adjustment causal. A partial correlation is conditional on the specified variables and linear projection. It does not, on its own, identify what an intervention would do.
Check Your Understanding
Use these questions to review the workflow and its interpretation.
- What does the sample partial correlation between \(X\) and \(Y\) controlling for \(Z\) equal, in terms of two least-squares regressions?
- In the worked correlation-matrix example, why does adding \(X\) to the model with \(Z_1,Z_2\) increase \(R^2\) by \(0.29^2/0.66\)?
- What feature of a residual-versus-fitted plot would make a straight-line summary questionable?
- Why is a large change in correlation after omitting one observation a reason to investigate, rather than an automatic reason to exclude that observation?
- What assumptions underlie the approximate Fisher-\(z\) interval for a partial correlation, and what important diagnostic problems does it not address?
- Why does controlling for measured covariates not, by itself, turn a partial correlation into a causal effect?