Incremental R-Squared from Partial Correlation

Why Partial Correlation and Incremental R-Squared Belong Together

A multiple regression model may already explain a substantial part of the response before a new predictor is added. The new predictor’s contribution should therefore be understood relative to what remains unexplained, not relative to the response’s total variation alone. Partial correlation expresses the strength of the association with that remaining response variation; incremental R-squared expresses how much the model’s ordinary R-squared increases.

These two quantities are linked by an exact identity for nested least-squares models fitted to the same observations. The identity also explains a frequent misinterpretation: the square of the partial correlation is not usually the raw increase in R-squared. It is the fraction of the reduced model’s remaining error sum of squares removed by adding the predictor.

Reduced and Full Models

Consider a response vector $y$ observed on $n$ cases, a matrix $C$ containing an intercept and the existing controls, and a candidate predictor vector $x$. The reduced model regresses $y$ on the columns of $C$. The full model regresses $y$ on the columns of $C$ and on $x$. Both models use the same observations and the same response.

Write $P_C$ for the orthogonal projection onto the column space of $C$, and $M_C=I-P_C$ for the residual-making projection. The reduced-model residuals for the response and candidate predictor are, respectively, $e_y=M_Cy$ and $e_x=M_Cx$. Their ordinary sample correlation is the partial correlation between $y$ and $x$ controlling for the variables in $C$.

Definition: Suppose the reduced model contains an intercept, the response is not constant, $e_y\ne 0$, and $e_x\ne 0$. Define the reduced-model error sum of squares by $\mathrm{SSE}_R=e_y'e_y$, and the total sum of squares by $\mathrm{SST}=\sum_{i=1}^n(y_i-\bar y)^2$. The reduced-model coefficient of determination is $R_R^2=1-\mathrm{SSE}_R/\mathrm{SST}$. Define $r_{yx\cdot C}$ as the sample correlation between $e_y$ and $e_x$. The incremental R-squared is $\Delta R^2=R_F^2-R_R^2$, where $R_F^2$ is the full-model R-squared.

The controls do not have to be linearly independent: the projection onto their column space is still well-defined. The essential additional-predictor condition is that $x$ is not in that column space, so $e_x\ne 0$. If the controls already reproduce $x$ exactly, adding it cannot improve the fit and the partial correlation is undefined because its residual has zero length.

The Exact Incremental R-Squared Identity

After the controls have been fitted, adding $x$ is equivalent to projecting the reduced response residual $e_y$ onto the residualized predictor $e_x$. This one-dimensional projection gives the improvement in fit directly.

Theorem: Under the conditions in the definition, for ordinary least squares with an intercept and the same observations in both models, $$ \Delta R^2=(1-R_R^2)\,r_{yx\cdot C}^{\,2}. $$ Equivalently, $r_{yx\cdot C}^{\,2}=\Delta R^2/(1-R_R^2)$. The denominator is positive because $e_y\ne 0$.

To prove the result, first note that any coefficient adjustment to the controls can account for the component of $y$ in the column space of $C$, but it cannot change the remaining residual $e_y$. The additional fitted component must lie along $e_x$, which is orthogonal to the controls. Its least-squares coefficient is $e_x'e_y/(e_x'e_x)$.

\widehat{\gamma}=\frac{e_x'e_y}{e_x'e_x}, \qquad \mathrm{SSE}_F =\left(e_y-e_x\widehat{\gamma}\right)' \left(e_y-e_x\widehat{\gamma}\right) =e_y'e_y-\frac{(e_x'e_y)^2}{e_x'e_x}.

The final equality follows by expanding the quadratic and substituting $\widehat{\gamma}$: the two cross-product terms together subtract $2(e_x'e_y)^2/(e_x'e_x)$, while the squared fitted term adds back $(e_x'e_y)^2/(e_x'e_x)$. Thus the reduction in error sum of squares is $(e_x'e_y)^2/(e_x'e_x)$. By the definition of the partial correlation,

r_{yx\cdot C}^{\,2} =\frac{(e_x'e_y)^2}{(e_x'e_x)(e_y'e_y)}.

Since the models use the same response and observations, their total sum of squares is the same. Therefore,

\Delta R^2 =R_F^2-R_R^2 =\frac{\mathrm{SSE}_R-\mathrm{SSE}_F}{\mathrm{SST}} =\frac{(e_x'e_y)^2}{(e_x'e_x)\mathrm{SST}} =r_{yx\cdot C}^{\,2}\frac{e_y'e_y}{\mathrm{SST}} =r_{yx\cdot C}^{\,2}(1-R_R^2).

This proves the identity. It shows precisely what the square of a partial correlation measures: the proportion of the reduced model’s residual sum of squares removed by adding $x$. Multiplying by the fraction of total variation still unexplained, $1-R_R^2$, converts that proportion into an increase on the ordinary R-squared scale. $\square$

Worked Calculations

Worked Example: From Partial Correlation to the R-Squared Increase

A reduced model has $R_R^2=0.40$. The partial correlation of a candidate predictor with the response, controlling for the existing predictors, is $r_{yx\cdot C}=0.50$. Find the incremental and full-model R-squared.

The reduced model leaves $1-0.40=0.60$ of the total response variation unexplained. The partial correlation squared is $0.50^2=0.25$, so the new predictor removes 25% of that remaining variation. Hence

$\Delta R^2=0.60(0.25)=0.15$, and $R_F^2=0.40+0.15=0.55$.

The calculation can also be checked with sums of squares. Take $\mathrm{SST}=100$, so $\mathrm{SSE}_R=60$. If $e_x'e_x=4$, then $e_y'e_y=60$ and the partial correlation condition gives $e_x'e_y=0.50\sqrt{4\cdot60}=\sqrt{60}$. The SSE reduction is $(\sqrt{60})^2/4=15$, leaving $\mathrm{SSE}_F=45$ and $R_F^2=1-45/100=0.55$.

Worked Example: A Negative Partial Correlation

Suppose a reduced model has $R_R^2=0.72$, while the candidate predictor’s partial correlation is $-0.30$. Calculate the increase and the full-model R-squared.

The sign records the direction of the association, but the amount of fit improvement depends on the square:

$\Delta R^2=(1-0.72)(-0.30)^2=0.28(0.09)=0.0252$.

Thus $R_F^2=0.72+0.0252=0.7452$. To verify the scale, let $\mathrm{SST}=250$. Then $\mathrm{SSE}_R=0.28(250)=70$. For any nonzero $e_x$, the reduction in SSE implied by partial correlation $-0.30$ is $0.09(70)=6.3$; therefore $\mathrm{SSE}_F=63.7$ and $1-63.7/250=0.7452$. The negative sign does not imply a negative increase in R-squared.

Worked Example: Recovering the Signed Partial Correlation

The reduced model has $R_R^2=0.25$, and the full model has $R_F^2=0.40$. The fitted coefficient for the added predictor, after accounting for the controls, is negative. Find the partial correlation.

First calculate the increase: $\Delta R^2=0.40-0.25=0.15$. The reduced model’s unexplained fraction is $1-0.25=0.75$. Therefore,

$r_{yx\cdot C}^{\,2}=0.15/0.75=0.20$, so the magnitude of the partial correlation is $\sqrt{0.20}\approx0.4472$.

The sign agrees with the direction of the added predictor’s regression coefficient: the residualized response and predictor have a negative association. Thus $r_{yx\cdot C}=-\sqrt{0.20}\approx-0.4472$. Squaring alone would recover the magnitude but not the direction.

Population Version

The same reasoning applies to population linear projections. Let $Y$ and $X$ have finite second moments, and let the controls generate a finite-dimensional linear space that includes the constant random variable. Project $Y$ and $X$ onto this space, and call the residuals $U$ and $V$. Assume $\operatorname{Var}(Y)>0$, $\operatorname{Var}(U)>0$, and $\operatorname{Var}(V)>0$. Define each population $R^2$ as one minus its projection residual variance divided by $\operatorname{Var}(Y)$, and define the partial correlation as $\operatorname{Corr}(U,V)$.

Because $V$ is orthogonal to the controls, the full projection adds a multiple of $V$ to the reduced projection. For a coefficient $b$, the remaining response residual is $U-bV$. Expanding its expected square and minimizing over $b$ gives

E[(U-bV)^2] =E[U^2]-2bE[UV]+b^2E[V^2], \qquad b_{\mathrm{opt}}=\frac{E[UV]}{E[V^2]}.

Substitution shows the reduction in residual variance is $\operatorname{Cov}(U,V)^2/\operatorname{Var}(V)$, since the residuals have mean zero under projection onto a space containing the constant. Dividing this gain by $\operatorname{Var}(Y)$, and using $\operatorname{Var}(U)/\operatorname{Var}(Y)=1-R_R^2$, yields

\Delta R^2 =\frac{\operatorname{Cov}(U,V)^2} {\operatorname{Var}(V)\operatorname{Var}(Y)} =\operatorname{Corr}(U,V)^2 \frac{\operatorname{Var}(U)}{\operatorname{Var}(Y)} =(1-R_R^2)\,r_{YX\cdot C}^{\,2}.

This is a population projection identity; it does not require a sampling distribution or a significance test. Its sample counterpart follows from the corresponding finite-dimensional least-squares projections.

Interpretation and Common Pitfalls

The size of the increase depends on both the partial correlation and the reduced model’s remaining unexplained variation. Even a strong partial correlation can yield a modest increase in ordinary R-squared when the controls already explain nearly all the response variation. Conversely, the same partial correlation yields a larger increase when the reduced model leaves more variation unexplained.

Wrong: “The partial correlation is 0.40, so R-squared increases by 0.16.” This treats the squared partial correlation as a fraction of total variation. Correction: Multiply $0.16$ by $1-R_R^2$. If the reduced $R^2$ is $0.50$, the increase is $0.50(0.16)=0.08$.
Wrong: “A negative partial correlation means adding the predictor decreases R-squared.” Correction: In ordinary least squares, adding a predictor cannot increase SSE, and the R-squared increment is proportional to the squared partial correlation. The sign gives direction, not the sign of the increment.
Wrong: “The formula applies if I compare models fitted to different observations.” Correction: The derivation uses the same response vector and total sum of squares in both models. If missingness changes the analysis sample, this identity does not directly compare the two reported R-squared values.
Wrong: “This is the increase in adjusted R-squared.” Correction: The identity concerns ordinary R-squared. Adjusted R-squared includes a degrees-of-freedom penalty, so its change is not given by this formula.

Incremental R-squared is a descriptive measure of improvement in in-sample least-squares fit. By itself, it does not establish that the predictor’s association is statistically significant, causal, stable in a new sample, or practically important. Those conclusions require additional assumptions and analyses.

Key takeaway: For a predictor added to a reduced least-squares model, its partial correlation squared is the fraction of the reduced model’s residual variation removed. Its raw increase in ordinary R-squared is that fraction multiplied by $1-R_R^2$.

Check Your Understanding

Use the nested-model identity to answer these questions.

A reduced model has $R_R^2=0.30$, and the added predictor has partial correlation $0.20$. What are $\Delta R^2$ and $R_F^2$?
If the partial correlation is $-0.60$, what does its sign tell you, and what does its square contribute to the fit calculation?
A reduced model has $R_R^2=0.80$ and a partial correlation of $0.50$. Why is the increase smaller than $0.50^2$?
The reduced and full models have R-squared values $0.45$ and $0.56$. Find the squared partial correlation and its magnitude.
Name two conditions needed for the sample identity to apply to a comparison of reported ordinary R-squared values.