Separate Interaction from Correlation Between Predictors

Two Different Questions About Two Predictors

When a regression model includes two quantitative predictors, it is easy to confuse two different features of the data. Interaction asks whether the fitted relationship between one predictor and the response changes as the other predictor changes. Correlation between predictors asks whether their observed values tend to vary together. The first concerns the fitted conditional mean of the response; the second concerns the joint pattern of the predictors.

These features can occur separately or together. Predictors can be strongly correlated even when the fitted response relationship has no interaction. They can have little or no correlation and still have an interaction. A useful way to keep the ideas separate is to inspect the fitted equation for how the conditional slope varies, then inspect the paired predictor values for correlation.

The Model and the Two Definitions

For observations \(i=1,\ldots,n\), consider the linear regression model with a product term:

Y_i=\beta_0+\beta_1X_i+\beta_2Z_i+\beta_3X_iZ_i+\epsilon_i

Here \(Y_i\) is the response, \(X_i\) and \(Z_i\) are quantitative predictors, \(\beta_0,\beta_1,\beta_2,\beta_3\) are coefficients, and \(\epsilon_i\) is the error. For interpreting the fitted mean, the model assumes that the conditional mean of \(Y\), given \(X\) and \(Z\), follows the stated equation and that the errors have mean zero given the predictors. For standard regression inference, additional assumptions such as independent errors and constant error variance are needed; normal errors are commonly assumed for exact small-sample \(t\)- and \(F\)-procedures. A fitted equation alone does not establish that these assumptions are reasonable: residual checks and the study design matter.

Definition: In this model, there is a fitted interaction between \(X\) and \(Z\) when the conditional slope of the fitted response with respect to one predictor changes across values of the other predictor. The predictors are correlated when their observed values tend to vary together, as summarized, for example, by their sample correlation \(r_{XZ}\). These are distinct properties.

To see the interaction directly, hold \(Z=z\) fixed. The fitted response as \(X\) varies is

\hat{Y}=b_0+b_1X+b_2z+b_3Xz,

where \(b_0,b_1,b_2,b_3\) are the estimated coefficients. Collecting the terms involving \(X\) gives a fitted slope of \(b_1+b_3z\). Thus, if \(b_3\) is nonzero, this slope changes with \(z\). By contrast, \(r_{XZ}\) is calculated from the paired observed predictor values; it does not appear in this conditional-slope calculation.

A Short Derivation Using Finite Changes

The same distinction can be made without relying only on the word “slope.” Compare two values \(x_1\) and \(x_2\) of \(X\), while holding \(Z=z\) fixed. Subtract the fitted response at \(x_1\) from the fitted response at \(x_2\):

\begin{aligned} \hat{Y}(x_2,z)-\hat{Y}(x_1,z) &=\{b_0+b_1x_2+b_2z+b_3x_2z\}\\ &\quad-\{b_0+b_1x_1+b_2z+b_3x_1z\}\\ &=(x_2-x_1)(b_1+b_3z). \end{aligned}

The fitted change depends on \(z\) through \(b_3z\). If \(b_3=0\), then the fitted change is \(b_1(x_2-x_1)\) at every value of \(z\): the fitted relationship is additive, with no change in the conditional \(X\)-slope. Neither outcome depends on whether \(X\) and \(Z\) are correlated in the observed data.

Worked Examples

Worked Example: Correlated Predictors Without a Fitted Interaction

In a hypothetical dataset, four observed predictor pairs are \((X,Z)=(1,1),(2,3),(3,2),(4,4)\). An additive regression fit to the associated response values gives \(\hat{Y}=10+2X+3Z\). The observed predictor pairs and response values are shown below.

\(X\)	\(Z\)	\(Y\)
1	1	16
2	3	22
3	2	21
4	4	31

First calculate the sample correlation. Both predictor means are \(2.5\). The centered \(X\) values are \(-1.5,-0.5,0.5,1.5\), and the centered \(Z\) values are \(-1.5,0.5,-0.5,1.5\). Therefore,

r_{XZ}= \frac{(-1.5)(-1.5)+(-0.5)(0.5)+(0.5)(-0.5)+(1.5)(1.5)} {\sqrt{(1.5^2+0.5^2+0.5^2+1.5^2)(1.5^2+0.5^2+0.5^2+1.5^2)}} =\frac{4}{5}=0.800.

The observed predictors are strongly positively correlated in this small dataset. Now check the fitted conditional \(X\)-slope. There is no product term, so \(b_3=0\) and the slope is \(2\) response units per \(X\)-unit at every fixed value of \(Z\). For example, changing \(X\) from 1 to 3 at \(Z=2\) changes the fitted response by \(2(3-1)=4\). At \(Z=4\), the same change also gives \(2(3-1)=4\). Correlation is present, but the fitted equation has no interaction.

The equation is consistent with the displayed outcomes: its fitted values are \(15,23,22,30\), giving residuals \(1,-1,-1,1\). These residuals sum to zero and have zero cross-products with each predictor: \(1(1)-1(2)-1(3)+1(4)=0\) for \(X\), and \(1(1)-1(3)-1(2)+1(4)=0\) for \(Z\). Thus the displayed additive least-squares fit has the stated coefficients. This example illustrates the distinction; it is too small to support reliable inference about a broader population.

Worked Example: An Interaction With Uncorrelated Predictors

A hypothetical dataset contains observations at the four combinations of \(X=-1\) or \(1\) and \(Z=-1\) or \(1\), with each combination observed twice. Suppose the fitted equation is \(\hat{Y}=20+2X+3Z+4XZ\), and assume the model form and residual checks are reasonable for the intended interpretation. Because the design is balanced, each predictor has mean zero, and each combination occurs equally often. The products \(XZ\) are \(1,-1,-1,1\), so their sum is zero; hence the sample correlation between \(X\) and \(Z\) is zero.

Despite that zero correlation, the fitted conditional slope of \(Y\) with respect to \(X\) is \(2+4Z\). At \(Z=-1\), it is \(2+4(-1)=-2\) response units per \(X\)-unit. At \(Z=1\), it is \(2+4(1)=6\) response units per \(X\)-unit. For a change in \(X\) from \(-1\) to \(1\), the fitted response change at \(Z=-1\) is \((-2)(1-(-1))=-4\) response units. At \(Z=1\), it is \(6(1-(-1))=12\) response units.

The fitted changes differ, so this model has an interaction. The predictor values are uncorrelated in the stated balanced design, but that does not remove the changing conditional slope. Correlation and interaction answer different questions.

Worked Example: Correlation and Interaction Together

In a third hypothetical dataset, the observed predictor pairs are \((0,0),(1,1),(2,1),(3,3),(4,5)\). A fitted model, assumed to be an adequate description over the predictor range used, is \(\hat{Y}=5+X+2Z+0.5XZ\). The observed \(X\) and \(Z\) means are both 2. The centered values are \((-2,-1,0,1,2)\) for \(X\) and \((-2,-1,-1,1,3)\) for \(Z\). Thus,

r_{XZ}= \frac{4+1+0+1+6}{\sqrt{(4+1+0+1+4)(4+1+1+1+9)}} =\frac{12}{\sqrt{160}} \approx 0.949.

The observed predictors are strongly positively correlated. Separately, the fitted conditional \(X\)-slope is \(1+0.5Z\). At \(Z=0\), this slope is 1 response unit per \(X\)-unit, so increasing \(X\) from 1 to 3 gives a fitted change of \(1(3-1)=2\) response units. At \(Z=4\), the slope is \(1+0.5(4)=3\), and the same \(X\) change gives \(3(3-1)=6\) response units. Both comparison values of \(Z\) fall within the observed range from 0 to 5.

Here both features occur: the predictors are correlated, and the fitted \(X\)-slope changes with \(Z\). The correlation does not itself create or quantify the fitted slope change; the product term does that in the model. Because the predictors tend to vary together, separating their contributions may be less precise than it would be with more independent variation. The numerical comparisons describe fitted associations, not causal effects.

Why the Distinction Matters

A correlation describes how the predictor values are distributed together in the observed sample. An interaction describes how the model’s fitted response relationship varies across predictor values. It is possible for the predictors to have a large positive or negative correlation while the fitted interaction coefficient is zero. It is also possible for the predictors to be uncorrelated while the fitted interaction coefficient is not zero. Neither quantity is a substitute for the other.

Predictor correlation still matters for analysis. If predictors move together, the data may contain limited information about their separate contributions, which can make coefficient estimates less precise. That is a concern about what the design lets us estimate clearly. It does not change the definition of interaction, and a high predictor correlation is not evidence by itself that the conditional slope changes.

When assessing a fitted model, make the two checks separately:

Use the fitted equation to calculate the conditional slope, or a finite fitted change, at different values of the other predictor. A changing value indicates a fitted interaction.
Use the observed paired predictor values to assess their correlation. This describes how the predictors vary together in the sample.

Interpret these comparisons only where the predictor values are supported by the data, and consider model fit and residual assumptions before drawing conclusions. A fitted interaction is an association in the specified model; it does not, on its own, show that changing one predictor would cause a change in the response.

Common Mistake

Wrong reasoning: “The predictors are highly correlated, so they must interact.” This treats a pattern among predictor values as if it were a change in the response relationship. Correlation alone gives no such conclusion.

Correction: Hold one predictor fixed in the fitted equation and calculate the slope or finite change for the other predictor. Repeat at another value of the fixed predictor. A difference between those fitted slopes or changes is the relevant evidence of an interaction in the model. Then report predictor correlation separately as a feature of the observed data.

Key takeaway: Interaction is about a changing conditional slope in the fitted response model. Predictor correlation is about how the predictor values vary together. They can occur separately or simultaneously, so examine the fitted equation and the predictor pairs as two distinct pieces of information.

Check Your Understanding

For each question, distinguish what the fitted equation says from what the predictor values say.

A fitted model is \(\hat{Y}=8+3X+2Z\). What is the fitted slope of \(Y\) with respect to \(X\) at \(Z=0\) and at \(Z=5\)? Is there a fitted interaction?
A sample has \(r_{XZ}=-0.85\), and a fitted model has a product coefficient of zero. What does the correlation describe, and what does the fitted model say about the conditional \(X\)-slope?
For \(\hat{Y}=4+2X-Z+0.25XZ\), calculate the fitted \(X\)-slope at \(Z=0\) and \(Z=4\). Does the value of \(r_{XZ}\) determine either slope?
Can two predictors have zero sample correlation and a fitted interaction? Explain using the conditional-slope formula.
Why is “the predictors are correlated, so they interact” incorrect reasoning?