Centering and Collinearity in Interaction Models

Why Centering Matters for Interaction Models

An interaction model includes a product of predictors, such as \(XZ\), to represent the possibility that the association between one predictor and the outcome varies with the other. The product can be strongly correlated with one or both predictors, especially when predictors are measured far from zero. This can make individual coefficient estimates less precise, even when the model's overall fitted values are useful.

Centering changes the reference point of a predictor: for example, \(X_c=X-\bar{x}\). In an interaction model, this changes the product column as well as the main-effect columns. It may reduce some correlations that arise from the arbitrary location of zero, but it does not guarantee that all correlations will fall. In particular, subtracting means cannot change the correlation between two predictors.

The Interaction Model and Its Assumptions

Consider the multiple regression model

Here \(Y_i\) is the outcome for observation \(i\), \(X_i\) and \(Z_i\) are predictors, and \(\varepsilon_i\) is the error. The coefficient \(\beta_3\) describes the interaction: the conditional slope of \(X\) is \(\beta_1+\beta_3Z\). For the usual least-squares coefficient estimates and their standard inferential procedures, assume the model's mean is correctly specified, the errors have conditional mean zero, and observations' errors are independent with constant variance. Normal errors are additionally assumed for exact small-sample \(t\)- and \(F\)-procedures. Collinearity is about the predictor columns; it is not itself one of these error assumptions.

If \(X\) is replaced by \(X_c=X-a\) and \(Z\) by \(Z_c=Z-b\), where \(a\) and \(b\) are fixed centering constants, then

Thus, the centered interaction column is the original product column plus a combination of the intercept and main-effect columns. In a model containing the intercept, both main effects, and the interaction, this re-expression does not change the set of fitted values that the model can produce. It changes the coefficient interpretation and can change correlations among the columns.

How a Product Column Changes

The formula also explains why centering sometimes helps. If \(X\) is already centered and only \(Z\) is shifted by \(b\), then \(X_cZ_c=X_cZ-bX_c\). The new product column differs from the old one by a multiple of the \(X_c\) column. Correlation between a predictor and product can therefore change substantially. But the amount and direction depend on the observed data; there is no rule that the new correlation must be closer to zero.

Pearson correlation is computed from deviations from each column's mean. If the product column has mean \(\overline{XZ}\), its correlation with \(X\) uses \(XZ-\overline{XZ}\), not the uncentered product values as if they had mean zero. The examples below make this distinction explicit.

Compute Correlations Using Deviations

The next example illustrates two important checks: a product column need not have mean zero, even if one predictor has been centered, and the correlation denominator must use the product column's sum of squared deviations. The data are hypothetical.

Centering Cannot Undo Predictor Dependence

For any two variables, adding or subtracting constants changes their means but not their deviations from those means. Therefore their Pearson correlation is unchanged by centering. If predictors are strongly associated because of their observed values, changing their origins cannot remove that association. This is the distinction between a correlation involving an interaction column that may partly reflect the chosen zero and dependence among the substantive predictors themselves.

What Correlations Can—and Cannot—Tell You

A correlation table can be a useful first check: examine the association between the two predictors and between each predictor and the product term, both before and after centering. Interpret it as a description of pairwise linear association in the observed data, not as a complete diagnosis of the design matrix. Several columns can jointly be nearly dependent even when no single pair has an extreme correlation.

Collinearity does not, by itself, make least-squares estimates biased when the regression model and error assumptions hold. With strong but imperfect collinearity, however, estimates can be imprecise and sensitive to small data changes. With exact linear dependence among columns, the separate coefficients are not uniquely estimable. Centering may improve the numerical relationship between a product and a main effect, but it cannot create information absent from the observations.

Keep the model's purpose in view. If the interaction is part of the scientific question, do not drop it solely because a product column is correlated with a main effect. Such a change alters the model. Instead, use centering when a meaningful reference point helps interpretation or improves a nonessential correlation, and report which variables and product terms were centered.

Common Mistake: Assuming Centering Always Fixes Collinearity

A common but incorrect argument is: “The predictors have been centered, so the interaction column must be uncorrelated with the main effects.” Centering makes each predictor's mean zero; it does not impose zero correlation between a predictor and a product. The product can have a nonzero mean, and its deviations can still align strongly with a predictor's deviations. Calculate correlations from centered deviations, and check the observed values rather than relying on a general promise about centering.

A related error is to treat \(X\) and \(Z\) as independent simply because both have been centered. Their mutual correlation is unchanged by subtracting constants. Centering changes the origin, not the pattern of paired observations.