Why Include a Predictor Product?
An additive multiple-regression model assigns each predictor a constant fitted effect, holding the other predictors fixed. That may be too restrictive. For example, the fitted association between a hypothetical building’s insulation level and heating use might differ by outdoor temperature. A product of the predictors lets the fitted effect of one vary with the value of the other.
An interaction is represented in the regression calculation by a product column in the design matrix. It is not a product of the response and a predictor, and it does not mean that the observed response itself is multiplied by the predictors.
For estimation by ordinary least squares, each row of the design matrix contains a 1 for the intercept, the observed \(x_1\), the observed \(x_2\), and the product \(x_1x_2\). The fitted response at specified predictor values is obtained by multiplying each model coefficient by its corresponding column value and adding the results.
The model describes the conditional mean when the error has conditional mean zero, \(\operatorname{E}[\varepsilon\mid x_1,x_2]=0\). The least-squares coefficients are uniquely determined when the design matrix has full column rank. If its columns are linearly dependent, the fitted values may still be determined, but the individual coefficients need not be unique. The model’s fitted values are not automatically causal effects; an interaction in a regression model describes a changing fitted association unless additional causal assumptions are justified.
How the Product Changes a Fitted Effect
The interaction coefficient does not, by itself, give the fitted effect of \(x_1\) at every predictor setting. To find the change in the fitted mean associated with changing \(x_1\), hold \(x_2\) fixed and compare the two fitted means.
For any nonzero change \(h\), the fitted change per unit of \(x_1\) is therefore \(\beta_1+\beta_{12}x_2\). This is a direct algebraic consequence of the product column: increasing \(x_1\) by \(h\) increases \(x_1x_2\) by \(hx_2\). In the fitted model, the effect per unit of \(x_1\) varies with \(x_2\) when \(\beta_{12}\ne 0\). Reversing the roles of the predictors gives the fitted effect per unit of \(x_2\), \(\beta_2+\beta_{12}x_1\).
The product term also describes departure from additivity. If \(x_1\) increases by \(h\) and \(x_2\) increases by \(k\), the change in the fitted mean beyond the sum of the separate changes is \(\beta_{12}hk\). Thus, with \(\beta_{12}=0\), the fitted model is additive: the fitted effect of each predictor does not depend on the other predictor.
Worked Examples
Worked Example: Building a Product Column and Finding a Fitted Value
A hypothetical analyst models a response \(Y\), measured in dollars, using two quantitative predictors: \(x_1\), measured in tens of operating hours, and \(x_2\), measured in units of workload. The fitted model is \(\widehat{y}=5+2x_1+3x_2+0.5x_1x_2\). For one case, \(x_1=2\) and \(x_2=4\). Construct its product-column value, calculate the fitted response, and state the interaction’s contribution.
Method. First calculate the product for the design-matrix row: \(x_1x_2=2(4)=8\). The row is \((1,2,4,8)\).
Substitute all four entries into the fitted equation: \(\widehat{y}=5+2(2)+3(4)+0.5(8)\). The terms are \(5\), \(4\), \(12\), and \(4\), respectively, so \(\widehat{y}=5+4+12+4=25\) dollars. The interaction contributes \(0.5(8)=4\) dollars to this fitted value.
Check. Omitting the product term would give \(5+4+12=21\) dollars. Adding the interaction contribution of \(4\) gives \(25\) dollars, as calculated. The common mistake avoided here is to calculate the product correctly but then leave its coefficient-weighted contribution out of the fitted value.
Worked Example: Recovering a Main-Effect Coefficient
A hypothetical fitted model is \(\widehat{y}=8+b_1x_1+2x_2+0.25x_1x_2\). At \(x_2=8\), the fitted change per unit of \(x_1\) is \(5\). Find \(b_1\), then calculate the fitted response at \(x_1=2,\ x_2=8\).
Method. In this model the fitted effect per unit of \(x_1\) at \(x_2=8\) is \(b_1+0.25(8)\). Set it equal to the stated value: \(b_1+2=5\), so \(b_1=3\).
Now calculate the product-column value, \(x_1x_2=2(8)=16\), and substitute: \[ \widehat{y}=8+3(2)+2(8)+0.25(16) =8+6+16+4=34. \] The fitted response is \(34\).
Check. At \(x_2=8\), the model’s fitted effect per unit of \(x_1\) is \(3+0.25(8)=5\), matching the condition. The common mistake avoided is treating \(b_1\) as the effect at \(x_2=8\); it is the effect at \(x_2=0\).
Worked Example: Comparing an Interaction Model With an Additive Model
For a hypothetical response, the fitted interaction model is \(\widehat{y}=20+1.5x_1+2x_2+0.5x_1x_2\). At \(x_1=4,\ x_2=6\), calculate the fitted response and compare it with the fitted value from the additive model that keeps the same intercept and main-effect coefficients but omits the product term.
Method. The product-column value is \(x_1x_2=4(6)=24\), so its contribution is \(0.5(24)=12\). The full interaction-model fitted response is \[ 20+1.5(4)+2(6)+0.5(24) =20+6+12+12=50. \]
The additive model omits only the product term. Its fitted response at the same predictor values is \(20+1.5(4)+2(6)=20+6+12=38\). The interaction model’s fitted response is \(50-38=12\) higher at this predictor setting.
Check. The difference between the two fitted values equals the interaction contribution, \(0.5(4)(6)=12\). The common mistake avoided is to add the product contribution incorrectly: it is \(12\), making the full fitted response \(50\), not \(47\).
Interpreting Coefficients and Product Units
An interaction coefficient must be interpreted together with the predictor values and units. In the model above, the fitted effect per unit of \(x_1\) is \(\beta_1+\beta_{12}x_2\); it increases or decreases as \(x_2\) changes, according to the sign of \(\beta_{12}\). A positive interaction coefficient means that the fitted effect of \(x_1\) increases as \(x_2\) increases. A negative coefficient means that it decreases. Neither statement alone says whether the fitted effect itself is positive at every value of \(x_2\).
The units of \(\beta_{12}\) are response units divided by the product of the units of \(x_1\) and \(x_2\). For instance, if \(Y\) is measured in dollars, \(x_1\) in tens of operating hours, and \(x_2\) in workload units, then \(\beta_{12}\) is measured in dollars per (ten operating hours times workload unit). Changing the measurement units changes the numerical coefficient, so a coefficient should be read in the model’s stated units.
The main effects are usually retained when a product is included. Without \(\beta_1x_1\), for example, the fitted effect per unit of \(x_1\) would be \(\beta_{12}x_2\), forcing that effect to be zero at \(x_2=0\). Retaining both main effects avoids imposing that restriction without a substantive reason. The choice of zero also matters for reading the main-effect coefficients: \(\beta_1\) describes the fitted effect of \(x_1\) when \(x_2=0\), which may or may not be a practically meaningful setting.
Common Mistakes
- Leaving out the product’s coefficient-weighted contribution. The product-column value is \(x_1x_2\), but its contribution to the fitted response is \(\beta_{12}x_1x_2\).
- Calling \(\beta_1\) the effect of \(x_1\) at all settings. The fitted effect at a specified \(x_2\) is \(\beta_1+\beta_{12}x_2\).
- Multiplying the response by a predictor. The interaction is a constructed predictor column; the response is not part of the product.
- Comparing models at different predictor values. To isolate the product’s contribution, compare the interaction and additive fitted values at the same values of \(x_1\) and \(x_2\).
- Ignoring units or the zero point. The main-effect coefficient refers to the other predictor being zero, and the interaction coefficient’s units depend on both predictor scales.
Check Your Understanding
For each problem, show the product-column calculation before using the fitted equation.
- A fitted model is \(\widehat{y}=12+1.5x_1+2x_2+0.25x_1x_2\). Find the product-column value and fitted response at \(x_1=4,\ x_2=6\). (Answer: product \(24\); fitted response \(36\))
- A model is \(\widehat{y}=9+3x_1-x_2+0.4x_1x_2\). At \(x_1=2,\ x_2=5\), find the interaction contribution and full fitted response. (Answer: contribution \(4\); fitted response \(14\))
- A fitted model has \(\beta_1=2.2\) and \(\beta_{12}=0.3\). Find the fitted effect per unit of \(x_1\) at \(x_2=4\), then find the fitted change when \(x_1\) increases by \(2\) with \(x_2\) held fixed. (Answer: effect \(3.4\) per unit; fitted change \(6.8\))
- A model is \(\widehat{y}=10+b_1x_1+1.5x_2+0.2x_1x_2\). The fitted effect per unit of \(x_1\) at \(x_2=5\) is \(4\). Find \(b_1\) and the fitted response at \(x_1=2,\ x_2=5\). (Answer: \(b_1=3\); fitted response \(25.5\))
- A fitted model is \(\widehat{y}=15+2x_1+1.25x_2+0.5x_1x_2\). At \(x_1=3,\ x_2=4\), find the interaction-model fitted response, the corresponding additive-model fitted response, and their difference. (Answer: interaction \(32\); additive \(26\); difference \(6\))