Two-Factor Interactions as Product Terms

From Separate Factor Effects to Product Columns

A model with two factors can describe each factor's contribution separately, or allow their joint combinations to contribute additional terms. In a generalized linear model (GLM), this choice is made in the linear predictor. The interaction columns are constructed directly from the factor coding columns: each is the product of one column for the first factor and one for the second.

The product construction is simple, but it has an important structural consequence. It adds columns for combinations of factor contrasts, not a new set of arbitrary labels. The terms added, and whether they can be estimated separately, depend on the coding and on which factor combinations are observed.

Definition: Suppose factor \(A\) is represented by coding columns \(x_{A1},\ldots,x_{Ar_A}\), and factor \(B\) by \(x_{B1},\ldots,x_{Br_B}\). For observation \(i\), the \(r_A r_B\) two-factor interaction columns are \(x_{Aik}x_{Bil}\), for \(k=1,\ldots,r_A\) and \(l=1,\ldots,r_B\). The interaction-augmented linear predictor includes coefficients for these product columns in addition to the intercept and the two sets of factor columns.

For a GLM with link function \(g\) and mean \(\mu_i=E[Y_i]\), the additive and interaction-augmented specifications are

g(\mu_i)=\eta_i =\beta_0+\sum_{k=1}^{r_A}\beta_{Ak}x_{Aik} +\sum_{l=1}^{r_B}\beta_{Bl}x_{Bil}

and

g(\mu_i)=\eta_i =\beta_0+\sum_{k=1}^{r_A}\beta_{Ak}x_{Aik} +\sum_{l=1}^{r_B}\beta_{Bl}x_{Bil} +\sum_{k=1}^{r_A}\sum_{l=1}^{r_B}\beta_{AB,kl}x_{Aik}x_{Bil}.

The first specification is additive on the linear-predictor scale. The second allows additional terms formed from the joint factor coding. “Additive” here describes \(\eta_i\); for a nonlinear link, it does not generally mean that the response mean is the sum of separate effects.

Worked Constructions

Worked Example: Two Binary Factors With Reference Coding

Let \(A\) have levels \(A_0,A_1\), and \(B\) have levels \(B_0,B_1\). Use \(A_0\) and \(B_0\) as references, with \(x_A=1\) at \(A_1\) and \(x_B=1\) at \(B_1\). The interaction column is \(x_Ax_B\).

Factor combination	Intercept	\(x_A\)	\(x_B\)	\(x_Ax_B\)
\(A_0,B_0\)	1	0	0	0
\(A_1,B_0\)	1	1	0	0
\(A_0,B_1\)	1	0	1	0
\(A_1,B_1\)	1	1	1	1

For coefficient values \(\beta_0=1\), \(\beta_A=0.4\), \(\beta_B=-0.2\), and \(\beta_{AB}=0.7\), the predictor is \(\eta=1+0.4x_A-0.2x_B+0.7x_Ax_B\). At the four combinations, its values are \(1\), \(1+0.4=1.4\), \(1-0.2=0.8\), and \(1+0.4-0.2+0.7=1.9\), respectively. In the additive specification, the product column is omitted, and the last value would instead be \(1+0.4-0.2=1.2\).

Worked Example: A Three-Level Factor and a Binary Factor

Let \(A\) have levels \(A_0,A_1,A_2\), with \(A_0\) as reference, and let \(B\) have levels \(B_0,B_1\), with \(B_0\) as reference. Define \(x_{A1}=1\) at \(A_1\), \(x_{A2}=1\) at \(A_2\), and \(x_B=1\) at \(B_1\); all are zero at their respective reference levels. Multiplying each \(A\) column by \(x_B\) gives two interaction columns.

Combination	\(x_{A1}\)	\(x_{A2}\)	\(x_B\)	\(x_{A1}x_B\)	\(x_{A2}x_B\)
\(A_0,B_0\)	0	0	0	0	0
\(A_1,B_0\)	1	0	0	0	0
\(A_2,B_0\)	0	1	0	0	0
\(A_0,B_1\)	0	0	1	0	0
\(A_1,B_1\)	1	0	1	1	0
\(A_2,B_1\)	0	1	1	0	1

There are \(2\times1=2\) product columns. In the interaction-augmented predictor, their contribution is \(\beta_{A1B}x_{A1}x_B+\beta_{A2B}x_{A2}x_B\). For example, at \(A_2,B_1\), this contribution is \(0\beta_{A1B}+1\beta_{A2B}=\beta_{A2B}\); at \(A_2,B_0\), it is zero. This construction uses the same factor columns as the additive part and forms each product explicitly.

Worked Example: Product Terms in a Poisson Log-Linear Predictor

Suppose counts have a Poisson model with a log link, and use the binary reference coding from the first example. Set \(\beta_0=\log 2\), \(\beta_A=\log(3/2)\), \(\beta_B=\log 2\), and \(\beta_{AB}=\log(3/2)\). The mean is \(\mu=\exp(\eta)\).

At \(A_0,B_0\), \(\eta=\log 2\), so \(\mu=2\). At \(A_1,B_0\), \(\eta=\log 2+\log(3/2)=\log 3\), so \(\mu=3\). At \(A_0,B_1\), \(\eta=\log 2+\log 2=\log 4\), so \(\mu=4\). At \(A_1,B_1\), all four terms are present:

\eta=\log 2+\log(3/2)+\log 2+\log(3/2)=\log 9, \qquad \mu=\exp(\eta)=9.

The product column is zero in the first three rows and one in the joint non-reference row, so its coefficient enters only the last predictor in this coding. This is a calculation of the specified model, not a general claim about how a coefficient must be interpreted under every coding.

How Many Interaction Columns?

If \(A\) has \(a\) levels and \(B\) has \(b\) levels, reference coding gives \(r_A=a-1\) and \(r_B=b-1\). The number of products is therefore \((a-1)(b-1)\). The same count applies to any full-rank contrast coding that represents each factor with \(a-1\) and \(b-1\) columns. The next result explains why these columns are genuinely distinct on a complete factorial design.

Theorem: Suppose every one of the \(ab\) combinations of two factors is represented in the design, and let \(U\) and \(V\) be full-rank contrast matrices with dimensions \(a\times(a-1)\) and \(b\times(b-1)\). Assume that adjoining a column of ones to each matrix gives a full-rank matrix. On the complete grid of factor combinations, the columns formed from an intercept, the columns of \(U\), the columns of \(V\), and all pairwise products of columns of \(U\) and \(V\) are linearly independent. In particular, the interaction block has dimension \((a-1)(b-1)\).

Proof. Put \(U_0=[\mathbf{1}_a\ U]\) and \(V_0=[\mathbf{1}_b\ V]\), where \(\mathbf{1}_a\) and \(\mathbf{1}_b\) are vectors of ones. By assumption, both matrices have full column rank. Index rows of the full factorial design by pairs \((i,j)\). The column that pairs column \(s\) of \(U_0\) with column \(t\) of \(V_0\) has entry \((U_0)_{is}(V_0)_{jt}\) at row \((i,j)\). These columns comprise the full set: intercept, both main-effect blocks, and all products. If a linear combination with coefficient matrix \(C\) is zero at every grid point, then

(U_0 C V_0^{\mathsf T})_{ij}=0\quad\text{for every }i,j, \qquad\text{so}\qquad U_0 C V_0^{\mathsf T}=0.

Since \(U_0\) and \(V_0\) have full column rank, they have left inverses \(L_U\) and \(L_V\) satisfying \(L_UU_0=I\) and \(L_VV_0=I\). Multiplying the zero matrix on the left by \(L_U\) and on the right by \(L_V^{\mathsf T}\) gives \(C=0\). Thus the columns are linearly independent. The interaction block pairs each of the \(a-1\) columns of \(U\) with each of the \(b-1\) columns of \(V\), giving \((a-1)(b-1)\) independent columns. \(\square\)

The complete-grid condition matters. If some combinations are absent, the product columns can still be constructed, but the observed design matrix may no longer have full column rank. In that case, distinct coefficient values need not correspond to distinct fitted linear predictors.

The Interaction Space Does Not Depend on the Contrast Basis

Different full-rank contrast choices can produce different numerical columns. When they span the same contrast spaces for each factor, however, they produce the same interaction column space. This statement concerns the span of the columns, not equality of the coefficients attached to them.

Result: Let \(U'=UA\) and \(V'=VB\), where \(A\) and \(B\) are invertible square matrices. Then the pairwise-product columns made from \(U'\) and \(V'\) span the same space as those made from \(U\) and \(V\).

Proof. Write the interaction columns from \(U,V\) as \(U_{\cdot k}\otimes V_{\cdot l}\), with rows ordered over the full grid of factor combinations. A column formed from \(U'_{\cdot s}\) and \(V'_{\cdot t}\) is

U'_{\cdot s}\otimes V'_{\cdot t} =\left(\sum_k U_{\cdot k}A_{ks}\right) \otimes\left(\sum_l V_{\cdot l}B_{lt}\right) =\sum_k\sum_l A_{ks}B_{lt} (U_{\cdot k}\otimes V_{\cdot l}).

Thus every product column from \(U',V'\) is a linear combination of the original product columns. Because \(A\) and \(B\) are invertible, the same calculation using \(A^{-1}\) and \(B^{-1}\) expresses every original product column as a linear combination of the new ones. Each set therefore spans the same space. \(\square\)

Reading the Model Structure Correctly

The additive model contains the intercept and the two separate factor blocks. The interaction-augmented model adds product columns for pairs of factor coding columns. These are different specifications of the linear predictor: one restricts it to the additive terms, while the other makes additional joint-factor terms available. The GLM response distribution and link determine how a change in the predictor relates to the mean response.

A common mistake is to multiply factor labels or level numbers rather than the actual coding columns. For example, coding levels as 1 and 2 and multiplying those labels creates a numerical product whose values depend on arbitrary labels. The correct operation is to take the columns used to represent the factors in the model matrix and multiply those columns observation by observation. With indicator or contrast coding, the resulting product columns correspond to products of the chosen codes.

Another pitfall is assuming that writing down product columns guarantees their separate estimation. The full-grid result above requires all combinations and full-rank factor coding. With missing combinations or other design dependencies, inspect the rank of the observed design matrix before treating its coefficients as separately identifiable.

Key takeaway: A two-factor interaction block is built by taking every product of one coding column for factor \(A\) and one for factor \(B\). Adding this block gives the predictor terms for joint factor combinations beyond the additive specification; its estimability depends on the observed design.

Check Your Understanding

Try these questions without looking back.

Factor \(A\) has four levels and factor \(B\) has three. Under full-rank reference coding, how many product columns are included?
For two binary factors coded with indicators for their non-reference levels, what is the interaction column at the reference-reference combination?
State the difference between the additive and interaction-augmented linear predictors in terms of their columns.
Why can a contrast recoding change the numerical interaction columns without changing their span?
What design feature can prevent all constructed interaction coefficients from being separately identifiable?