Interpreting Factor-by-Factor Interactions

What a Factor-by-Factor Interaction Changes

When a model contains two factors and their interaction, a coefficient for one factor does not generally describe a single contrast that holds across every level of the other factor. With reference coding, each main-effect coefficient has a specific comparison built into it: it compares a factor level with its reference level when the other factor is at its reference level. Interaction coefficients describe how those reference-level comparisons change elsewhere.

The safest way to interpret such a model is to calculate the linear predictor for every combination of factor levels. This makes clear which coefficients contribute to each cell and which comparisons each coefficient represents. In a generalized linear model, these calculations are on the linear-predictor scale; converting them to fitted means or probabilities requires applying the inverse link.

Reference-Coded Model for Two Factors

Let factor $A$ have levels $A_0,A_1,\ldots,A_{p-1}$, and factor $B$ have levels $B_0,B_1,\ldots,B_{q-1}$. The levels $A_0$ and $B_0$ are the reference levels. Write $\eta_{ij}$ for the linear predictor at the combination $A_i,B_j$. In a GLM, $\eta_{ij}=g(\mu_{ij})$, where $g$ is the link function and $\mu_{ij}$ is the fitted mean for that combination.

Definition: Under reference coding, the two-factor model with an interaction is $$ \eta_{ij}=\beta_0+\alpha_i+\delta_j+\gamma_{ij}, $$ where $\alpha_0=0$, $\delta_0=0$, and every interaction term involving a reference level is zero: $\gamma_{i0}=\gamma_{0j}=0$. Thus $\beta_0$ is the linear predictor at $A_0,B_0$; $\alpha_i$ and $\delta_j$ are main-effect coefficients; and $\gamma_{ij}$ is the interaction coefficient for nonreference levels $A_i,B_j$.

Equivalently, the model uses indicators for nonreference levels: a term $\alpha_i$ is included when $A=A_i$, a term $\delta_j$ when $B=B_j$, and a term $\gamma_{ij}$ when both conditions hold. The interaction indicator is the product of the two factor indicators. For any particular cell, include the intercept and precisely the terms whose indicators equal one.

Worked Example: Calculating All Predictors in a Two-by-Two Model

Suppose $A$ and $B$ each have a reference level (level 0) and one nonreference level (level 1). Let the fitted coefficients be $\hat\beta_0=1.2$, $\hat\alpha_1=-0.4$, $\hat\delta_1=0.7$, and $\hat\gamma_{11}=0.5$. Calculate all four fitted linear predictors.

At $A_0,B_0$, both factor indicators are zero, so $\hat\eta_{00}=\hat\beta_0=1.2$.

At $A_1,B_0$, only the $A$ main-effect indicator is one: $\hat\eta_{10}=1.2-0.4=0.8$.

At $A_0,B_1$, only the $B$ main-effect indicator is one: $\hat\eta_{01}=1.2+0.7=1.9$.

At $A_1,B_1$, both main effects and the interaction are included: $\hat\eta_{11}=1.2-0.4+0.7+0.5=2.0$.

The $A$ contrast at $B_0$ is $0.8-1.2=-0.4$, exactly $\hat\alpha_1$. At $B_1$, the $A$ contrast is $2.0-1.9=0.1$, not $-0.4$. The interaction changes the $A$ contrast by $0.1-(-0.4)=0.5$, the value of $\hat\gamma_{11}$.

What Each Coefficient Compares

The interpretation follows directly by subtracting cell predictors. For a nonreference level $A_i$, compare $A_i$ with $A_0$ while holding $B$ at its reference level. The terms for $B_0$ and the interaction are zero in both cells, leaving $\alpha_i$. Likewise, comparing $B_j$ with $B_0$ while holding $A=A_0$ leaves $\delta_j$.

Result: In a reference-coded two-factor interaction model, $$ \eta_{i0}-\eta_{00}=\alpha_i,\qquad \eta_{0j}-\eta_{00}=\delta_j. $$ For nonreference levels $i,j$, the interaction coefficient is $$ \gamma_{ij}=(\eta_{ij}-\eta_{0j})-(\eta_{i0}-\eta_{00}). $$ It is the change in the $A_i$-versus-$A_0$ contrast when $B$ changes from $B_0$ to $B_j$, on the linear-predictor scale.

To verify these identities, substitute the model expression into each contrast. First, $\eta_{i0}=\beta_0+\alpha_i$ and $\eta_{00}=\beta_0$, so their difference is $\alpha_i$. Similarly, $\eta_{0j}=\beta_0+\delta_j$, giving $\eta_{0j}-\eta_{00}=\delta_j$. For the last identity, $\eta_{ij}=\beta_0+\alpha_i+\delta_j+\gamma_{ij}$ and $\eta_{0j}=\beta_0+\delta_j$, so $\eta_{ij}-\eta_{0j}=\alpha_i+\gamma_{ij}$. Subtracting $\eta_{i0}-\eta_{00}=\alpha_i$ leaves $\gamma_{ij}$. This establishes the result. $\square$

The same reasoning shows that the $B_j$-versus-$B_0$ contrast at $A_i$ is $\delta_j+\gamma_{ij}$. Therefore, neither main-effect coefficient by itself is generally the contrast at every level of the other factor. Each is the contrast at that factor’s reference level.

Worked Example: A Three-Level Factor and a Two-Level Factor

Let $A$ denote treatment intensity, with usual care $A_0$, high intensity $A_1$, and intensive treatment $A_2$. Let $B_0$ be clinic delivery and $B_1$ be remote delivery. Suppose the fitted coefficients are $\hat\beta_0=2$, $\hat\alpha_1=0.6$, $\hat\alpha_2=1.1$, $\hat\delta_1=-0.3$, $\hat\gamma_{11}=-0.2$, and $\hat\gamma_{21}=0.4$.

At clinic delivery, the interaction terms are zero. The three predictors are $\hat\eta_{00}=2$, $\hat\eta_{10}=2+0.6=2.6$, and $\hat\eta_{20}=2+1.1=3.1$.

At remote delivery, add $\hat\delta_1=-0.3$, and for each nonreference treatment also add its interaction. Thus $\hat\eta_{01}=2-0.3=1.7$, $\hat\eta_{11}=2+0.6-0.3-0.2=2.1$, and $\hat\eta_{21}=2+1.1-0.3+0.4=3.2$. These are all six fitted linear predictors.

The high-intensity versus usual-care contrast is $2.6-2=0.6$ at clinic delivery, but $2.1-1.7=0.4$ at remote delivery. The coefficient $\hat\alpha_1=0.6$ is the clinic contrast; the interaction $\hat\gamma_{11}=-0.2$ changes it to $0.6-0.2=0.4$ remotely. For intensive treatment, the corresponding contrasts are $3.1-2=1.1$ at the clinic and $3.2-1.7=1.5$ remotely, because $\hat\gamma_{21}=0.4$.

The remote-versus-clinic contrast is $-0.3$ for usual care, $-0.3-0.2=-0.5$ for high intensity, and $-0.3+0.4=0.1$ for intensive treatment. Again, the $B$ main-effect coefficient alone is the contrast only at the reference level of $A$.

Recovering Coefficients from Cell Predictors

With all $pq$ combinations represented, the reference-coded interaction model has one coefficient for each cell predictor: one intercept, $p-1$ coefficients for $A$, $q-1$ for $B$, and $(p-1)(q-1)$ interactions. The following result makes precise how the cell predictors and coefficients correspond.

Theorem: If the linear predictor for every combination $A_i,B_j$ is specified, the reference-coded model determines its coefficients uniquely. They are $$ \beta_0=\eta_{00},\qquad \alpha_i=\eta_{i0}-\eta_{00},\qquad \delta_j=\eta_{0j}-\eta_{00},\qquad \gamma_{ij}=\eta_{ij}-\eta_{i0}-\eta_{0j}+\eta_{00}. $$

Proof. At the reference combination, the model gives $\eta_{00}=\beta_0$. At $A_i,B_0$, it gives $\eta_{i0}=\beta_0+\alpha_i$, so $\alpha_i=\eta_{i0}-\eta_{00}$. At $A_0,B_j$, it gives $\eta_{0j}=\beta_0+\delta_j$, so $\delta_j=\eta_{0j}-\eta_{00}$. Finally, at $A_i,B_j$, rearranging $\eta_{ij}=\beta_0+\alpha_i+\delta_j+\gamma_{ij}$ gives $\gamma_{ij}=\eta_{ij}-\beta_0-\alpha_i-\delta_j$. Substituting the first three expressions yields the stated formula. These equations determine every coefficient uniquely. Conversely, substituting these coefficients into the model reproduces each $\eta_{ij}$, including cells involving a reference level, so the correspondence works in both directions. $\square$

This result is an algebraic statement about the model’s cell predictors. It does not guarantee that every predictor can be estimated from a particular data set: the calculation presumes that the relevant cell predictors are available. The interpretation of the coefficients, however, remains the same whenever the model parameters are identified.

Worked Example: Interpreting a Logistic Interaction

Suppose a binary-outcome model uses the logit link. Factor $A$ indicates a new treatment, and factor $B$ indicates a follow-up intervention; level 0 is the reference for each. Let the fitted coefficients be $\hat\beta_0=-1.5$, $\hat\alpha_1=0.8$, $\hat\delta_1=0.4$, and $\hat\gamma_{11}=0.6$. Calculate all four linear predictors.

The reference cell has $\hat\eta_{00}=-1.5$. The treatment-only cell has $\hat\eta_{10}=-1.5+0.8=-0.7$. The follow-up-only cell has $\hat\eta_{01}=-1.5+0.4=-1.1$. With both interventions, all terms contribute, so $\hat\eta_{11}=-1.5+0.8+0.4+0.6=0.3$.

Here the main-effect coefficient $0.8$ is the treatment contrast in log odds when the follow-up intervention is absent. With the follow-up intervention present, the treatment contrast is $0.3-(-1.1)=1.4=0.8+0.6$. Thus $\hat\gamma_{11}=0.6$ is the increase in the treatment log-odds contrast when moving from no follow-up to follow-up.

For a response-scale check, the inverse-logit function is $1/(1+\exp(-\eta))$. The four fitted probabilities are approximately $0.182$, $0.332$, $0.250$, and $0.574$, in the order $(A_0,B_0),(A_1,B_0),(A_0,B_1),(A_1,B_1)$. These probabilities are not obtained by adding the coefficients: the coefficients combine to give logits, and the inverse link then gives probabilities.

Common Mistake and Practical Interpretation

A common mistake is to read $\alpha_i$ as “the effect of $A_i$ compared with $A_0$, regardless of $B$.” That reasoning ignores the interaction term. The correction is to specify the level of $B$: $\alpha_i$ is the contrast at $B_0$, while the contrast at $B_j$ is $\alpha_i+\gamma_{ij}$. In the reverse direction, $\delta_j$ is the $B$ contrast at $A_0$, while the contrast at $A_i$ is $\delta_j+\gamma_{ij}$.

These are comparisons on the linear-predictor scale. For a logit link, they are log-odds contrasts; for a log link, they are contrasts in log means. Applying an inverse link generally changes how differences appear on the response scale. Consequently, a coefficient should not be described as a constant difference in probability or mean response unless that interpretation follows from the particular link and model.

For a model with two factors, a reliable interpretation routine is to identify both reference levels, write the predictor for the two reference levels, and then add the terms for each nonreference level and their interaction. Calculate every cell predictor before describing main effects. This makes the conditioning in each coefficient explicit and keeps link-scale statements separate from response-scale statements.

Check Your Understanding

Use reference coding and interpret all contrasts on the linear-predictor scale.

For a two-by-two model with $\beta_0=3$, $\alpha_1=1$, $\delta_1=-2$, and $\gamma_{11}=0.5$, calculate all four cell predictors.
In that model, what is the $A_1$-versus-$A_0$ contrast at $B_0$? What is it at $B_1$?
For factors with three and four levels, respectively, how many interaction coefficients appear under reference coding?
In a logit model, explain why a main-effect coefficient is not generally a difference in fitted probabilities.
If $\eta_{00}=1$, $\eta_{10}=4$, $\eta_{01}=2$, and $\eta_{11}=7$, calculate $\beta_0,\alpha_1,\delta_1,\gamma_{11}$.