Tutorials › Generalized Linear Models › Common Misreadings of Factorial Coefficients

Home  /  Tutorials  /  Factors Contrasts and Interactions

Factors Contrasts and Interactions · Tutorial 157 of 981

Common Misreadings of Factorial Coefficients

Use fitted cell predictors and inverse links to interpret factorial coefficients correctly, including interactions and main effects.

Intermediate 10 min read

What You'll Learn

  • Write fitted linear predictors for all cells in a reference-coded two-factor model
  • Interpret main-effect coefficients conditionally at the other factor’s reference level
  • Distinguish link-scale interaction coefficients from response-scale differences
  • Calculate cell means and probabilities from fitted coefficients
  • Compare logistic, Poisson, and identity-link interpretations
  • Diagnose common errors when reporting factorial-model coefficients

Why Factorial Coefficients Are Easy to Misread

A coefficient table for a factorial GLM does not usually list every group’s fitted mean or probability. Instead, under reference coding, its coefficients combine to produce a linear predictor for each factor-level combination. The inverse link then converts each linear predictor to the response scale. Confusing these steps can lead to claims such as “the interaction coefficient is a probability difference” or “the main effect applies at every level of the other factor.”

The reliable interpretation starts with the fitted value for each cell. From those fitted values, we can calculate whichever response-scale differences or ratios answer the scientific question. The coefficients have direct interpretations too, but those interpretations are conditional on the coding and, for a nonlinear link, are generally on the link scale.

Definition: In a reference-coded two-factor model, each factor has a designated reference level, represented by zero-valued indicator variables. For binary factors \(A\) and \(B\), let \(A=0\) and \(B=0\) denote their reference levels, and \(A=1\) and \(B=1\) their other levels. A model with both main effects and their interaction has linear predictor \(\eta_{AB}=\beta_0+\beta_A A+\beta_B B+\beta_{AB}AB\). The GLM mean is \(\mu_{AB}=g^{-1}(\eta_{AB})\), where \(g\) is the link function.

Here, \(\beta_0,\beta_A,\beta_B,\beta_{AB}\) denote fitted coefficients when we discuss fitted values; equivalently, the same formulas apply to coefficient estimates \(\hat\beta_0,\hat\beta_A,\hat\beta_B,\hat\beta_{AB}\). We assume the four cell predictors are estimable and the model includes the displayed terms. The reference levels determine which comparisons the coefficients encode; the fitted cell means or probabilities are obtained only after applying the inverse link.

What the Coefficients Encode

Substitute the four combinations of indicator values into the model. This gives the four fitted linear predictors:

$$ \begin{aligned} \eta_{00}&=\beta_0,\\ \eta_{10}&=\beta_0+\beta_A,\\ \eta_{01}&=\beta_0+\beta_B,\\ \eta_{11}&=\beta_0+\beta_A+\beta_B+\beta_{AB}. \end{aligned} $$

Thus, \(\beta_0\) is the reference-reference cell’s linear predictor. The coefficient \(\beta_A\) is the change in linear predictor from \(A=0\) to \(A=1\) when \(B=0\). Similarly, \(\beta_B\) is the change from \(B=0\) to \(B=1\) when \(A=0\). Neither main-effect coefficient, by itself, describes the corresponding change at every level of the other factor.

Theorem: In the reference-coded two-factor model above, \(\beta_{AB}=(\eta_{11}-\eta_{01})-(\eta_{10}-\eta_{00})\). It is the difference between the two conditional changes in the linear predictor for factor \(A\), or equivalently the difference between the two conditional changes for factor \(B\).

Proof. From the four cell predictors, the change in \(\eta\) for \(A\) when \(B=0\) is \(\eta_{10}-\eta_{00}=\beta_A\). When \(B=1\), it is \(\eta_{11}-\eta_{01}=(\beta_0+\beta_A+\beta_B+\beta_{AB})-(\beta_0+\beta_B)=\beta_A+\beta_{AB}\). Subtracting the first change from the second gives \(\beta_{AB}\). For \(B\), the changes are \(\eta_{01}-\eta_{00}=\beta_B\) at \(A=0\), and \(\eta_{11}-\eta_{10}=\beta_B+\beta_{AB}\) at \(A=1\); their difference is also \(\beta_{AB}\). \(\square\)

This result concerns the linear predictor. The response-scale mean is \(\mu=g^{-1}(\eta)\). For a nonlinear inverse link, a difference in \(\eta\) does not generally equal a difference in \(\mu\). Consequently, the interaction coefficient is not generally a probability difference, a difference in counts, or an additive difference in means.

Key distinction: The coefficient \(\beta_{AB}\) is a difference-in-differences on the link scale. A response-scale difference-in-differences must instead be calculated from the four fitted means: \((\mu_{11}-\mu_{01})-(\mu_{10}-\mu_{00})\). These quantities coincide for the identity link, but not generally for a nonlinear link.

Worked Example: Logistic Interaction and Fitted Probabilities

Worked Example: Logistic Interaction and Fitted Probabilities

Suppose a binary-outcome GLM uses the logit link and has fitted coefficients \(\beta_0=-1\), \(\beta_A=0.8\), \(\beta_B=-0.5\), and \(\beta_{AB}=1.2\). The fitted probability is \(p_{AB}=1/(1+\exp(-\eta_{AB}))\). Calculate the four fitted probabilities and interpret the interaction.

First calculate the linear predictors:

$$ \begin{aligned} \eta_{00}&=-1,\\ \eta_{10}&=-1+0.8=-0.2,\\ \eta_{01}&=-1-0.5=-1.5,\\ \eta_{11}&=-1+0.8-0.5+1.2=0.5. \end{aligned} $$

In particular, the last predictor is \(0.5\), not \(-0.5\). Applying the inverse-logit to each value gives:

$$ \begin{aligned} p_{00}&=\frac{1}{1+\exp(1)}\approx 0.268941,\\ p_{10}&=\frac{1}{1+\exp(0.2)}\approx 0.450166,\\ p_{01}&=\frac{1}{1+\exp(1.5)}\approx 0.182426,\\ p_{11}&=\frac{1}{1+\exp(-0.5)}\approx 0.622459. \end{aligned} $$

The fitted probability change for \(A\) at \(B=0\) is \(0.450166-0.268941=0.181225\), about 18.1 percentage points. At \(B=1\), it is approximately \(0.440034\), about 44.0 percentage points. Using the unrounded fitted probabilities, the response-scale difference-in-differences is approximately \(0.258809\). It is not the interaction coefficient \(1.2\).

The coefficient \(1.2\) is a difference-in-differences in log odds. Exponentiating it gives \(\exp(1.2)\approx3.3201\): the odds ratio for the change in \(A\) at \(B=1\), divided by the odds ratio for the change in \(A\) at \(B=0\). Likewise, \(\beta_A=0.8\) is the log-odds change for \(A\) specifically at \(B=0\); the corresponding odds ratio is \(\exp(0.8)\approx2.2255\). It is not a constant probability difference.

For comparison, the fitted probability change for \(B\) is approximately \(-0.086516\) at \(A=0\), but \(0.622459-0.450166=0.172293\) at \(A=1\). These are response-scale changes, not the coefficients \(-0.5\) and \(-0.5+1.2=0.7\), which are conditional changes in log odds.

Why a Nonlinear Link Changes Response-Scale Comparisons

The calculation above illustrates a general fact. If the inverse link is nonlinear, the response-scale change associated with a fixed increment in the linear predictor depends on the starting value of that predictor. Even if two groups have the same increment in \(\eta\), their changes in \(\mu\) need not be the same.

Result: For any inverse link \(h=g^{-1}\), the response-scale difference-in-differences in the two-factor model is \(h(\beta_0+\beta_A+\beta_B+\beta_{AB})-h(\beta_0+\beta_B)-h(\beta_0+\beta_A)+h(\beta_0)\). This expression is not generally equal to \(\beta_{AB}\).

Derivation. By definition, \(\mu_{ab}=h(\eta_{ab})\). The response-scale difference-in-differences is \((\mu_{11}-\mu_{01})-(\mu_{10}-\mu_{00})\). Substituting \(\mu_{ab}=h(\eta_{ab})\), then substituting the four predictors, yields exactly the displayed expression. If \(h(x)=x\), the expression reduces to \(\eta_{11}-\eta_{01}-\eta_{10}+\eta_{00}=\beta_{AB}\). For a nonlinear \(h\), that simplification does not follow. \(\square\)

Worked Example: A Poisson Log-Link Model

Worked Example: A Poisson Log-Link Model

Suppose \(A\) has reference level 0 and two other levels, 1 and 2, while \(B\) is binary. Use reference-coded indicators and a log link. Let the fitted coefficients be \(\beta_0=\log 2\), \(\beta_{A1}=\log 1.5\), \(\beta_{A2}=\log 2\), \(\beta_B=\log 3\), \(\beta_{A1B}=\log 0.5\), and \(\beta_{A2B}=\log 2\). Calculate the fitted means.

For \(B=0\), the relevant interaction indicators are zero. The reference cell has mean \(\exp(\log 2)=2\); at \(A=1\), the mean is \(\exp(\log 2+\log 1.5)=2(1.5)=3\); and at \(A=2\), it is \(\exp(\log 2+\log 2)=2(2)=4\).

For \(B=1\), first calculate each predictor, then exponentiate it. At \(A=0\), \(\eta_{01}=\log 2+\log 3=\log 6\), giving mean 6. At \(A=1\), \(\eta_{11}=\log 2+\log 1.5+\log 3+\log 0.5=\log 4.5\), giving mean 4.5. At \(A=2\), \(\eta_{21}=\log 2+\log 2+\log 3+\log 2=\log 24\), giving mean 24.

The coefficient \(\beta_{A1}=\log 1.5\) means that at \(B=0\), the mean at \(A=1\) is 1.5 times the mean at \(A=0\): \(3/2=1.5\). At \(B=1\), the ratio is \(4.5/6=0.75\). The interaction coefficient \(\log 0.5\) modifies the ratio by a factor of 0.5: \(0.75/1.5=0.5\). It is a log ratio-of-ratios, not an additive difference in expected counts.

For \(A=2\), the mean ratio relative to \(A=0\) is \(4/2=2\) at \(B=0\) and \(24/6=4\) at \(B=1\). Their ratio is \(4/2=2=\exp(\beta_{A2B})\). This confirms that each interaction coefficient is tied to a particular comparison against the reference levels.

When a Coefficient Is a Response-Scale Difference

It would also be a mistake to conclude that coefficients can never equal response-scale differences. With the identity link, \(g(\mu)=\mu\), so \(\eta=\mu\). In that case, the linear-predictor differences are also mean differences, and the interaction coefficient is an additive difference-in-differences. The link function determines the scale of interpretation.

Worked Example: Identity-Link Factorial Model

Suppose a Gaussian identity-link model has fitted coefficients \(\beta_0=10\), \(\beta_A=3\), \(\beta_B=-2\), and \(\beta_{AB}=4\). The four fitted means equal their linear predictors: \(\mu_{00}=10\), \(\mu_{10}=10+3=13\), \(\mu_{01}=10-2=8\), and \(\mu_{11}=10+3-2+4=15\).

The mean difference for \(A\) is \(13-10=3\) at \(B=0\) and \(15-8=7\) at \(B=1\). Their difference is \(7-3=4=\beta_{AB}\). Here, \(\beta_{AB}\) really is an additive difference-in-differences in the response mean because the identity link leaves the predictor unchanged. The main-effect coefficient \(\beta_A=3\) still describes the change at the reference level \(B=0\), not the change at both levels of \(B\).

A Practical Interpretation Check

Before translating a coefficient into scientific language, identify three things: the coding, the link, and the cells being compared. For a reference-coded model, “main effect” does not mean “effect averaged over the other factor.” It means a particular conditional comparison at that factor’s reference level. If the desired quantity is a probability difference, mean difference, or other response-scale contrast, calculate it from fitted values on that scale.

1
Write the cell predictors.
Substitute the reference-coded indicator values for each relevant factor combination.
2
Apply the inverse link.
Convert each linear predictor to a fitted mean or probability before forming response-scale comparisons.
3
State the scale and condition.
Name the comparison, the level of the other factor, and whether the quantity is on the link or response scale.

Common Mistakes

  • Calling a logit interaction a probability difference. The logistic example’s interaction coefficient is 1.2 on the log-odds scale. The fitted probability difference-in-differences is about 0.2588, obtained from the four probabilities.
  • Interpreting a main effect unconditionally. Under reference coding, \(\beta_A\) is the \(A\) comparison when \(B\) is at its reference level. With an interaction, the \(A\) comparison at another \(B\) level includes the interaction coefficient.
  • Reading a log-link coefficient as an additive count change. In the Poisson example, \(\log 1.5\) encodes a multiplicative mean ratio of 1.5 at the reference level of \(B\), not an increase of \(\log 1.5\) expected events.
  • Ignoring the inverse link when calculating fitted values. Coefficients add to form \(\eta\); for a logit or log link, fitted probabilities or means require applying the inverse link.
  • Assuming every interaction has the same response-scale meaning. An identity link gives an additive mean difference-in-differences, whereas nonlinear links generally give a link-scale interaction whose response-scale counterpart must be calculated separately.
Key takeaway: A factorial coefficient is interpreted through the model’s coding and link. Under reference coding, main effects are conditional comparisons at reference levels; interactions compare those conditional effects on the linear-predictor scale. To report response-scale effects, calculate them from the fitted cell means or probabilities.

Check Your Understanding

For each question, identify the scale and the relevant cell comparisons before interpreting a coefficient.

  1. In a reference-coded logit model with two binary factors, what does the coefficient of \(A\) describe when an \(A\)-by-\(B\) interaction is included?
  2. Why is a logit interaction coefficient not generally the difference-in-differences in fitted probabilities?
  3. In a Poisson log-link model, what multiplicative comparison does the exponential of an \(A\)-by-\(B\) interaction coefficient represent?
  4. Under what link does the interaction coefficient equal an additive response-scale difference-in-differences?
  5. A fitted model has a nonzero interaction. What fitted quantities should be calculated if the scientific question asks for the change in probability for \(A\) at each level of \(B\)?

Previous

Next

Back to Tutorials
Clinical Biostats