Hierarchical Model Structure for Interactions

Why Hierarchy Matters

A factor interaction allows the association between one factor and the linear predictor to vary across levels of another. In a generalized linear model (GLM), these terms appear in the linear predictor, \(\eta\), and affect the response mean through the link function. When an interaction is included, a model specification must also decide which lower-order terms to retain. That decision matters: it determines which patterns the model can represent and what a comparison with another model changes.

Hierarchy is a rule for organizing those terms. It is not a claim that a lower-order effect must be statistically significant, nor does it dictate which interactions science should study. Rather, it gives a structured way to specify a model in which an interaction is not forced to stand in for a simpler contribution that has been excluded.

Definition: A term is associated with a set of one or more factors. A main effect, such as \(A\), is a one-factor term; an interaction, such as \(A:B\), is a term involving both factors. The lower-order terms associated with a term for factors in \(S\) are the terms for proper subsets of \(S\). Its immediate lower-order terms are those obtained by removing exactly one factor from \(S\).

Thus the immediate lower-order terms associated with \(A:B\) are \(A\) and \(B\). Those associated with \(A:B:C\) are \(A:B\), \(A:C\), and \(B:C\). The full set of proper lower-order terms for \(A:B:C\) also includes \(A\), \(B\), and \(C\). The intercept is usually included in the model, but it is not counted as one of these factor terms.

Definition: A model satisfies strong hierarchy if, whenever it includes a term, it includes every proper lower-order term associated with that term. It satisfies weak hierarchy under the recursive convention used here if every included term involving at least two factors has at least one included immediate lower-order term. For a two-factor interaction, weak hierarchy therefore requires at least one of its two main effects. For higher-order terms, the requirement applies recursively to the included lower-order terms.

For example, strong hierarchy for \(A:B:C\) requires all three two-factor interactions and all three main effects. Under weak hierarchy, a model containing \(A:B:C\) must include at least one of \(A:B\), \(A:C\), or \(B:C\); any included two-factor interaction must, in turn, have at least one of its main effects. A model can include all lower-order terms without including the highest-order interaction. Hierarchy imposes implications in one direction only: a lower-order term does not require a higher-order one.

Terms in a GLM Linear Predictor

For factors \(A\) and \(B\), write the linear predictor as an intercept, a contribution from each main effect, and a contribution from their interaction. For factors with \(r\) and \(s\) levels, reference coding uses \(r-1\) and \(s-1\) contrast columns for the main effects. The interaction block contains their pairwise products, with \((r-1)(s-1)\) columns. Other full-rank contrast codings can change the coefficient coordinates, but the term still represents the factor-by-factor interaction.

This block view is useful because hierarchy concerns which terms are present, not whether particular coefficient values happen to be zero. A factor interaction can have multiple coefficients, especially when a factor has more than two levels. Retaining an associated main-effect block means allowing that contribution in the predictor, regardless of whether any fitted coefficient is large or small.

Worked Example: Strong and Weak Hierarchy for Two Factors

Let \(A\) have two levels and \(B\) have three levels. Under reference coding, use \(a=1\) for the non-reference level of \(A\), and \(b_2\) and \(b_3\) for indicators of the two non-reference levels of \(B\). A model with the interaction has predictor

\eta=\beta_0+\beta_Aa+\beta_{B2}b_2+\beta_{B3}b_3 +\delta_2ab_2+\delta_3ab_3.

The \(A:B\) interaction has two columns, \(ab_2\) and \(ab_3\). Its associated main effects are \(A\) and \(B\), where the \(B\) term is a two-column block. Strong hierarchy includes both blocks, as in the displayed model. A weakly hierarchical model could retain \(A\) and \(A:B\) but omit \(B\):

\eta=\beta_0+\beta_Aa+\delta_2ab_2+\delta_3ab_3.

For example, set \(\beta_0=-1\), \(\beta_A=0.2\), \(\delta_2=0.5\), and \(\delta_3=-0.4\). At the reference level of \(B\), the predictor difference between the two \(A\) levels is \(0.2\). At \(B=2\), it is \(0.2+0.5=0.7\); at \(B=3\), it is \(0.2-0.4=-0.2\). The interaction allows the difference to vary, while the omitted \(B\) block constrains the predictor difference between \(B\) levels to be zero when \(a=0\). This model is weakly hierarchical but not strongly hierarchical.

Hierarchy for Three-Factor Interactions

For a three-factor interaction, it is important to distinguish its immediate lower-order terms from all of its proper lower-order terms. The immediate terms are the three two-factor interactions; the full set also includes the three main effects. Strong hierarchy requires all six. Under the recursive weak convention, the three-way interaction needs at least one two-way interaction, and each included two-way interaction needs at least one of its main effects.

Worked Example: Checking a Three-Factor Term

Consider the model with terms \(A\), \(A:B\), and \(A:B:C\), along with an intercept. The immediate lower-order terms of \(A:B:C\) are \(A:B\), \(A:C\), and \(B:C\); of these, \(A:B\) is present. The immediate lower-order terms of \(A:B\) are \(A\) and \(B\); \(A\) is present. Therefore this term set satisfies the recursive weak-hierarchy rule. It does not satisfy strong hierarchy, because \(A:C\), \(B:C\), \(B\), and \(C\) are absent.

If instead the model contains \(A:B:C\), \(A:B\), and \(A\), then it still meets the stated weak rule. If it contains \(A:B:C\) alone, it meets neither rule: it has no included immediate lower-order term. Listing the terms this way prevents the common error of checking only whether a three-way interaction has at least one main effect, without checking its associated two-way terms.

The term structure can be written schematically as

\eta=\beta_0+\text{terms for }A,B,C +\text{terms for }A:B,A:C,B:C +\text{terms for }A:B:C.

Each label represents a block of design columns, not necessarily one coefficient. For instance, if the factors have \(r\), \(s\), and \(t\) levels under reference coding, the three-way block has \((r-1)(s-1)(t-1)\) columns. Hierarchy applies to the inclusion of these blocks.

Why Lower-Order Terms Affect Interpretation

A lower-order term gives the predictor room to reflect a contribution that does not depend on the other factors. Without it, an interaction may be the only way to represent a change associated with a factor, and the model may force that change to occur only in particular combinations of factor levels. The resulting restriction can be unintended.

The following derivation makes this point for two binary factors under reference coding. Let \(a,b\in\{0,1\}\), with predictor

\eta(a,b)=\beta_0+\beta_Aa+\beta_Bb+\delta_{AB}ab.

At \(b=0\), the difference in predictor between \(a=1\) and \(a=0\) is

\eta(1,0)-\eta(0,0)=\beta_A.

At \(b=1\), that difference is

\eta(1,1)-\eta(0,1)=\beta_A+\delta_{AB}.

Therefore \(\beta_A\) is the \(A\) difference at the reference level of \(B\), and \(\delta_{AB}\) is the change in that difference at the other level. If \(\beta_A\) is omitted but \(\delta_{AB}\) remains, the \(A\) difference at \(b=0\) is forced to zero, while its difference at \(b=1\) can still be nonzero. This is a particular constraint, not a general statement that the \(A\) effect is absent.

Worked Example: A Two-by-Two Predictor With and Without a Main Effect

Suppose a binary-response GLM uses the logit link and has predictor \(\eta(a,b)= -0.4+0.8b+0.6ab\), omitting the \(A\) main effect. At \(b=0\), the predictor is \(-0.4\) for both values of \(a\). At \(b=1\), it is \(-0.4+0.8=0.4\) for \(a=0\), and \(-0.4+0.8+0.6=1.0\) for \(a=1\).

The corresponding fitted probabilities are \(\operatorname{logit}^{-1}(\eta)=1/(1+\exp(-\eta))\). Thus they are approximately \(0.401\), \(0.401\), \(0.599\), and \(0.731\), respectively, for \((a,b)=(0,0),(1,0),(0,1),(1,1)\). The model imposes equal fitted probabilities across \(A\) when \(B\) is at its reference level, but permits a difference at the other level. Including \(\beta_Aa\) would allow the \(A\) contrast at the reference level to be estimated rather than fixed at zero.

Hierarchy and Interpretable Model Comparisons

A practical reason to use hierarchy is that it supports comparisons that add or remove an interaction while retaining the terms that describe the lower-order structure. The meaning of “add the interaction” is then clearer: the comparison changes the interaction block, rather than changing it together with a missing main effect. This is a specification principle, not a test result; the question of how to assess an interaction in nested models is a separate step.

Result: Suppose two GLMs use the same observations, response, link, offset, and factor coding, and their predictors consist of an intercept and unrestricted blocks of design columns for the listed terms. If one model’s term set is contained in the other’s, then its linear-predictor values form a subset of those available under the larger model.

Proof. Let \(X_1\) be the design matrix for the smaller term set, and \(X_2\) the design matrix for the larger one. Because every term block in the smaller set is included in the larger set, the columns of \(X_1\) occur among the columns of \(X_2\), up to column ordering. For any coefficient vector \(\beta_1\), set the coefficients of the shared columns in the larger model equal to \(\beta_1\), and set the coefficients of its additional columns to zero. The resulting vector \(\beta_2\) satisfies \(X_2\beta_2=X_1\beta_1\). Adding the same offset to both predictors preserves equality. Hence every predictor available under the smaller model is available under the larger model. \(\square\)

Worked Example: Choosing Terms for a Clear Comparison

Suppose the scientific question is whether the association between treatment group \(A\) and the linear predictor varies by site \(B\). A strong-hierarchy specification compares a model with \(A\) and \(B\) against one with \(A\), \(B\), and \(A:B\), keeping the intercept, observations, offset, and other terms the same. The first model’s term set is contained in the second’s, and the only added block is \(A:B\).

If the comparison instead starts from a model containing \(A\) but not \(B\), then adds both \(B\) and \(A:B\), it changes two blocks at once. That may be a legitimate scientific comparison, but it is not an isolated comparison of the interaction. Stating the term sets explicitly reveals what is being changed and helps keep the interpretation aligned with the question.

Common Pitfall

A frequent mistake is to treat hierarchy as a rule that every lower-order coefficient must be nonzero whenever an interaction is included. That is incorrect. Strong hierarchy concerns whether the associated terms are included; it does not impose nonzero estimates or guarantee statistical significance. Conversely, retaining a main-effect term does not assert that its fitted contribution differs from zero.

Another mistake is to assume that an interaction makes its main effects irrelevant. With reference coding, a main-effect coefficient generally describes a contrast at the reference levels of the other factors. With other contrast codings, the coefficient coordinates and their interpretation change, but the term blocks still represent the specified model structure. The key correction is to distinguish inclusion of a term from the numerical value or coding-dependent interpretation of its coefficients.

Key takeaway: For each interaction, identify its associated lower-order terms before interpreting the model. Strong hierarchy includes all proper lower-order terms; weak hierarchy, under the recursive convention here, requires at least one immediate lower-order term at each step. Including these terms makes the model’s constraints visible and supports comparisons that isolate the interaction block.

Check Your Understanding

Use the term-inclusion rules and linear-predictor interpretations in this tutorial.

List the immediate and all proper lower-order terms associated with \(A:B:C\).
A model contains \(A\), \(B:C\), and \(A:B:C\), but no other terms apart from the intercept. Does it satisfy the recursive weak-hierarchy rule? Does it satisfy strong hierarchy?
For a two-factor interaction under reference coding, what does omitting the \(A\) main-effect term constrain when \(A:B\) remains?
Why is a comparison from \(A+B\) to \(A+B+A:B\) more directly interpretable as adding the interaction than a comparison from \(A\) to \(A+B+A:B\)?
Does strong hierarchy require every included lower-order coefficient to be statistically significant? Explain.