Preserve Lower-Order Terms With the Hierarchy Principle

Why Keep the Main Effects?

An interaction model lets the relationship between one quantitative predictor and the outcome depend on another predictor. Its product term is not usually intended to stand alone. If a model includes the product of two predictors, it ordinarily also includes each predictor's main effect. This convention is called the hierarchy principle.

The reason is not that algebra makes a product-only model impossible. It is that omitting the main effects imposes additional restrictions on predicted outcomes, and those restrictions can depend on where the predictors' zero values happen to be. Keeping the lower-order terms makes the specification more flexible and preserves its form when the predictors are shifted to new origins.

Two Specifications, Different Restrictions

Suppose \(Y_i\) is a quantitative outcome for observation \(i\), and \(x_{1i}\) and \(x_{2i}\) are quantitative predictors. A hierarchical interaction model for the conditional mean is

Here \(\beta_0\) is the intercept, \(\beta_1\) and \(\beta_2\) are the main-effect coefficients, and \(\beta_{12}\) is the product-term coefficient. A product-only specification instead omits both main effects:

The product-only model is a more restricted model. In particular, when either predictor is zero, the product is zero, so its predicted mean is \(\gamma_0\), regardless of the other predictor. The hierarchical model does not impose that restriction: when \(x_1=0\), its predicted mean is \(\beta_0+\beta_2x_2\); when \(x_2=0\), it is \(\beta_0+\beta_1x_1\).

These equations describe conditional means. A corresponding regression model can be written as an outcome equal to the specified mean plus an error term \(\varepsilon_i\). The key assumption for interpreting the mean equation is \(E(\varepsilon_i\mid x_{1i},x_{2i})=0\). The predictor values and the chosen equation determine fitted means; the examples below treat the coefficients as given and make no claim about statistical significance or causation.

Why the Predictor Origins Matter

Zero is not always a scientifically important value. It can be a convenient reference point chosen by how a variable was recorded. For example, a predictor might be measured from an arbitrary baseline, or it might be centered by subtracting a chosen value. A useful model specification should not acquire a different set of required terms merely because the origin has changed.

To see what happens, define shifted predictors \(u=x_1-c\) and \(v=x_2-d\), where \(c\) and \(d\) are constants. Thus \(x_1=u+c\) and \(x_2=v+d\). Substituting these into a hierarchical model gives

The shifted equation still has an intercept, both main effects, and the interaction. Its coefficients may change, but its form is preserved. The same is not true of a product-only equation: after shifting, its product expands to include terms in \(u\) and \(v\). A product-only restriction therefore depends on the chosen origins.

What a Product-Only Model Implies After Shifting

The shift calculation also clarifies why a product-only model is a special restriction rather than simply a shorter version of the hierarchical model. Consider \(\hat{Y}=\gamma_0+\gamma_{12}x_1x_2\). Replacing \(x_1\) with \(u+c\) and \(x_2\) with \(v+d\) yields

Unless \(c\) and \(d\) happen to make the new main-effect coefficients zero, the shifted equation contains both lower-order terms. Therefore the product-only form at the original origins does not remain product-only at other origins.

When Might a Lower-Order Term Be Omitted?

The hierarchy principle is a strong default, not an absolute rule. A researcher might omit a main effect for a clearly stated scientific or design reason. But the resulting equation must be interpreted as imposing the corresponding restriction. In a product-only model, for instance, the mean is constrained to be constant in \(x_2\) whenever \(x_1=0\), and constant in \(x_1\) whenever \(x_2=0\). Those conditions should be defensible in context and meaningful at the chosen origins.

A predictor having a small estimated main-effect coefficient is not, by itself, a reason to delete it while retaining the interaction. Removing that term changes the model's predictions and its behavior under shifts of origin. The scientific question, the meaning of zero, and the consequences for the fitted mean all matter.

Common Mistake

Wrong reasoning: “The interaction term already contains both predictors, so separate main effects would be redundant.” The product \(x_1x_2\) does not generally represent the separate contributions associated with changing either predictor. It equals zero along either predictor's zero value, which forces the product-only prediction to remain at its intercept there.

Correction: Treat the product and main effects as distinct terms with distinct roles in the mean equation. Unless there is a substantive reason to impose the product-only restrictions, include the intercept, both main effects, and the product term. Then interpret the resulting predictions at the predictor values of interest.