Apply Hierarchy to Interactions and Curvature Terms

Why Lower-Order Terms Belong in a Candidate Formula

A proposed regression formula may include a product of predictors or a squared predictor to represent a particular pattern. But writing down only that product or square can impose unintended restrictions on the relationship. The hierarchy principle provides a consistent rule for completing the candidate formula: when a higher-order term is included, include its relevant lower-order terms as well.

This tutorial focuses on applying that rule after a candidate interaction or curvature term has been proposed. Hierarchy tells us which companion terms belong in the formula; by itself, it does not decide whether the scientific question warrants proposing the higher-order term.

What the Hierarchy Principle Requires

Consider a regression model for an outcome \(Y_i\), with measured predictors \(x_i\) and \(z_i\). A model’s formula specifies which functions of those predictors are included in its conditional mean. For example, an interaction term \(x_i z_i\) is a product of two predictors, while a quadratic term \(x_i^2\) represents a squared predictor.

Definition: Under the strong hierarchy principle, including a higher-order term requires including all of its lower-order constituent terms. Thus, a two-way interaction \(xz\) is accompanied by the main effects \(x\) and \(z\); a three-way interaction \(xyz\) is accompanied by all three main effects and all three two-way interactions; and a squared term \(x^2\) is accompanied by the linear term \(x\). In a regression model with an intercept, the intercept is also retained.

The rule follows the structure of the term. For \(xz\), the constituent predictors are \(x\) and \(z\). For \(x^2\), the lower-degree version of the same predictor is \(x\). For \(xyz\), the constituent terms include each predictor by itself and each pair. A candidate formula that includes all these terms is called hierarchical.

Some methods use a weaker condition called weak hierarchy: an interaction is accompanied by at least one of its main effects, rather than both. This tutorial uses strong hierarchy as the default rule for constructing candidate formulas. It is a clear, symmetric convention and avoids retaining a product while omitting one of the predictors that forms it.

A model’s term list is separate from the values of its fitted coefficients. Hierarchy concerns which terms are in the formula, not whether every corresponding coefficient is large, statistically significant, or precisely estimated. A lower-order term can therefore remain in a candidate specification even when its individual test is weak.

Quadratic Terms: Keep the Linear Term

A quadratic term \(x^2\) allows the conditional mean to bend as \(x\) changes. The hierarchical formula includes both the linear and quadratic terms:

\operatorname{E}(Y_i\mid x_i)=\beta_0+\beta_1 x_i+\beta_2 x_i^2.

If the linear term is omitted, the formula becomes \(\beta_0+\beta_2 x_i^2\). That version forces the fitted conditional mean to have the same value at \(x\) and \(-x\), because \(x^2=(-x)^2\). Unless that symmetry is intended, omitting \(x\) imposes an extra shape restriction. Including the linear term allows the curve to have different values on either side of zero.

Worked Example: Completing a Quadratic Candidate Formula

Suppose a hypothetical analyst proposes a quadratic relationship between a measured exposure \(x\), recorded in units, and an outcome \(Y\). The candidate values of \(x\) are 0, 5, and 10 units. Apply hierarchy and write the formula.

The proposed curvature term is \(x^2\), so strong hierarchy requires the linear term \(x\) as well. With an intercept, the candidate conditional mean is:

\operatorname{E}(Y_i\mid x_i)=\beta_0+\beta_1 x_i+\beta_2 x_i^2.

The values of the squared term at the three stated exposure levels are \(0^2=0\), \(5^2=25\), and \(10^2=100\) square units. The linear-term values are \(0\), \(5\), and \(10\) units. Both columns therefore enter the candidate formula, along with the intercept.

At \(x=5\), for example, the formula gives \(\beta_0+5\beta_1+25\beta_2\). A formula containing only \(x^2\) would instead restrict the mean to \(\beta_0+25\beta_2\) at \(x=5\), and would force the same fitted value at \(x=-5\). Including \(x\) avoids imposing that symmetry unless it is intended.

Two-Way Interactions: Keep Both Main Effects

A two-way interaction \(xz\) allows the association between one predictor and the outcome to vary across values of the other predictor. The hierarchical formula includes \(x\) and \(z\) as well as their product:

\operatorname{E}(Y_i\mid x_i,z_i) =\beta_0+\beta_1 x_i+\beta_2 z_i+\beta_3 x_i z_i.

For a fixed value of \(z_i\), the coefficient multiplying \(x_i\) is \(\beta_1+\beta_3 z_i\). Thus, the interaction changes the slope associated with \(x\) across values of \(z\). The main effects give the formula its lower-order components: \(\beta_1\) is the \(x\)-slope when \(z=0\), and \(\beta_2\) is the difference associated with \(z\) when \(x=0\). Those reference values matter for interpretation, though they do not change which terms hierarchy calls for.

Worked Example: Completing an Interaction Formula

In a hypothetical study, \(x\) is study time in hours and \(z\) is an indicator coded 0 for no tutoring and 1 for tutoring. An analyst proposes that the association between study time and a quiz outcome may differ by tutoring group. Identify the required terms and show how the formula behaves at four hours of study.

The proposed interaction is \(xz\). Strong hierarchy requires the main effects \(x\) and \(z\), plus an intercept. The candidate formula is:

\operatorname{E}(Y_i\mid x_i,z_i) =\beta_0+\beta_1 x_i+\beta_2 z_i+\beta_3 x_i z_i.

At \(x=4\) hours and \(z=0\), the interaction value is \(4\times 0=0\), so the fitted conditional mean is \(\beta_0+4\beta_1\). At \(x=4\) and \(z=1\), the interaction value is \(4\times 1=4\), so the mean is \(\beta_0+4\beta_1+\beta_2+4\beta_3\). The difference between those two fitted means is \(\beta_2+4\beta_3\).

If the analyst included \(xz\) but omitted \(z\), that difference would be forced to equal \(4\beta_3\), with no separate group difference at zero study hours. If the analyst omitted \(x\), the formula would not include the common linear study-time component represented by \(\beta_1 x\). The hierarchical candidate retains both main effects rather than imposing either restriction by omission.

Higher-Order Interactions: Include Every Lower Order

The same rule extends to interactions involving more than two predictors. A three-way interaction \(xyz\) is accompanied by the three main effects \(x\), \(y\), and \(z\), and by the three two-way interactions \(xy\), \(xz\), and \(yz\). The complete candidate formula is:

\operatorname{E}(Y_i\mid x_i,y_i,z_i) =\beta_0+\beta_1x_i+\beta_2y_i+\beta_3z_i +\beta_4x_iy_i+\beta_5x_iz_i+\beta_6y_iz_i +\beta_7x_iy_iz_i.

This list contains one intercept, three main effects, three two-way interactions, and one three-way interaction. Omitting any of the lower-order interactions would make the formula nonhierarchical under the strong rule, even if all three main effects were present.

Worked Example: Checking a Three-Way Candidate

A hypothetical analyst proposes \(xyz\), where \(x\), \(y\), and \(z\) are three measured predictors. At one observation, their values are 2, 3, and 4 in their respective measurement units. List the terms required by hierarchy and calculate their values at this observation.

The three main effects are \(x=2\), \(y=3\), and \(z=4\). The two-way products are \(xy=2\times3=6\), \(xz=2\times4=8\), and \(yz=3\times4=12\). The proposed three-way product is \(xyz=2\times3\times4=24\). Thus, the candidate formula must include the intercept and all seven predictor terms: \(x\), \(y\), \(z\), \(xy\), \(xz\), \(yz\), and \(xyz\).

A formula containing only \(x\), \(y\), \(z\), and \(xyz\) would omit all three two-way interactions. A formula containing \(xyz\) alone would omit both the main effects and the two-way interactions. Both would fail strong hierarchy.

Why Centering Does Not Remove the Requirement

Predictors are sometimes centered by subtracting a chosen reference value. Centering can change the interpretation of coefficients, but it does not make hierarchy optional. For example, let \(x_c=x-c\), where \(c\) is a constant. Expanding the centered square gives:

$$ x_c^2=(x-c)^2=x^2-2cx+c^2. $$

With an intercept and a linear term, a model using \(1\), \(x_c\), and \(x_c^2\) spans the same set of quadratic curves as a model using \(1\), \(x\), and \(x^2\). The constant \(c^2\) contributes to the intercept, and the term \(-2cx\) is a linear component. The linear term is therefore part of the hierarchical quadratic specification even when the predictor has been centered.

A similar expansion applies to an interaction. If \(x_c=x-c\) and \(z_d=z-d\), then:

$$ x_c z_d=(x-c)(z-d)=xz-dx-cz+cd. $$

The centered product combines the uncentered interaction \(xz\), both main effects, and a constant. Retaining the intercept and both main effects means that changing the centering values changes the coefficient representation, not the basic collection of curves available to the model. This is one reason a consistent hierarchical term list is useful.

A Practical Hierarchy Check

Before finalizing a candidate formula, inspect each proposed higher-order term and list the lower-order terms it implies. The following check applies directly to polynomial and interaction terms:

Identify the higher-order term.
For example, decide whether the proposed term is \(x^2\), \(xz\), or \(xyz\).

Write its constituent lower-order terms.
For \(x^2\), write \(x\). For \(xz\), write \(x\) and \(z\). For \(xyz\), write \(x\), \(y\), \(z\), \(xy\), \(xz\), and \(yz\).

Check the complete candidate formula.
Confirm that every required lower-order term and the intercept are present. Do not use individual significance tests as a substitute for this structural check.

This procedure checks hierarchy only. Other considerations are still needed to decide whether the candidate terms are scientifically appropriate, estimable from the available data, or suitable for the intended use.

Common Mistake: Keeping the Product but Dropping Its Components

A frequent error is to include an interaction because it is of interest, then remove one or both main effects because their individual tests appear weak. That changes the meaning of the formula: it forces the omitted lower-order contribution to be zero under the model, rather than merely allowing the data to estimate it.

The correction is to distinguish term inclusion from coefficient evidence. Under strong hierarchy, retain the required lower-order terms when the higher-order term is in the candidate formula. This is a rule for coherent specification, not a claim that every coefficient must be nonzero or that the higher-order term must be retained in every possible model.

Key takeaway: A squared term \(x^2\) calls for \(x\); an interaction \(xz\) calls for both \(x\) and \(z\); and a three-way interaction \(xyz\) calls for all of its main effects and two-way interactions. Check the full term set before interpreting or evaluating a candidate formula.

Check Your Understanding

For each question, identify the terms required by strong hierarchy.

A candidate formula includes \(w^2\). Which lower-order predictor term belongs in the formula?
A candidate formula includes \(ab\). List all required main effects and state whether the intercept is retained.
A candidate formula includes \(rst\). List every main effect and two-way interaction required by hierarchy.
Why can a formula containing \(x^2\) but not \(x\) force an unintended restriction on the fitted curve?
An interaction coefficient has a weak individual test, but the interaction remains in the candidate formula. What does strong hierarchy imply about its main effects?