Introduction
A regression formula is more than a list of variables. It describes how the fitted response is allowed to change as predictors change. Before estimating a candidate model, it is therefore useful to ask what relationships its terms permit—and what relationships they rule out. A formula can be simple to fit yet answer the wrong scientific question if its restrictions do not match the question being asked.
This tutorial focuses on additive formulas: formulas that include predictor terms but no product term between them. For a numerical predictor and a group indicator, this structure has a direct interpretation. It allows a group difference in fitted response, while requiring the fitted change per unit of the numerical predictor to be the same in each group.
Translate the Formula Into Fitted Relationships
The formula can be read by substituting each group’s indicator value. For the reference group, \(G=0\), so the conditional mean is \(\beta_0+\beta_1x\). For the other group, \(G=1\), it is \(\beta_0+\beta_1x+\beta_2\). The intercept for the second group is shifted by \(\beta_2\), but the coefficient of \(x\) remains \(\beta_1\).
This gives two key implications. First, at any fixed value of \(x\), the fitted difference between the groups is \(\beta_2\). Second, within either group, increasing \(x\) by one unit changes the fitted mean by \(\beta_1\). The lines are parallel: a formula without an interaction does not allow the fitted group difference to vary with \(x\).
The restrictions can be justified directly by subtraction. Comparing the conditional means for the two groups at the same \(x\) cancels the intercept and the \(x\) contribution. Comparing two \(x\)-values within one group cancels the group contribution.
The first difference does not depend on \(x\), and the second does not depend on \(G\). That is the precise sense in which the group gap is constant and the slope is shared. If the scientific question requires a group gap that changes with \(x\), the additive formula does not express that relationship.
Worked Example: Interpreting a Two-Group Formula
Worked Example: Interpreting a Two-Group Formula
Suppose a hypothetical experiment records plant height \(Y\), in centimeters, at different daily light exposures \(x\), measured in hours. Plants are in either standard nutrient conditions (\(G=0\)) or enriched nutrient conditions (\(G=1\)). A candidate formula implies the conditional mean
For standard nutrient conditions, substitute \(G=0\):
At 2 hours of light, the fitted mean is \(12+1.8(2)=15.6\) cm. At 6 hours, it is \(12+1.8(6)=22.8\) cm. The fitted increase is \(22.8-15.6=7.2\) cm over 4 hours, or \(7.2/4=1.8\) cm per additional hour.
For enriched nutrient conditions, substitute \(G=1\):
At 2 hours, the fitted mean is \(16+1.8(2)=19.6\) cm; at 6 hours, it is \(16+1.8(6)=26.8\) cm. At both light exposures, the fitted difference is \(4\) cm. The formula therefore represents a question about a constant nutrient-group difference across light exposures. It would not represent a scientific question in which the nutrient effect is expected to grow or shrink as light exposure increases.
Check the Restriction Against Grouped Data
Grouped summaries can make the implications easier to see. Imagine that hypothetical observations are summarized by group at several values of \(x\). For each group, calculate the change in the group mean over a change in \(x\). If those changes are alike, the summaries are compatible with a common-slope description. Likewise, compare the group means at the same \(x\): a constant difference is compatible with the additive group term.
Compatibility is not proof that the model is scientifically correct. Grouped means may vary because of sampling variation, and summaries can conceal important patterns within groups. The purpose of this check is to make the formula’s restriction visible and ask whether it is plausible for the question at hand.
Worked Example: Checking Parallel Group Trends in Summaries
A hypothetical analyst records a response score at three practice amounts for learners using either Method A or Method B. The table gives the mean score in each group at each amount.
| Practice amount, \(x\) | Mean score, Method A | Mean score, Method B |
|---|---|---|
| 0 | 10 | 13 |
| 2 | 14 | 17 |
| 4 | 18 | 21 |
The change in Method A’s mean from \(x=0\) to \(x=4\) is \(18-10=8\) points over 4 practice units, giving \(8/4=2\) points per unit. Method B’s change is \(21-13=8\) points over the same interval, also \(8/4=2\) points per unit. The group differences are \(13-10=3\), \(17-14=3\), and \(21-18=3\) points at the three observed practice amounts.
Thus these summaries are exactly compatible with parallel fitted relationships: a shared slope of 2 and a constant Method B difference of 3 points. A formula with an additive practice term and method term expresses those restrictions. If the scientific question is whether the methods differ by a stable amount across practice levels, that structure matches the question’s stated form. The summaries alone do not establish that the same pattern holds beyond the observed data or at the individual level.
What a Group-Only Formula Permits
Not every additive formula has a numerical predictor. A formula containing only a categorical group predictor permits a separate fitted mean for each group, but it does not describe a trend within groups or adjust for another variable. For a three-level group \(A,B,C\), one way to write the mean is
where \(I(G=B)\) equals 1 for members of group B and 0 otherwise, with the same definition for group C. Group A is the reference. The fitted means are \(\beta_0\) for A, \(\beta_0+\beta_B\) for B, and \(\beta_0+\beta_C\) for C. This parameterization places no requirement that the three means be equally spaced or ordered.
Worked Example: Deciding Whether a Group-Only Formula Fits the Question
A hypothetical service evaluation records a satisfaction score \(Y\) for clients at three service centers. The scientific question is: “Do average satisfaction scores differ among the three centers?” A candidate conditional-mean formula is
For center A, both indicators are 0, so its fitted mean is \(\beta_0\). For center B, \(I(G=B)=1\) and \(I(G=C)=0\), so its fitted mean is \(\beta_0+\beta_B\). For center C, the fitted mean is \(\beta_0+\beta_C\). Each center has its own fitted mean, so this formula permits the comparisons named in the question without forcing a particular ordering or equal spacing.
Now suppose the question instead asks whether satisfaction changes with clients’ waiting time within each center. The group-only formula contains no waiting-time term. For any fixed center, it gives the same fitted mean at every waiting time. Therefore it does not express the requested within-center relationship. The issue is not whether the formula can be calculated; it is whether its permitted fitted relationships address the scientific question.
Assumptions and Scope of the Check
To interpret these formulas as conditional means, write the response for observation \(i\) as its specified mean plus an error:
The mean-structure assumption is \(E(\epsilon_i\mid x_i,G_i)=0\). It says that, at given predictor values and group, the error has mean zero, so the stated expression is the conditional mean of \(Y_i\). It does not say that each error is zero, that all observations follow the fitted mean exactly, or that group membership causes a response difference.
The formula check concerns this mean structure. If a later analysis uses standard regression inference, additional conditions—such as an appropriate sampling design and assumptions about errors—must be assessed for that procedure. A visual match between group summaries and parallel lines does not verify those conditions. Nor does an additive formula itself justify causal language: interpreting a difference causally requires a design and assumptions that support a causal conclusion.
Common Mistakes
- Assuming “additive” means groups have the same fitted mean. The group term can shift the fitted mean. What is shared here is the slope for the numerical predictor, not the group level.
- Reading a group coefficient as an overall difference regardless of \(x\). In the two-group formula, \(\beta_2\) is a difference at the same \(x\). The model makes that difference constant across \(x\); it is not a comparison of groups with potentially different distributions of \(x\).
- Treating fitted means as individual outcomes. A fitted mean of 20 cm does not predict that every plant will be exactly 20 cm tall. Individual responses can vary around the conditional mean.
- Taking agreement in grouped summaries as proof. Similar slopes and gaps in a small table are evidence of compatibility, not proof of a true constant difference or a complete account of the data.
- Choosing a formula only because it is convenient. A formula should encode the relationship relevant to the scientific question. A group-only formula cannot answer a question about a within-group trend if it omits the predictor defining that trend.
Check Your Understanding
For each question, identify the fitted relationship implied by the formula and consider whether it suits the stated purpose.
- For \(E(Y\mid x,G)=\beta_0+\beta_1x+\beta_2G\), what is the fitted difference between groups at a fixed value of \(x\)?
- In the same formula, how much does the fitted mean change when \(x\) increases by 3 units while group is held fixed?
- A grouped summary shows a group difference of 2 points at \(x=1\) and 6 points at \(x=5\). Is that pattern compatible with an exactly constant group difference in the conditional mean? Explain.
- What fitted means does a group-only formula allow for a three-level categorical predictor, and what within-group relationship does it leave unspecified?
- Why does a close match between observed group means and parallel fitted lines not establish that the formula is scientifically correct?