Group-Specific Intercepts From Factor Effects

Why a Factor Can Give Each Group Its Own Fitted Intercept

A categorical predictor can be used to estimate a separate fitted response for each of its groups. In a linear model with a factor and no other predictors, the fitted value for a group is its sample mean. Under treatment coding, the intercept is the fitted value for a designated reference group, and each factor coefficient is the difference between another group’s fitted value and that reference value.

This representation is useful when the modeling question is descriptive or predictive: how do the observed average responses compare across categories, and what fitted value does the model assign to each category? It does not, by itself, explain why the groups differ. In particular, a factor coefficient is not automatically a causal effect.

Definition: A factor-only linear model has a quantitative response and one categorical predictor, with no other predictors. With $K$ factor levels and treatment coding, its model is $Y_i=\beta_0+\sum_{k=2}^{K}\beta_k I(G_i=k)+\epsilon_i$, where level 1 is the reference, $I(G_i=k)$ equals 1 for an observation in level $k$ and 0 otherwise, and $\epsilon_i$ is the error. The group-specific fitted intercept is the model’s fitted response for that group: $\beta_0$ for the reference and $\beta_0+\beta_k$ for level $k$.

The model describes a separate conditional mean for each group. The least-squares fitted value for a group does not depend on the order of its observations: every member of that group receives the same fitted value. The residuals, which are observed responses minus fitted responses, can still differ within the group.

For interpreting the coefficients, treatment coding is especially direct. The model’s reference category determines the intercept, while each other coefficient is a difference from that reference. Changing the reference changes the coefficient values, but it does not change the fitted value assigned to any group.

Why the Fitted Value Is the Group Mean

The least-squares result follows by considering the model’s sum of squared residuals separately within each group. Suppose group $k$ contains $n_k$ responses $y_{k1},\ldots,y_{kn_k}$, and the model assigns that group a fitted value $c_k$. The contribution of that group to the sum of squared residuals is $\sum_{j=1}^{n_k}(y_{kj}-c_k)^2$. The value of $c_k$ that minimizes this quantity is the group’s arithmetic mean.

\begin{aligned} Q_k(c_k)&=\sum_{j=1}^{n_k}(y_{kj}-c_k)^2,\\ \frac{dQ_k}{dc_k}&=-2\sum_{j=1}^{n_k}(y_{kj}-c_k),\\ \frac{dQ_k}{dc_k}=0 &\Longrightarrow \sum_{j=1}^{n_k}y_{kj}-n_kc_k=0\\ &\Longrightarrow c_k=\bar y_k=\frac{1}{n_k}\sum_{j=1}^{n_k}y_{kj}. \end{aligned}

The second derivative is $d^2Q_k/dc_k^2=2n_k>0$, so this stationary point is a minimum. Because the factor-only model can choose a separate $c_k$ for each observed group, minimizing the total sum of squared residuals minimizes each group’s contribution at its mean. This result concerns the least-squares fit; assumptions about errors are needed for standard errors and formal inference, not to establish the minimizing fitted values.

Model assumptions: The factor-only linear model specifies a mean response for each factor level. For the usual classical standard errors and exact small-sample $t$- and $F$-inference, errors are assumed independent with mean zero and common variance; normal errors are additionally used for exact finite-sample reference distributions. Least-squares group means can still be computed without normal errors. A group with no observations has no estimable group mean, and its fitted value cannot be learned from this factor-only model.

From Group Means to Treatment-Coded Coefficients

For a factor with reference level 1, let $\bar y_k$ denote the observed response mean for level $k$. Because the fitted value for each group is its mean, the treatment-coded parameters satisfy

\hat\beta_0=\bar y_1,\qquad \hat\beta_k=\bar y_k-\bar y_1\quad (k=2,\ldots,K).

Here $\hat\beta_0$ is the fitted intercept for the reference group. For a non-reference group $k$, its fitted intercept is the sum $\hat\beta_0+\hat\beta_k$, which equals $\bar y_k$. Thus, the coefficient is a difference between fitted group means; it is not the group’s fitted value on its own.

In R, a factor-only model can be fit with a formula such as lm(response ~ group, data = dat). With treatment contrasts, the first factor level is the reference unless the levels or contrasts have been explicitly changed. Check the factor levels rather than guessing the reference from how the data were described. For example, relevel(dat$group, ref = "Alpha") sets Alpha as the reference. The exact model-matrix columns depend on the factor’s contrasts, so coefficient interpretation should always be tied to the coding actually used.

Worked Example: Recovering Three Group-Specific Fitted Values

A hypothetical service team records resolution times, in hours, for three support queues. Alpha has observed times 8, 10, and 12; Beta has 13 and 15; Gamma has 7, 9, and 11. Fit a factor-only least-squares model using Alpha as the treatment-coded reference. Find all three fitted values and coefficients, then check the fit using the residual sum of squares.

Method. Compute each group mean first. With Alpha as the reference, the intercept is Alpha’s mean, and each other coefficient is that group’s mean minus Alpha’s mean.

Alpha mean: $(8+10+12)/3=30/3=10$. Beta mean: $(13+15)/2=28/2=14$. Gamma mean: $(7+9+11)/3=27/3=9$.

The estimated intercept is $\hat\beta_0=10$. The Beta coefficient is $\hat\beta_{\text{Beta}}=14-10=4$. The Gamma coefficient is $\hat\beta_{\text{Gamma}}=9-10=-1$. Therefore, the model’s fitted values are Alpha: $10$, Beta: $10+4=14$, and Gamma: $10+(-1)=9$ hours.

Check: Alpha’s residuals are $8-10=-2$, $10-10=0$, and $12-10=2$, giving squared residuals $4,0,4$, which sum to 8. Beta’s residuals are $13-14=-1$ and $15-14=1$, giving $1+1=2$. Gamma’s residuals are $7-9=-2$, $9-9=0$, and $11-9=2$, giving $4+0+4=8$. The total residual sum of squares is $8+2+8=18$. Each group’s residuals sum to zero, as expected when its fitted value is its mean.

Common mistake avoided: The Beta coefficient is $4$, not $14$. Fourteen is Beta’s fitted value; the treatment-coded coefficient is Beta’s difference from the Alpha reference fitted value.

Changing the Reference Changes the Coefficients, Not the Fit

A reference category is a choice of parameterization. If the reference changes, the intercept becomes the fitted value for the new reference, and the other coefficients become differences from it. The predicted response for every observed group remains its group mean. This is useful when a different comparison is easier to communicate, but a changed coefficient should not be mistaken for a changed underlying model fit.

Worked Example: Re-expressing the Same Fits With a New Reference

A hypothetical operations study records average monthly processing cost, in dollars, for three teams. North has four observations totaling $480, Central has five observations totaling $540, and South has three observations totaling $396. Find the group-specific fitted values and treatment-coded coefficients first with Central as the reference, then with South as the reference. Confirm that the fitted values are unchanged.

Method. Divide each group total by its observation count to obtain its fitted value. Then subtract the selected reference mean from each non-reference mean.

The North mean is $480/4=120$. The Central mean is $540/5=108$. The South mean is $396/3=132$. Thus the group-specific fitted values are $120, $108, and $132, respectively.

With Central as reference, the intercept is $\hat\beta_0=108$. The North coefficient is $120-108=12$, and the South coefficient is $132-108=24$. The implied fitted values are Central: $108$, North: $108+12=120$, and South: $108+24=132$.

With South as reference, the intercept is $\hat\beta_0=132$. The North coefficient is $120-132=-12$, and the Central coefficient is $108-132=-24$. The implied fitted values are South: $132$, North: $132+(-12)=120$, and Central: $132+(-24)=108$. Both parameterizations therefore produce exactly the same fitted value for each team.

Check: In either parameterization, the fitted values in North, Central, and South order are $120,108,132$. Common mistake avoided: A changed intercept or coefficient does not mean the fitted group averages changed; it means the comparisons are now expressed relative to a different group.

Interpret Group Differences Without Claiming Causation

A group coefficient in a factor-only regression describes an observed difference in mean response between that group and the reference. It does not establish that membership in the group caused the difference. Groups may differ in other ways that are not included in the model, and those differences may help explain the observed response contrast.

This distinction matters in predictive analytics. A group factor can provide useful information for predicting responses among groups represented in the modeling data, while still being an imperfect description of how the response would change if an individual or organization were assigned to another group. A predictive association is not, by itself, a causal effect.

Worked Example: Reporting a Group Difference as an Association

A hypothetical insurer reviews claim amounts, in thousands of dollars, for policies sold through two channels. Four online-channel claims are 10, 12, 14, and 16. Three agent-channel claims are 12, 15, and 18. Fit a factor-only model with Online as the reference. State the fitted claim amount for each channel and explain what the coefficient does and does not mean.

Method. Use the sample mean in each channel as its fitted value, then take Agent’s mean minus Online’s mean to obtain the treatment-coded coefficient.

The Online total is $10+12+14+16=52$, so its mean is $52/4=13$ thousand dollars. The Agent total is $12+15+18=45$, so its mean is $45/3=15$ thousand dollars. The estimated intercept is $\hat\beta_0=13$, and the Agent coefficient is $\hat\beta_{\text{Agent}}=15-13=2$.

The fitted claim amount is $13 thousand for Online and $13+2=15$ thousand for Agent. The coefficient says that the observed Agent-channel mean is $2 thousand higher than the observed Online-channel mean in this data. It does not show that changing a policy’s sales channel would cause its claim amount to increase by $2 thousand. For example, the two channels could serve different policyholder or risk mixes, and this model does not adjust for those differences.

Check: The Online residuals are $-3,-1,1,3$, which sum to zero; the Agent residuals are $-3,0,3$, which also sum to zero. The group means, fitted values, and coefficient are consistent. Common mistake avoided: Calling the $2 thousand contrast a causal channel effect overstates what a factor-only observational comparison establishes.

Common Interpretation and Prediction Traps

Calling a non-reference coefficient the group mean. Under treatment coding, the group’s fitted value is the intercept plus its coefficient. The coefficient alone is the difference from the reference.
Ignoring the reference level. The intercept and each contrast cannot be interpreted correctly until the reference category is known.
Treating a reference change as a refit with different predictions. For the same factor-only least-squares model, a change of treatment reference changes the coordinate system for the coefficients, not the fitted group means.
Describing an association as a cause. A factor coefficient reports a fitted contrast. Causal interpretation requires additional design and assumptions not supplied by this regression alone.
Predicting for an unobserved factor level as if its effect were estimated. A factor-only model has no observed group mean for a level absent from the fitting data, so it cannot estimate a data-based fitted value for that level.
Assuming one fitted value per group is always adequate. The factor-only model assigns the same fitted value to every observation in a group. If important within-group predictors matter, this simple representation may leave systematic variation unexplained.

Key takeaway: In a treatment-coded factor-only least-squares model, the intercept is the reference group’s mean, each other factor coefficient is that group’s mean minus the reference mean, and each group’s fitted value is its observed mean. Interpret these as model-based group differences, not automatically as causal effects.

Check Your Understanding

For each problem, identify the fitted group values and interpret the treatment-coded coefficients in context.

A factor-only model has reference group Cedar with fitted value 42. The coefficient for Birch is $-6$. What is Birch’s fitted value, and how does it compare with Cedar? (Answer: Birch’s fitted value is 36, which is 6 lower than Cedar’s.)
A hypothetical call center records waiting times of 4, 6, and 8 minutes for Desk A, and 5 and 9 minutes for Desk B. With A as the reference, find the intercept, B coefficient, and both fitted values. (Answer: The intercept is 6 minutes, the B coefficient is 1 minute, and fitted values are A = 6 and B = 7 minutes.)
A factor has group means East = 30, West = 24, and Central = 27. With West as the reference, give the intercept and the East and Central coefficients. Then state the East fitted value. (Answer: The intercept is 24, the East coefficient is 6, the Central coefficient is 3, and the East fitted value is 30.)
A treatment-coded model uses East as its reference, with fitted group means East = 50, North = 56, and South = 47. What are the intercept and the North and South coefficients? If South becomes the reference, what are the new intercept and the North coefficient? (Answer: With East as reference, the intercept is 50, North is 6, and South is $-3$; with South as reference, the intercept is 47 and North is 9.)
A hypothetical observational study finds mean repair costs of $900 for Group M and $1,050 for Group N. With M as the reference in a factor-only model, interpret the coefficient for N and state one conclusion it does not justify. (Answer: The N coefficient is $150, so the observed fitted mean for N is $150 higher; this alone does not justify claiming that group membership causes repair cost to increase by $150.)

What You'll Learn