Predicted Means Across Multi-Level Factors

From Indicator Coefficients to Group Means

A regression model can include a categorical predictor with several levels, such as four types of service or three regions. Its coefficient table does not usually list one fitted mean for every group directly. Instead, it reports an intercept and indicator coefficients relative to a reference level. To obtain the fitted mean for each group, evaluate the regression equation at that group’s indicator values.

The same calculation works when the model also includes quantitative covariates. In that case, specify the covariate values and use the same values for each group if the goal is to lay out comparable predictions at one profile. These are model-based fitted means; they need not equal the groups’ observed sample averages.

Definition: Suppose a categorical predictor \(G\) has \(K\) levels, numbered \(1,\ldots,K\), and level 1 is the reference level. For each \(j=2,\ldots,K\), define an indicator \(D_j\) that equals 1 when \(G\) is at level \(j\), and equals 0 otherwise. In a linear model with quantitative covariates \(x_1,\ldots,x_q\), the additive conditional-mean model is

\(E(Y\mid G,x_1,\ldots,x_q)=\beta_0+\sum_{j=2}^{K}\beta_jD_j+\sum_{r=1}^{q}\gamma_rx_r.\)

The model assumes that this linear expression describes the conditional mean, with the same covariate slopes \(\gamma_r\) at every factor level. The error has conditional mean zero given the predictors. The fitted mean is obtained by replacing the coefficients with their estimates and evaluating the expression at specified predictor values.

At the reference level, every indicator is 0, so its fitted mean at covariate values \(x_1,\ldots,x_q\) is \(\hat\beta_0+\sum_{r=1}^{q}\hat\gamma_rx_r\). At level \(j\), the indicator \(D_j\) is 1 and all the other factor indicators are 0. Thus the fitted mean there adds \(\hat\beta_j\) to the reference-level expression. If there are no quantitative covariates, the reference mean is simply \(\hat\beta_0\), and the mean for level \(j\) is \(\hat\beta_0+\hat\beta_j\).

Why the Calculation Works

The indicator values encode which factor level applies. For example, when \(G\) is at level 3, \(D_3=1\), while every other indicator is 0. Substituting those values into the model removes all factor terms except \(\beta_3\). The conditional-mean assumption gives the model expression for the expected response; the fitted expression follows by using the estimated coefficients.

\begin{aligned} E(Y\mid G=1,x_1,\ldots,x_q) &=\beta_0+\sum_{r=1}^{q}\gamma_rx_r,\\ E(Y\mid G=j,x_1,\ldots,x_q) &=\beta_0+\beta_j+\sum_{r=1}^{q}\gamma_rx_r,\quad j=2,\ldots,K,\\ \hat\mu_j(x_1,\ldots,x_q) &=\hat\beta_0+\hat\beta_j+\sum_{r=1}^{q}\hat\gamma_rx_r,\quad j=2,\ldots,K, \end{aligned}

In the last line, the reference-level fitted mean is given by the same formula without \(\hat\beta_j\). These calculations rely on an additive model: the covariate slopes do not vary across levels of \(G\). If the model includes interactions between the factor and covariates, this simple formula must be extended to include the corresponding interaction terms.

Worked Example: Four Levels Without Other Predictors

Suppose a hypothetical regression predicts a satisfaction score, measured in points, from a four-level service type. Level A is the reference level. The fitted intercept is 48.0 points, and the estimated indicator coefficients are \(\hat\beta_B=-3.2\), \(\hat\beta_C=4.5\), and \(\hat\beta_D=1.1\). Calculate the fitted mean for every service type.

With no other predictors, the fitted mean for the reference level is the intercept:

\hat\mu_A=\hat\beta_0=48.0\text{ points}.

For each other level, add its indicator coefficient to the intercept:

\begin{aligned} \hat\mu_B&=48.0+(-3.2)=44.8\text{ points},\\ \hat\mu_C&=48.0+4.5=52.5\text{ points},\\ \hat\mu_D&=48.0+1.1=49.1\text{ points}. \end{aligned}

The model-based fitted means are therefore 48.0 points for A, 44.8 for B, 52.5 for C, and 49.1 for D. The negative coefficient for B lowers its fitted mean relative to the reference level; it does not mean that B’s fitted mean is negative.

Adding Covariates: Choose a Profile First

When covariates appear in the model, a factor level alone does not determine the fitted mean. The values of the covariates matter too. A calculation should therefore state the profile at which predictions are made. For a comparison across factor levels at one profile, keep the covariate values fixed throughout the calculations.

Worked Example: Four Groups at One Age

Consider a hypothetical model for a test score in points, with a four-level program factor and age in years. Program A is the reference level. The fitted model is

\hat\mu=20.0+0.35(\text{age})+2.4D_B-1.1D_C+3.0D_D.

Calculate the fitted mean for each program at age 40 years. The model is additive, so it uses the same age slope for all four programs. At age 40, the shared intercept-and-age part is \(20.0+0.35(40)=20.0+14.0=34.0\) points.

\begin{aligned} \hat\mu_A(40)&=20.0+0.35(40)=34.0\text{ points},\\ \hat\mu_B(40)&=20.0+0.35(40)+2.4=36.4\text{ points},\\ \hat\mu_C(40)&=20.0+0.35(40)-1.1=32.9\text{ points},\\ \hat\mu_D(40)&=20.0+0.35(40)+3.0=37.0\text{ points}. \end{aligned}

At the specified age, the fitted means range from 32.9 points for C to 37.0 points for D. Each value is a prediction for the conditional mean score at age 40 under the fitted model, not a guarantee about an individual’s score.

A table makes the full set of predictions easy to inspect. Include the factor level, the reference-coded contribution, and the resulting fitted mean; state the covariate profile in the surrounding text or in a table heading. A compact table can also help reveal arithmetic mistakes, such as using one group’s indicator coefficient for another group.

Worked Example: Three Regions at a Specified Profile

A hypothetical regression predicts an assessment score in points from region and two quantitative covariates: training hours and a baseline score in points. North is the reference region. The fitted equation is

\hat\mu=15.0+2.0D_{\text{East}}-1.5D_{\text{West}} +0.80(\text{training hours})+0.40(\text{baseline score}).

Calculate the fitted means for all regions at 5 training hours and a baseline score of 60 points. First substitute the covariate values. Their combined contribution with the intercept is \(15.0+0.80(5)+0.40(60)=15.0+4.0+24.0=43.0\) points. North has both indicators equal to 0; East has \(D_{\text{East}}=1\); West has \(D_{\text{West}}=1\).

Region	Indicator contribution	Fitted mean at 5 hours and baseline score 60
North (reference)	0	\(43.0+0=43.0\) points
East	2.0	\(43.0+2.0=45.0\) points
West	-1.5	\(43.0-1.5=41.5\) points

The table gives three model-based predictions at exactly the same covariate values. For instance, the fitted mean for West is 41.5 points at 5 training hours and a baseline score of 60 points. These values are conditional predictions from this model; they are not averages over whatever covariate values happen to occur in each region.

Conditions and Interpretation

The indicator calculation is short, but its meaning depends on the model being used. In the additive model above, the conditional mean is linear in the included covariates, and the covariate slopes are common across factor levels. The assumption that the error has conditional mean zero is what allows the regression expression to represent \(E(Y\mid G,x_1,\ldots,x_q)\). If the chosen model does not describe the conditional mean adequately, the calculations still evaluate its equation, but the results may not represent the conditional means well.

For calculating point predictions, normal errors and equal error variances are not required. Those assumptions concern other features of a regression analysis, not the substitution of fitted coefficients into this mean equation. The arithmetic also does not establish that the fitted model is useful or that a predicted mean is precise; those questions require separate evaluation.

The reference coefficient interpretation can be checked directly from the formula. In the additive model, \(\hat\beta_j\) is the amount added to the fitted mean when moving from the reference level to level \(j\) while holding the covariates fixed. This is a model-based difference at the specified covariate values, not necessarily the difference between the groups’ raw sample averages.

Common Mistakes

Treating every indicator coefficient as a group mean. An indicator coefficient is an addition to the reference-level prediction. Add it to the intercept, and, when present, to the covariate contribution.
Forgetting that the reference level has no active indicators. Its indicators are all 0, so its fitted mean comes from the intercept and any covariate terms.
Comparing predictions made at different covariate values. Such predictions combine a factor change with a covariate change. To lay out group predictions at one profile, use the same profile for every level.
Calling fitted means observed averages. A fitted mean comes from the estimated regression equation. It can differ from the observed average for a group, especially when covariates are included.
Using the additive formula when the fitted model includes interactions. With interactions, the factor contribution can depend on covariate values, so those terms must also be included when evaluating each level.

Key takeaway: To calculate fitted means for a multi-level factor, begin with the reference-level prediction, then add the estimated indicator coefficient for the level being evaluated. Include every covariate term at the stated profile, and use the same profile for all levels when constructing a directly comparable set of predictions.

Check Your Understanding

Use the model and definitions in this tutorial to answer the following questions.

A three-level factor has reference-level fitted mean 12.0 and indicator coefficients 1.5 and -2.0 for its other levels. What are the fitted means for all three levels?
In an additive model with age as a covariate, what information must be specified to calculate the fitted mean for a non-reference factor level?
Why does the reference-level fitted mean contain no indicator coefficient for the factor?
A model includes a factor and a quantitative covariate. Why should the covariate value be held fixed when arranging factor-level predictions for comparison at one profile?
True or false: a fitted mean calculated from a regression equation must equal the observed sample average for that factor level. Explain briefly.