Predicted Means for a Two-Level Factor

From a Two-Level Factor to Two Fitted Means

A regression model can include a factor such as program status, clinic type, or whether a policy is in place. If the factor has two levels, reference coding represents it with one indicator variable. The fitted regression equation can then provide a predicted mean for either level. When the model includes other predictors, each prediction also depends on the values specified for those covariates.

The key is to keep the factor level and the covariate profile explicit. A prediction for one level is not a universal mean for everyone in that group when other predictors are in the model: it is the model’s fitted mean at the particular covariate values entered.

The Model and Its Fitted Means

Suppose \(Y_i\) is a quantitative response for observation \(i\), \(D_i\) is the indicator for a two-level factor, and \(x_{i1},\ldots,x_{ip}\) are additional covariates. An additive linear model is

Here the conditional mean is assumed to follow the displayed linear expression, so \(E(\varepsilon_i\mid D_i,x_{i1},\ldots,x_{ip})=0\). The factor has the same additive shift at every covariate profile: this model contains no interaction between \(D_i\) and the covariates. For ordinary least squares fitting, observations are assumed to be independent, and the specified conditional mean must be appropriate for the prediction being made. Constant error variance and normal errors are additional assumptions used for some forms of inference, but are not needed just to substitute values into a fitted equation.

After fitting the model, replace each coefficient by its estimate. With one covariate for illustration, the fitted equation is \(\widehat{Y}=\hat{\beta}_0+\hat{\beta}_1D+\hat{\beta}_2x\). Setting \(D=0\) gives the reference-level fitted mean; setting \(D=1\) gives the comparison-level fitted mean. Covariate values must be supplied for both.

In the model with several covariates, include every estimated covariate term in both calculations. The intercept is the fitted mean for the reference level when all numerical covariates equal zero; it need not describe a practically relevant person or case. The fitted mean for the comparison level also includes \(\hat{\beta}_1\), the estimated indicator coefficient.

Why the Factor-Only Model Gives Group Averages

This result follows directly from least squares. Let \(a\) be the fitted value for the reference group and \(c\) the fitted value for the comparison group. The intercept-and-indicator model can represent these two values because \(a=\beta_0\) and \(c=\beta_0+\beta_1\). Its residual sum of squares is the sum of squared deviations from \(a\) in the reference group and from \(c\) in the comparison group.

For any group with observations \(y_1,\ldots,y_m\) and average \(\bar{y}\), the following identity holds for any proposed fitted value \(q\):

The first term on the right does not depend on \(q\), and the second is minimized when \(q=\bar{y}\). Thus least squares chooses the average of the observed responses separately in each group. The fitted intercept is the reference-group average, and the indicator coefficient is the comparison-group average minus the reference-group average. This exact group-average result applies to the factor-only model; with covariates included, predictions instead depend on the supplied covariate profile.

Adding Covariates: Predict at a Specified Profile

When covariates are included, calculate a fitted mean by putting the desired values into every term of the fitted equation. To predict for both levels of the factor at the same covariate profile, keep the covariate values fixed and change only \(D\). To predict at a different profile, substitute the new covariate values as well. These are profile-based predictions, not necessarily averages over all people observed at either factor level.

Reading the Predictions Carefully

A fitted mean is the value of the model’s estimated conditional mean at the specified predictor values. It describes the average response represented by the model for cases with those values, not the exact response of a particular individual. Individual outcomes can vary around the conditional mean.

Check three things before reporting a prediction:

• Reference coding: state which factor level has \(D=0\) and which has \(D=1\).
• Profile: report the values used for each additional covariate, including units.
• Response: give the predicted value in the response’s units and label it as a fitted mean.

It is also important that a profile be meaningful for the fitted model. For example, substituting covariate values far outside the range represented by the data requires extrapolation, and the prediction may not be reliable. A correct substitution is necessary, but it does not by itself ensure that the model is appropriate for every profile.

Common Mistake: Treating the Indicator Coefficient as the Prediction

A common error is to report \(\hat{\beta}_1\) itself as the fitted mean for the comparison group. In the additive reference-coded model, \(\hat{\beta}_1\) is only the indicator’s contribution to the equation. The comparison-level fitted mean also contains the intercept and all covariate contributions. For example, in the blood-pressure model, the fitted mean for the new program at age 50 and baseline pressure 120 mmHg is 91 mmHg, not \(-3\) mmHg.

Another mistake is to give a covariate-based prediction without stating the covariate values. A fitted mean of 94 mmHg in the example is tied to age 50 years and baseline pressure 120 mmHg; it should not be presented as the mean for all people receiving standard care. State the factor level, the covariate profile, and the predicted response together.