Treatment Coding for Factors With Several Levels

Why Treatment Coding Is Useful

A categorical predictor may have several levels: for example, a hypothetical service site could be classified as local, regional, or specialist. A regression model needs numerical inputs, but assigning the levels arbitrary numbers such as 1, 2, and 3 would imply a numerical spacing and ordering that may not exist. Treatment coding instead represents each nonreference level with an indicator variable. The reference level is represented by zeros for all of those indicators.

This encoding gives a direct way to calculate a fitted value for every category. The intercept supplies the fitted value for the reference category when other predictors are zero, and each category’s indicator determines how its fitted value differs in the model’s linear predictor. The same construction can be used in a linear model or as part of a generalized linear model’s linear predictor.

Definition: For a factor with $K$ levels, treatment coding selects one level as the reference category and creates $K-1$ indicator variables. An observation in the reference category has value 0 for all $K-1$ indicators. An observation in any other category has value 1 for that category’s indicator and 0 for the others.

Building the Indicator Variables

Suppose a factor has levels A, B, and C, with A as the reference category. Define $I_{B,i}=1$ when observation $i$ is in B and 0 otherwise; define $I_{C,i}$ similarly for C. The treatment-coded rows are:

Factor level	$I_B$	$I_C$
A (reference)	0	0
B	1	0
C	0	1

There are $K-1$, not $K$, indicators because one level is represented by the all-zero row. For a factor with four levels, there are three indicators. In R, a factor is generally represented using treatment contrasts by default for a model with an intercept; the factor’s levels and contrast settings determine which level is the reference. Here the important step is to translate the chosen reference and level labels into the correct indicator pattern.

Fitted Values in a Linear Model

First consider a linear regression model with one multi-level factor and possibly other predictors. Assume the conditional mean is linear in the predictors and the model has an intercept. For the usual least-squares model, the errors are assumed to have conditional mean zero; the additional constant-variance and independence assumptions concern inference and do not change how the design row produces a fitted value. With three levels and A as reference, the model can be written:

\widehat{y}_i=\widehat{\beta}_0+\widehat{\beta}_B I_{B,i}+\widehat{\beta}_C I_{C,i}+\widehat{\beta}_x x_i

When $x_i=0$, the fitted values for A, B, and C are respectively $\widehat{\beta}_0$, $\widehat{\beta}_0+\widehat{\beta}_B$, and $\widehat{\beta}_0+\widehat{\beta}_C$. When $x_i$ is not zero, its contribution is added to each category’s expression. In each calculation, only the indicator for the observation’s nonreference level is 1.

Worked Example: Constructing Rows and Fitted Values

A hypothetical model predicts annual service costs in dollars from site type and years of experience, $x$. Local is the reference level. The estimated model is $\widehat{y}=120+18I_{\text{regional}}-7I_{\text{specialist}}+2.5x$. Calculate the fitted value for each of these cases: local with 4 years of experience; regional with 3 years; and specialist with 6 years.

Method. Write the two indicators for each case, then substitute them and the stated experience into the estimated equation. Local is the reference, so its indicator values are both zero.

For the local site, $I_{\text{regional}}=0$ and $I_{\text{specialist}}=0$. Thus the fitted value is $120+18(0)-7(0)+2.5(4)=120+0+0+10=\$130$.

For the regional site, $I_{\text{regional}}=1$ and $I_{\text{specialist}}=0$. Thus the fitted value is $120+18(1)-7(0)+2.5(3)=120+18+0+7.5=\$145.50$.

For the specialist site, $I_{\text{regional}}=0$ and $I_{\text{specialist}}=1$. Thus the fitted value is $120+18(0)-7(1)+2.5(6)=120+0-7+15=\$128$.

Check. Each case includes the same intercept and the experience contribution appropriate to its $x$. The category contribution is zero for local and uses exactly one category indicator for each nonreference site. A common mistake avoided here is setting both indicators to 1 for a single site.

Why Category-Only Least-Squares Fits Equal Group Means

For a category-only linear model, treatment coding does more than provide a convenient representation: ordinary least squares fits each category at its sample mean. The reason follows from minimizing the sum of squared residuals. Within any one category, the model assigns the same fitted value $c$ to every observation. If the observed responses in that category are $y_1,\ldots,y_n$, its contribution to the sum of squares is $\sum_{j=1}^n(y_j-c)^2$. Differentiating with respect to $c$ and setting the derivative to zero gives:

\frac{d}{dc}\sum_{j=1}^n(y_j-c)^2=-2\sum_{j=1}^n(y_j-c)=0 \quad\Longrightarrow\quad c=\frac{1}{n}\sum_{j=1}^n y_j

Therefore, the fitted value for each category is its observed group mean. With A as reference, the intercept is the mean for A, and the indicator coefficient for another category is that category’s mean minus the mean for A. This result assumes an ordinary least-squares model containing the category factor alone and an intercept; additional predictors or a different model structure can change the fitted values.

Worked Example: Deriving Coefficients From Group Means

A hypothetical training study records an outcome score for three program formats. The observed scores are A: 8, 10, 12; B: 13, 14, 15; and C: 6, 8, 10. Fit a category-only least-squares model using A as the reference, and calculate the fitted value for each format.

Method. In a category-only least-squares model, each fitted category value is its sample mean. First calculate the three means:

For A, $(8+10+12)/3=30/3=10$. For B, $(13+14+15)/3=42/3=14$. For C, $(6+8+10)/3=24/3=8$.

The reference category A has fitted value 10, so $\widehat{\beta}_0=10$. The B indicator coefficient is $14-10=4$, and the C indicator coefficient is $8-10=-2$. Thus the treatment-coded equation is $\widehat{y}=10+4I_B-2I_C$.

For A, both indicators are zero, giving $\widehat{y}=10$. For B, $I_B=1$ and $I_C=0$, giving $\widehat{y}=10+4=14$. For C, $I_B=0$ and $I_C=1$, giving $\widehat{y}=10-2=8$.

Check. The three fitted values match the calculated group means. Within each group, the residuals sum to zero: for A they are $-2,0,2$; for B they are $-1,0,1$; and for C they are $-2,0,2$. This is consistent with the least-squares condition derived above. The common mistake avoided is treating each indicator coefficient as the category’s fitted mean rather than as its difference from the reference mean.

Treatment Coding in a Logistic Model

Treatment-coded indicators can also enter a generalized linear model. For a binary response modeled with a logit link, assume $Y_i$ is Bernoulli with conditional probability $p_i=P(Y_i=1\mid\text{predictors})$, and use the model $ \log\{p_i/(1-p_i)\}=\eta_i$. With A as reference and no other predictors, $\eta_i=\beta_0+\beta_B I_{B,i}+\beta_C I_{C,i}$. The indicator construction is unchanged, but the linear predictor is on the log-odds scale; probabilities are obtained by applying the inverse-logit function.

Worked Example: Calculating Fitted Probabilities by Category

A hypothetical logistic model predicts whether a scheduled appointment is missed. A is the reference service arrangement, and the fitted linear predictor is $\widehat{\eta}=-2+0.5I_B-0.4I_C$. Calculate the fitted probability for each arrangement, rounding probabilities to four decimal places.

Method. Assign indicators for each category, calculate the linear predictor, then apply $\widehat{p}=1/(1+\exp(-\widehat{\eta}))$. Keep full precision until the final rounding.

For A, both indicators are zero, so $\widehat{\eta}_A=-2$. Therefore $\widehat{p}_A=1/(1+\exp(2))\approx0.1192029$, which is 0.1192 to four decimal places.

For B, $I_B=1$ and $I_C=0$, so $\widehat{\eta}_B=-2+0.5=-1.5$. Therefore $\widehat{p}_B=1/(1+\exp(1.5))\approx0.1824255$, which is 0.1824 to four decimal places.

For C, $I_B=0$ and $I_C=1$, so $\widehat{\eta}_C=-2-0.4=-2.4$. Therefore $\widehat{p}_C=1/(1+\exp(2.4))\approx0.0831727$, which is 0.0832 to four decimal places.

Check. Each probability lies between 0 and 1, and applying the logit to each unrounded probability returns its corresponding linear predictor. The indicators are used in the linear predictor, not added directly to the probability.

Common Coding Trap

A frequent error is to mark more than one indicator as 1 for an observation that belongs to exactly one level of a single factor. For example, with A as reference and indicators for B and C, a C observation has row $(I_B,I_C)=(0,1)$, not $(1,1)$. If the fitted equation is $\widehat{y}=10+4I_B-2I_C$, the incorrect row $(1,1)$ gives $10+4-2=12$. The correct C row gives $10+4(0)-2(1)=8$. The incorrect result adds a B contribution to an observation that is not in B.

Another check is to inspect the reference row: it must be all zeros for the treatment indicators. If it is not, either the coding has been assigned inconsistently or the stated reference level has not been used. Always map each observation’s factor level to its indicator row before substituting values into the model.

Key takeaway: With $K$ factor levels, treatment coding uses $K-1$ indicators. The reference category has all-zero indicators; each other category has one active indicator. In a linear predictor, the intercept gives the reference-category contribution when other predictors are zero, and the relevant indicator adds that category’s model contribution.

Check Your Understanding

For each problem, first assign the treatment indicators and then calculate the requested fitted value or probability.

A factor has levels A, B, and C, with A as reference. A category-only fitted model is $\widehat{y}=7+2I_B-3I_C$. Give the fitted values for A, B, and C. (Answer: A = 7, B = 9, C = 4.)
A category factor has reference level D and indicators $I_A,I_B,I_C$. The model is $\widehat{y}=20+4I_A-2I_B+I_C+0.5x$. Calculate fitted values at $x=6$ for D, A, B, and C. (Answer: D = 23, A = 27, B = 21, C = 24.)
A category-only least-squares model uses North as the reference. The observed group means are North = 12, East = 17, and West = 9. Give the intercept and the treatment coefficients for East and West. (Answer: Intercept = 12, East coefficient = 5, West coefficient = -3.)
A logistic model has reference level A and fitted linear predictor $\widehat{\eta}=-2+0.5I_B-0.4I_C$. Calculate the fitted probability for C, rounding to four decimal places. (Answer: 0.0832.)
With A as the reference, a model is $\widehat{y}=30+6I_B-5I_C$. An analyst codes an observation in C as $(I_B,I_C)=(1,1)$. State the correct indicator row and fitted value, and identify the incorrect fitted value from the analyst’s row. (Answer: Correct row $(0,1)$, correct fitted value 25; incorrect row gives 31.)

What You'll Learn