Coding Multi-Level Factors in Practice

One Factor, Several Equivalent Coordinate Systems

A factor with four levels contributes three degrees of freedom to a model that also has an intercept. There are several ways to represent those three degrees of freedom in the design matrix. Reference coding compares each nonreference level with one chosen level; effect coding describes deviations from an equal-level mean; custom contrasts encode comparisons selected for a scientific question.

The coding determines what the coefficients mean, but it need not determine the fitted model. When two codings span the same set of linear predictors, they are different coordinate systems for the same model. The practical task is therefore twofold: know what each coefficient represents, and be able to reconstruct the predictor for every factor level.

Definition: Let a categorical predictor have levels A, B, C, and D. Its level-specific linear predictor at level \(l\) is the value \(\eta_l\) obtained by substituting that level’s design row into the model’s linear predictor. In a GLM with link \(g\), the corresponding mean is \(\mu_l=g^{-1}(\eta_l)\). A contrast column assigns one number to each level; it is a contrast when its four entries sum to zero.

Throughout the examples, consider a model with an intercept and this factor as its only predictor. With other predictors present, the same calculations apply to the factor’s contribution while those other terms are held fixed. Assume all four levels are represented in the data whenever discussing estimability of all four level predictors.

Reference Coding

Choose A as the reference level. The design row for A has zeros in all three factor columns; the rows for B, C, and D each have a one in their own column. The model’s linear predictors are

\eta_A=\alpha,\qquad \eta_B=\alpha+\beta_B,\qquad \eta_C=\alpha+\beta_C,\qquad \eta_D=\alpha+\beta_D.

Thus, \(\alpha\) is the predictor at A, and each \(\beta\) is a difference from A on the linear-predictor scale. In a logistic regression, for example, these are differences in log odds, not differences in probabilities.

Worked Example: Reading Reference-Coded Coefficients

Suppose a Gaussian GLM with identity link uses reference coding with A as the reference, and the fitted coefficients are \(\hat\alpha=1\), \(\hat\beta_B=2\), \(\hat\beta_C=1\), and \(\hat\beta_D=4\). Find the fitted linear predictor at each level.

At A, all three indicator columns are zero, so \(\hat\eta_A=\hat\alpha=1\).

At B, only the B column contributes: \(\hat\eta_B=1+2=3\).

At C, \(\hat\eta_C=1+1=2\). At D, \(\hat\eta_D=1+4=5\).

The four fitted predictors are therefore \(1,3,2,5\). With the identity link, these are also the four fitted means. The coefficients \(2,1,4\) are respectively the differences \(3-1\), \(2-1\), and \(5-1\).

Effect Coding and an Equal-Level Mean

For a common form of effect coding, assign three columns to A, B, and C. A level receives \(1\) in its own column and \(0\) in the other two; D receives \(-1\) in all three columns. The level predictors become

\begin{aligned} \eta_A&=\alpha+\gamma_A,\\ \eta_B&=\alpha+\gamma_B,\\ \eta_C&=\alpha+\gamma_C,\\ \eta_D&=\alpha-\gamma_A-\gamma_B-\gamma_C. \end{aligned}

Each column sums to zero across the four levels. Consequently, the three deviations cancel when the level predictors are added, and \(\alpha\) is their unweighted mean. This is an equal-level mean: it gives each factor level equal weight, regardless of how many observations occur at that level.

Result: Under the effect coding just defined, \(\alpha=(\eta_A+\eta_B+\eta_C+\eta_D)/4\), and \(\gamma_A=\eta_A-\alpha\), \(\gamma_B=\eta_B-\alpha\), and \(\gamma_C=\eta_C-\alpha\). The D deviation is not an additional coefficient: it is \(\eta_D-\alpha=-(\gamma_A+\gamma_B+\gamma_C)\).

To verify the result, add the four displayed equations. Each \(\gamma\) appears once with a plus sign and once with a minus sign, leaving \(\eta_A+\eta_B+\eta_C+\eta_D=4\alpha\). Subtracting \(\alpha\) from the first three equations gives the stated deviations. Subtracting \(\alpha\) from the D equation gives the stated negative sum. These equations also show how to recover every predictor from the coefficients. \(\square\)

Worked Example: Effect-Coded Coefficients

Use the same four fitted predictors \(1,3,2,5\), now expressed with the effect coding above. First calculate their equal-level mean:

\hat\alpha=\frac{1+3+2+5}{4}=\frac{11}{4}=2.75.

The first three deviations from this mean are \(\hat\gamma_A=1-2.75=-1.75\), \(\hat\gamma_B=3-2.75=0.25\), and \(\hat\gamma_C=2-2.75=-0.75\). The implied deviation at D is \(-(-1.75+0.25-0.75)=2.25\), which equals \(5-2.75\).

Substitution checks all four predictors. At A, \(2.75-1.75=1\); at B, \(2.75+0.25=3\); at C, \(2.75-0.75=2\); and at D, \(2.75-(-1.75)-0.25-(-0.75)=5\). Their mean is \((1+3+2+5)/4=2.75\), as required.

Custom Contrasts for Planned Comparisons

A custom coding begins by specifying the comparisons of interest. Here is one choice with three columns: A versus B, C versus D, and the average of A and B versus the average of C and D. Rows are ordered A, B, C, D.

Level	A versus B	C versus D	Mean of AB versus mean of CD
A	1	0	1/2
B	-1	0	1/2
C	0	1	-1/2
D	0	-1	-1/2

Every column sums to zero, and the columns are linearly independent. With coefficient vector \((\delta_1,\delta_2,\delta_3)\), the level predictor is the intercept plus the row’s weighted sum of these columns. In this particular scaling, \(\eta_A-\eta_B=2\delta_1\), \(\eta_C-\eta_D=2\delta_2\), and the mean predictor for A and B minus the mean predictor for C and D is \(\delta_3\). The coefficient scales follow from the entries in the table; they should not be guessed from the contrast names alone.

Because each column sums to zero, the intercept in this coding is again the equal-level mean. A custom contrast coefficient does not generally equal a raw difference between individual levels: its exact interpretation follows by applying the contrast column to the level predictors.

Worked Example: Custom Contrasts in a Logistic Model

Suppose the four fitted linear predictors are \(\hat\eta_A=-1\), \(\hat\eta_B=0\), \(\hat\eta_C=0.5\), and \(\hat\eta_D=1.2\). Use the custom contrast matrix above to find the coefficients, then calculate the fitted probabilities for a logistic GLM, whose inverse link is \(g^{-1}(\eta)=\exp(\eta)/(1+\exp(\eta))\).

The intercept is the equal-level mean: \(\hat\alpha=(-1+0+0.5+1.2)/4=0.7/4=0.175\). Since \(\eta_A-\eta_B=2\delta_1\), \(\hat\delta_1=(-1-0)/2=-0.5\). Similarly, \(\hat\delta_2=(0.5-1.2)/2=-0.35\). The mean for A and B is \((-1+0)/2=-0.5\), and the mean for C and D is \((0.5+1.2)/2=0.85\), so \(\hat\delta_3=-0.5-0.85=-1.35\).

Reconstructing the predictors checks the coding: A gives \(0.175-0.5-1.35/2=-1\); B gives \(0.175+0.5-1.35/2=0\); C gives \(0.175-0.35+1.35/2=0.5\); and D gives \(0.175+0.35+1.35/2=1.2\).

Now transform each predictor. At A, \(\hat\mu_A=\exp(-1)/(1+\exp(-1))\approx0.269\). At B, \(\hat\mu_B=1/(1+1)=0.500\). At C, \(\hat\mu_C=\exp(0.5)/(1+\exp(0.5))\approx0.622\). At D, \(\hat\mu_D=\exp(1.2)/(1+\exp(1.2))\approx0.769\). The coding changes the coefficient interpretation, not these reconstructed probabilities.

Why Equivalent Codings Preserve the Fit

Theorem: Suppose two full-rank codings for the same four-level factor each contain an intercept and three factor columns, and their columns span the same space of level-specific predictors. Then they represent the same set of possible level predictors. If all other model terms are unchanged, they give the same possible linear predictors for the observations and the same GLM likelihood values at corresponding parameter values.

For a proof, write the four-by-four level design matrices as \(M_1\) and \(M_2\), with one row per level and one column for the intercept or a factor column. Since each matrix has rank four and their column spaces are the same, each column of \(M_2\) is a linear combination of columns of \(M_1\). Thus \(M_2=M_1T\) for some matrix \(T\). The full ranks imply that \(T\) is invertible: if \(T v=0\), then \(M_2v=M_1Tv=0\), and the full column rank of \(M_2\) gives \(v=0\).

For any coefficient vector \(b_2\) in coding 2, set \(b_1=Tb_2\). Then \(M_2b_2=M_1Tb_2=M_1b_1\). Conversely, since \(T\) is invertible, every \(b_1\) corresponds to \(b_2=T^{-1}b_1\). Therefore both codings produce exactly the same set of four level predictors. Repeating each level’s row for its observations preserves equality of the observation-level predictors. Unchanged model terms can be added to both sides. Equal predictors give equal GLM means through the inverse link; with the same response distribution and other model specifications, they give equal likelihoods. \(\square\)

Reference, effect, and the custom coding above meet the theorem’s conditions: each has an intercept and three independent factor columns, and each can represent any four level predictors. In practice, a coefficient estimate may look different after recoding, but the corresponding fitted values do not change. This conclusion assumes the same model space and data; it does not imply that arbitrary coding changes preserve a fit.

Common Mistakes

Reading every coefficient as a level mean. Under reference coding, the intercept is the reference-level predictor. Under the effect coding shown here, it is the unweighted mean across levels. Under custom coding, its meaning depends on the column sums.
Forgetting the omitted level’s effect-coded row. D has \(-1\) in all three effect-coded columns. Its predictor is \(\alpha-\gamma_A-\gamma_B-\gamma_C\), not just \(\alpha\).
Calling a contrast coefficient the comparison without checking its scale. In the custom example, the A-versus-B difference is \(2\delta_1\), not \(\delta_1\), because the column entries are \(1\) and \(-1\).
Confusing an equal-level mean with an observation-weighted mean. Effect coding gives the mean of four level predictors when each level is counted once. If the level sample sizes differ, this need not equal the mean predictor across all observations.
Transforming a coefficient instead of the level predictor. In a logistic model, use the inverse link on the reconstructed \(\eta_l\). A contrast coefficient is generally not itself a logit or a probability.

Key takeaway: Write the design row for each factor level, substitute the coefficients to obtain every \(\eta_l\), and only then interpret the coefficients or apply the inverse link. Different full-rank codings can change coefficient meanings while preserving the same fitted predictors.

Check Your Understanding

For each question, identify the coding rule or calculation needed before interpreting a coefficient.

With A as the reference level, what does the coefficient for C represent on the linear-predictor scale?
Under the effect coding in this tutorial, how would you recover the predictor for D from \(\alpha,\gamma_A,\gamma_B,\gamma_C\)?
For the custom contrast columns shown, why is the difference \(\eta_A-\eta_B\) twice the first contrast coefficient?
If the four level predictors are \(2,4,1,5\), what is the effect-coded intercept and what are \(\gamma_A,\gamma_B,\gamma_C\)?
In a logistic model, what quantity should be passed through the inverse link to obtain the fitted probability at a specified factor level?