Sum Coding for Categorical Predictors

Why Use Sum Coding?

A categorical predictor with three levels can be represented by two numerical columns in a regression model. Reference-cell indicator coding makes one category the comparison baseline: the intercept gives that category’s fitted mean, and the other coefficients give differences from it. Sum coding uses a different parameterization: its code values balance to zero across the three categories, so the intercept and coefficients relate to the category means in a different way.

The point is not to change the fitted values. For a given set of three category means, reference-cell coding and sum coding can represent the same means. What changes is how those means are expressed using the model coefficients.

Constructing Two Sum-to-Zero Codes

Call the categories A, B, and C. Assign two codes to each category using the following convention. The code in each column sums to zero over the three categories.

For \(x_1\), the three category codes add to \(1+0+(-1)=0\). For \(x_2\), they add to \(0+1+(-1)=0\). These are codes for the categories, not values that vary with the outcome.

To state the model, let \(Y_i\) be the outcome for observation \(i\), and let \(x_{1i}\) and \(x_{2i}\) be the codes for that observation’s category. The one-factor regression model is

Here \(\beta_0,\beta_1,\beta_2\) are the regression coefficients and \(\varepsilon_i\) is the error for observation \(i\). For the usual least-squares model, assume errors have mean zero and are independent with a common variance. The category coding determines the fitted means; those assumptions concern the errors and are relevant to statistical inference about the model.

From the Codes to Category Means

The fitted value for a category is obtained by substituting that category’s two codes into the model. Denote the fitted means for A, B, and C by \(\hat{Y}_A,\hat{Y}_B,\hat{Y}_C\). The model gives

These equations show how two coefficients and an intercept encode three category means. Adding the three fitted means gives

The two code-coefficient terms cancel. Dividing by three proves that the intercept is the unweighted average of the three fitted category means:

Solving the category equations for the coefficients gives another useful description:

So the first two coefficients are the deviations of A and B from the unweighted average of the three fitted means. The deviation for C is determined by the other two: the three deviations sum to zero. These are algebraic descriptions of the coefficients under this coding. The next tutorial develops their interpretation further.

Contrast With Reference-Cell Indicator Coefficients

With reference-cell indicator coding and A chosen as the reference, the model uses an indicator for B and an indicator for C. Each indicator equals 1 for its named category and 0 otherwise. The corresponding model can be written

In this model, the fitted means are \(\hat{Y}_A=\alpha\), \(\hat{Y}_B=\alpha+\delta_B\), and \(\hat{Y}_C=\alpha+\delta_C\). Thus the intercept is A’s fitted mean, while \(\delta_B\) and \(\delta_C\) are differences from A. Under sum coding, the fitted means instead follow the three equations above: the intercept is their unweighted average, and the two coefficients are deviations for A and B from that average.

The coefficient values are connected by the category means. In terms of sum-coded coefficients, \(\alpha=\beta_0+\beta_1\), \(\delta_B=\beta_2-\beta_1\), and \(\delta_C=-2\beta_1-\beta_2\). These follow by expressing each reference-coded quantity using \(\hat{Y}_A=\beta_0+\beta_1\), \(\hat{Y}_B=\beta_0+\beta_2\), and \(\hat{Y}_C=\beta_0-\beta_1-\beta_2\).

Why the Choice of Coding Matters

Coding determines the meanings attached to individual coefficients, so a coefficient should not be interpreted without checking the code assignments and the model equation. The fitted values are the predictions for each category; changing from sum coding to reference coding changes how those predictions are represented, not the predictions themselves.

For an ordinary least-squares model with a single categorical predictor, the fitted mean for each category is its observed sample mean. Sum coding expresses these three fitted means through an intercept and two coefficients. It does not make the category sample sizes equal, and the unweighted average in the intercept formula is an average over the three category means, not an average over all observations.

Common Mistake

A common mistake is to treat a sum-coded coefficient as the difference between the category named by its code and the category with the omitted reference indicator. For example, \(\beta_1\) is not the A-minus-C difference under the coding used here. The equations show that A-minus-C is \((\beta_0+\beta_1)-(\beta_0-\beta_1-\beta_2)=2\beta_1+\beta_2\). The correction is to substitute the codes for both categories into the model, obtain both fitted means, and then subtract in the requested order.