Interpreting Coefficients Under Sum Coding

Introduction

When a categorical predictor has several levels, the coding scheme determines how the regression coefficients describe the category means. With reference coding, a coefficient describes a difference from the reference category. With sum coding, the comparison point is different: the intercept is the unweighted average of the fitted category means, and the category effects describe deviations from that average.

This interpretation is useful when no single category should serve as the natural baseline. It also makes the coefficients’ constraint visible: across all categories, the deviations from the average must sum to zero. The calculations below use a factor-only model, so the fitted value depends on the category and no other predictors.

The Sum-Coded Factor-Only Model

Suppose a factor has \(K\) categories, labelled \(1,\ldots,K\), and the quantitative response is measured in some specified units. Use \(K-1\) coded predictors. For categories \(1\) through \(K-1\), the code for a category is \(+1\) in its own column and \(0\) in the other columns. Category \(K\) receives \(-1\) in every column. This is one standard sum-coding convention.

The factor-only regression model is

Here \(Y_i\) is the response for observation \(i\), \(x_{ij}\) is the code in column \(j\) for that observation’s category, and \(\epsilon_i\) is its error. For the usual linear-model interpretation, assume errors have mean zero conditional on category, are independent across observations, and have a common variance. Normality is not needed for the coefficient meanings or fitted-mean calculations below; it is relevant to some forms of exact small-sample inference.

“Unweighted” means that each category contributes once to the average, regardless of how many observations are in it. It does not mean an average weighted by category sample sizes. The fitted category means are model predictions for the categories, not necessarily the same thing as the overall average of all observed responses.

How the Coefficients Produce Category Means

For category \(j\), where \(j<K\), its own coded column is \(+1\) and its other columns are \(0\). Substituting those codes into the model gives fitted mean \(\beta_0+\beta_j\). For category \(K\), all \(K-1\) codes are \(-1\), so its fitted mean is \(\beta_0-\beta_1-\cdots-\beta_{K-1}\).

A hat here indicates a fitted category mean. Averaging these \(K\) fitted means gives

The positive and negative coefficient terms cancel, leaving the intercept. Therefore the fitted mean for any category minus \(\beta_0\) is that category’s effect, and the effects sum to zero. This is an algebraic property of the coding, not a claim that every category has the same fitted mean.

The Last Category’s Effect

Only \(K-1\) coefficients are estimated for the factor, so the last category does not receive a separate coefficient in the equation. Its effect is still determined: the effects across all categories must sum to zero. If the first \(K-1\) effects add to a positive number, the last effect is the negative of that sum; if they add to a negative number, the last effect is positive.

Category Effects Are Not Pairwise Differences

A category effect compares that category with the unweighted average across all categories. A pairwise difference compares two particular categories. If categories \(r\) and \(s\) have effects \(\alpha_r\) and \(\alpha_s\), their fitted-mean difference is \(\alpha_r-\alpha_s\), because the shared intercept cancels. It is generally not equal to either effect alone.

Why This Interpretation Matters

Sum coding changes how the coefficients are described, not the fitted category means represented by a correctly specified factor-only model. The intercept provides a central summary of the fitted means, and each effect shows how far one category lies above or below that center. This can be a useful description when the categories have no natural baseline.

The meaning of the average deserves care. It is the unweighted average across the \(K\) category means: each distinct category receives equal weight in that calculation. If category sizes differ, this need not equal the average response across all individual observations. Nor does a coefficient’s sign establish that a category differs reliably from the average; deciding that requires an inferential procedure and its assumptions.

Common Mistakes

• Treating the intercept as a reference-category mean. Under the sum coding defined here, the intercept is the unweighted average of the fitted category means. There is no reference category in this interpretation.
• Reading a coefficient as a pairwise difference. A coefficient is a deviation from the average. To compare two categories, subtract their effects; the intercept cancels.
• Forgetting the last category. It has no separately listed coefficient, but its effect is the negative sum of the other effects. It is not automatically zero.
• Using the sample-size-weighted average instead. The intercept’s interpretation follows from the coding constraint and averages the category means equally, rather than weighting categories by their numbers of observations.
• Confusing a model interpretation with a causal claim. A category coefficient describes fitted associations in the model. By itself it does not show that belonging to a category caused a response difference.