Why Sum Coding Changes the Intercept
A categorical predictor with several levels can be represented by contrast columns in a regression model. Under sum coding, the coded effects across all levels add to zero. This constraint makes each level’s effect a deviation from the unweighted average of the factor’s level-specific fitted values, on the model’s linear-predictor scale.
The intercept therefore has a specific interpretation: with a factor-only model, it is the unweighted average of the fitted values for the factor levels. It is not generally the average across all observations. If the categories have different sample sizes, those two averages can differ. Adding numeric predictors or changing their zero point also affects what the intercept represents.
For a three-level factor \(A,B,C\), one common sum-coded design assigns the rows \((1,0)\), \((0,1)\), and \((-1,-1)\), respectively. The model is
Here \(\eta_k\) is the fitted linear predictor for level \(k\). For an ordinary linear regression with the identity link, it is the fitted response. For a model using another link, it is on the link scale, not necessarily the response scale. The coefficients satisfy \(\alpha_A=\beta_A\), \(\alpha_B=\beta_B\), and \(\alpha_C=-\beta_A-\beta_B\), so their sum is zero.
Building and Reading the Contrasts
The coefficient values are easiest to understand by first writing the fitted value for every factor level. The intercept is the average of those fitted values, and each level effect is that level’s fitted value minus the intercept. Only \(K-1\) effects are freely estimated: the final effect must make the total equal zero.
These equations describe the parameterization, not a claim that the intercept is always the sample-wide mean. In a factor-only ordinary least squares model, with at least one observation in each level, the fitted value for a level is its sample mean. The intercept is consequently the unweighted mean of the \(K\) sample means. The sample-wide mean, by contrast, weights each category mean by its number of observations.
Worked Example: Three Category Means With Sum Coding
A hypothetical portfolio has three risk bands, \(A\), \(B\), and \(C\). In a factor-only ordinary least squares model for an annual outcome, the fitted category means are 9 for \(A\), 13 for \(B\), and 6 for \(C\). Construct the sum-coded coefficients and check each fitted value.
Method. With three levels, take \(A\) and \(B\) as the two explicitly coded levels, and represent \(C\) by \((-1,-1)\). The intercept is the unweighted average of the three fitted category values:
The level effects are each category mean minus this intercept:
The implied effect for \(C\) is the negative of the first two effects:
Check. The three effects sum to \(-1/3+11/3-10/3=0\). The fitted values are \(9.333\ldots-0.333\ldots=9\) for \(A\), \(9.333\ldots+3.666\ldots=13\) for \(B\), and \(9.333\ldots-3.333\ldots=6\) for \(C\). Rounded to three decimals, the intercept is 9.333 and the effects are \(-0.333\), 3.667, and \(-3.333\). A common mistake is to treat the last effect as zero because it has no separate contrast column; its effect is instead determined by the negative sum of the other effects.
Why the Intercept Is an Unweighted Average
The average interpretation follows directly from the zero-sum constraint. Since each fitted level value is the intercept plus its level effect, averaging the \(K\) values gives the intercept plus the average effect. The average effect is zero, so what remains is the intercept.
For a factor-only ordinary least squares model, the fitted value for category \(k\) is its observed sample mean \(\bar y_k\). To see why, let \(n_k\) be the number of observations in category \(k\), and let \(c_k\) be the single fitted value assigned to that category. The sum of squared errors within category \(k\) is minimized at \(c_k=\bar y_k\): writing \(y_{ik}-c_k=(y_{ik}-\bar y_k)+(\bar y_k-c_k)\), the cross-product terms sum to zero, leaving a sum of squares plus \(n_k(\bar y_k-c_k)^2\). That last term is minimized at \(c_k=\bar y_k\). Thus the sum-coded intercept is the unweighted average of the category sample means.
The conclusion depends on the design. If category \(k\) has \(n_k\) observations, the sample-wide mean is \(\sum_k n_k\bar y_k/\sum_k n_k\), which weights larger categories more heavily. In a factor-only model, sum coding does not change the fitted category means or the model’s predictions; it changes how those fitted values are expressed as an intercept and constrained effects.
Worked Example: Unequal Category Frequencies
A hypothetical service team records an outcome for three customer groups. Group \(A\) has values 2 and 4; group \(B\) has values 8, 10, 12, and 14; and group \(C\) has one value, 5. Fit the factor-only ordinary least squares model conceptually, then find the sum-coded intercept and effects. Compare the intercept with the overall sample mean.
Step 1: Find the category means. The group means are
Step 2: Find the sum-coded intercept. Average the three category means, giving each category equal weight:
Step 3: Find the effects. Subtract the intercept from each category mean:
Check the constraint. The effects sum to \((-10+14-4)/3=0\), and adding each effect to \(19/3\) gives 3, 11, and 5. There are seven observations, with total outcome \(6+44+5=55\), so the overall sample mean is \(55/7=7.857\) to three decimals. It is not the intercept. The common mistake is to use \(55/7\) as the sum-coded intercept; that is the observation-weighted mean, while the intercept here averages the three category means equally. The rounded sum-coded intercept is 6.333.
What Changes When the Model Has Numeric Predictors?
When a numeric predictor is included, factor deviations are interpreted at a particular value of that predictor. For a model with one numeric predictor \(x\), a common slope, and a sum-coded factor, write
At \(x=0\), the category-specific fitted linear predictors are \(\beta_0+\alpha_k\), whose unweighted average is \(\beta_0\). At another value \(x=c\), the category values are shifted by the common amount \(\beta_xc\), so their unweighted average is \(\beta_0+\beta_xc\). Thus the intercept is tied to the numeric predictor’s zero point. If zero is outside the useful range, the intercept may be algebraically valid but not substantively informative. Centering \(x\) changes the point at which the intercept is interpreted; it does not make the factor effects into observation-weighted deviations.
The same care is needed if the factor appears in interactions or if the model is not a factor-only model. The zero-sum constraint still describes the parameterization, but the fitted values can depend on other predictors and their combinations. Do not automatically call the intercept the overall response mean. State the values of other predictors at which the factor-level comparison is being made, and identify whether the quantities are on the response scale or the linear-predictor scale.
Worked Example: Sum-Coded Deviations Alongside a Numeric Predictor
A hypothetical pricing study models an outcome using a numeric exposure score \(x\) and a three-level region factor. The observed pairs follow these fitted lines: region \(A\) has values \(10+2x\), region \(B\) has values \(14+2x\), and region \(C\) has values \(6+2x\). The data include \(x=0,1\) in \(A\); \(x=0,1,2\) in \(B\); and \(x=0,1\) in \(C\). Express the model using sum coding, check its fitted values, and explain the effect of centering \(x\) at 1.
Step 1: Identify the common slope and category intercepts. Each line increases by 2 for a one-unit increase in \(x\), so \(\beta_x=2\). At \(x=0\), the category fitted values are 10, 14, and 6.
Step 2: Calculate the sum-coded intercept. This is the unweighted average of the three category fitted values at \(x=0\):
Step 3: Calculate and check the effects. Subtract 10 from each category value. This gives \(\alpha_A=0\), \(\alpha_B=4\), and \(\alpha_C=-4\), which sum to zero. The model is \(\eta=10+2x+\alpha_k\). At \(x=0\), its fitted values are 10, 14, and 6. At \(x=1\), they are 12, 16, and 8. At \(x=2\), the region \(B\) fitted value is \(10+2(2)+4=18\). These agree with the stated lines and observed ranges.
Step 4: Re-express the model using \(x'=x-1\). Since \(x=x'+1\), the common part becomes \(10+2(x'+1)=12+2x'\). The intercept is now 12, while the factor effects remain \(0,4,-4\). At \(x'=0\), which means \(x=1\), the three fitted values are 12, 16, and 8; their unweighted average is 12. The common mistake is to interpret 10 as an average independent of the numeric predictor. It is the average category fitted value specifically at \(x=0\). After centering at 1, the corresponding intercept is 12.
Interpretation and Exam Traps
Sum coding is useful when comparisons with the factor average are more meaningful than comparisons with one designated category. A positive coefficient says that the fitted value for that level is above the unweighted factor-level average, at the relevant settings of the other predictors. A negative coefficient says it is below. The size and units of the coefficient are those of the linear predictor; for an identity-link model, they are the response units.
Keep three different averages distinct: the unweighted average of category fitted values, the observation-weighted average of the data, and the average fitted value over a chosen set of observations. They coincide in some designs, such as equally sized groups in a factor-only model, but they are not interchangeable in general.
| Quantity | How it is formed | Typical interpretation |
|---|---|---|
| Sum-coded intercept in a factor-only model | Unweighted average of the \(K\) fitted category values | Average across factor levels |
| Overall sample mean | Category means weighted by category sample sizes | Average across observed records |
| Intercept with a numeric predictor | Average of category fitted values when numeric predictors equal zero | Factor-level average at that specified predictor setting |
- Do not treat the missing contrast column as a zero effect. The level represented by the all-negative row has an effect equal to minus the sum of the other effects.
- Do not confuse equal weighting of levels with equal weighting of observations. Unequal category sizes usually make these averages different.
- Do not interpret the intercept without its predictor settings. With numeric predictors, it applies at their zero values, unless the model or a re-expression establishes another reference point.
- Do not assume the coefficients are response-scale differences in every model. In a model with a non-identity link, the sum constraint and deviations are on the linear-predictor scale.
Check Your Understanding
For each problem, construct or interpret the sum-coded quantities and check whether the stated average is weighted by factor levels or by observations.
- A factor-only ordinary least squares model has fitted category means 7, 10, and 13 for levels \(A,B,C\). Find the intercept and all three sum-coded effects. (Answer: intercept 10; effects \(A=-3\), \(B=0\), \(C=3\).)
- Three groups have sample sizes 1, 3, and 2, and their sample means are 2, 8, and 14. Find the sum-coded intercept and effects, then find the overall sample mean. (Answer: intercept 8; effects \(-6,0,6\); overall sample mean 9.)
- A four-level factor has fitted category values 4, 9, 12, and 15 in a factor-only model. Find the sum-coded intercept and the four effects, and verify the constraint. (Answer: intercept 10; effects \(-6,-1,2,5\), which sum to 0.)
- A model has category lines \(20+3x\), \(26+3x\), and \(14+3x\) for levels \(A,B,C\). Find the sum-coded intercept and effects at \(x=0\), then give the three fitted values at \(x=2\). (Answer: intercept 20; effects \(0,6,-6\); fitted values at \(x=2\) are 26, 32, and 20.)
- A factor-only model has four category means 5, 11, 17, and 7, with category counts 1, 1, 1, and 7. Find the sum-coded intercept and effects, and compare the intercept with the overall sample mean. (Answer: intercept 10; effects \(-5,1,7,-3\); overall sample mean 8.2.)