Why Contrast Coding Matters
A categorical predictor with several levels can be represented by different sets of numerical columns in a regression model. Those columns determine what the coefficients mean: a coefficient may describe a difference from a reference category, a deviation from an overall level, or a contrast between combinations of categories. If the codings represent the same model, however, changing the coding changes the coefficient labels and values—not the fitted values for the observations.
This distinction matters when interpreting a regression output or checking a hand calculation. A coefficient cannot be interpreted without knowing the coding, and coefficients from two different codings cannot generally be compared one-for-one. The relevant comparison is whether both design matrices span the same set of possible fitted-value vectors.
Reference-Level and Sum-to-Zero Coding
In reference-level coding, one level is designated as the reference. Its indicator columns are all zero. With three levels A, B, and C and A as the reference, the model is \(y_i=\beta_0+\beta_B I(B_i)+\beta_C I(C_i)+\varepsilon_i\). Here \(I(B_i)\) is 1 for an observation in B and 0 otherwise, and \(I(C_i)\) is defined similarly. In a model containing only this factor and an intercept, \(\beta_0\) is the fitted mean for A, while \(\beta_B\) and \(\beta_C\) are the differences between the fitted means for B and A, and C and A.
In sum-to-zero coding, the level effects are constrained to add to zero. For three levels, use two columns with rows A \((1,0)\), B \((0,1)\), and C \((-1,-1)\). The model is \(y_i=\mu+\alpha_A c_{1i}+\alpha_B c_{2i}+\varepsilon_i\), with the implied effect \(\alpha_C=-\alpha_A-\alpha_B\). The intercept \(\mu\) is the unweighted average of the three fitted level means. The two displayed coefficients are the effects for A and B relative to that average; the effect for C follows from the constraint.
These statements about the intercept and level means apply to a model containing the factor and an intercept, with all levels represented and no other predictors. In models with additional predictors, the coefficients still describe the specified contrast conditional on those predictors, but the simple interpretation of the intercept as an average of observed group means need not apply.
Worked Example: Re-expressing Three Group Means
A hypothetical study records an outcome for six subjects: level A has values 5 and 7, level B has values 8 and 10, and level C has values 11 and 15. Fit a model with an intercept and this three-level factor. Calculate the coefficients under reference-level coding with A as reference and under sum-to-zero coding, then check whether the fitted values agree.
Step 1: Calculate the fitted level means. The level means are \(\bar y_A=(5+7)/2=6\), \(\bar y_B=(8+10)/2=9\), and \(\bar y_C=(11+15)/2=13\). The overall mean of all six observations is \((5+7+8+10+11+15)/6=56/6=28/3\).
Step 2: Find the reference-level coefficients. With A as the reference, the intercept is 6. The B coefficient is \(9-6=3\), and the C coefficient is \(13-6=7\). Thus the fitted means are \(6\) for A, \(6+3=9\) for B, and \(6+7=13\) for C.
Step 3: Find the sum-to-zero coefficients. The intercept is the average of the three level means: \(\mu=(6+9+13)/3=28/3\). The A effect is \(6-28/3=-10/3\), and the B effect is \(9-28/3=-1/3\). The implied C effect is \(-(-10/3)-(-1/3)=11/3\), so the three effects sum to zero.
Step 4: Check the fitted means. For A, the fitted value is \(28/3-10/3=18/3=6\). For B, it is \(28/3-1/3=27/3=9\). For C, it is \(28/3+11/3=39/3=13\). Both codings therefore give the same fitted value for each level.
Step 5: Check the residual fit. In either coding, the fitted values for the six observations are \(6,6,9,9,13,13\). The residuals are \(-1,1,-1,1,-2,2\), so the residual sum of squares is \(1+1+1+1+4+4=12\). The calculation confirms that changing the coding has not changed the fitted values or residual sum of squares.
Method and mistake avoided. Calculate the level means first, then express them in the chosen coding. Do not transfer reference-coded coefficients into the sum-to-zero design: for instance, using the reference coefficients \(6,3,7\) as if they were \(\mu,\alpha_A,\alpha_B\) would give a nonsensical C fit of \(6-3-7=-4\), not 13.
Why Equivalent Codings Give the Same Fit
Write the regression model in matrix form as \(y=X\beta+\varepsilon\). Here \(X\) is the design matrix, \(\beta\) is the coefficient vector, and the ordinary least squares (OLS) fitted vector is the vector of values minimizing the residual sum of squares \((y-X\beta)^\mathsf{T}(y-X\beta)\). Suppose another coding uses \(X^*=XA\), where \(A\) is an invertible matrix. The columns are changed by an invertible transformation, so \(X\) and \(X^*\) span the same column space. If \(\beta^*=A^{-1}\beta\), then \(X^*\beta^*=XA A^{-1}\beta=X\beta\).
The same argument applies to OLS estimation: every fitted vector available using \(X\) is available using \(X^*\), and vice versa. Consequently, both models minimize the residual sum of squares over the same set of fitted vectors. The fitted vector is the unique projection of \(y\) onto that column space, so the fitted values are identical. If the design matrix has full column rank, the coefficient vectors are also uniquely related by the stated transformation.
Write down the row of contrast values assigned to each factor level, as well as the intercept column.
Substitute each level's contrast values into the model. This shows which combinations of coefficients produce that level's fitted mean.
If one design matrix is an invertible re-expression of the other, the models allow the same fitted vectors. If not, identical fits are not guaranteed.
Orthogonal Contrasts as Another Coding
A contrast coding can also express targeted comparisons among levels. For three levels, consider two contrast columns whose values are A \((1,1)\), B \((-1,1)\), and C \((0,-2)\). The first contrast compares A with B; the second compares the average of A and B with C. Each column sums to zero across the three levels, so the intercept in this factor-only model is the unweighted average of the level means. The columns are orthogonal across the three levels because \(1(1)+(-1)(1)+0(-2)=0\). Orthogonality describes the contrast columns; it does not mean the contrast coefficients are reference-level differences.
Worked Example: Interpreting Orthogonal Contrast Coefficients
A hypothetical analysis has fitted response means of 12, 15, and 21 for levels A, B, and C. Use the orthogonal contrast rows A \((1,1)\), B \((-1,1)\), and C \((0,-2)\). Find the intercept and both coefficients, then reconstruct each fitted mean.
Step 1: Find the intercept. Since both contrast columns sum to zero across levels, the intercept is \(\mu=(12+15+21)/3=48/3=16\).
Step 2: Solve for the first coefficient. Write the fitted means as \(12=\mu+\gamma_1+\gamma_2\), \(15=\mu-\gamma_1+\gamma_2\), and \(21=\mu-2\gamma_2\). Subtracting the B equation from the A equation gives \(12-15=2\gamma_1\), so \(\gamma_1=-3/2=-1.5\).
Step 3: Solve for the second coefficient. Adding the A and B equations and subtracting twice the C equation gives \(12+15-2(21)=6\gamma_2\). Thus \(-15=6\gamma_2\), and \(\gamma_2=-15/6=-2.5\).
Step 4: Verify every level. A has fitted mean \(16-1.5-2.5=12\). B has fitted mean \(16+1.5-2.5=15\). C has fitted mean \(16+5=21\). Each value matches the stated mean. The coefficients are the specified contrasts, not the differences from A.
Method and mistake avoided. Use the actual contrast values for each level when reconstructing fitted means. Do not interpret \(\gamma_1=-1.5\) as the B-minus-A difference; that difference is \(15-12=3\).
Unequal Group Sizes and the Intercept Trap
With sum-to-zero coding, the intercept is the unweighted average of the fitted level means in the factor-only model. It is not necessarily the overall sample mean. The overall sample mean weights each level according to its number of observations. When group sizes differ, these two averages can differ even though the fitted values for all observations remain unchanged.
Worked Example: Unequal Replication Under Two Codings
For a hypothetical outcome, level A has observations 5 and 7, level B has one observation, 9, and level C has observations 10, 12, and 17. Fit a factor-only model using A as the reference, then express the same fitted level means using sum-to-zero coding. Check the residual sum of squares and distinguish the two averages.
Step 1: Calculate fitted level means. A has mean \((5+7)/2=6\), B has mean \(9\), and C has mean \((10+12+17)/3=39/3=13\). The sample mean across all six observations is \((5+7+9+10+12+17)/6=60/6=10\).
Step 2: Calculate reference-level coefficients. With A as reference, the intercept is 6, the B coefficient is \(9-6=3\), and the C coefficient is \(13-6=7\). The fitted level means are therefore 6, 9, and 13.
Step 3: Calculate sum-to-zero coefficients. The intercept is the unweighted average of the three level means: \(\mu=(6+9+13)/3=28/3\). The A and B effects are \(6-28/3=-10/3\) and \(9-28/3=-1/3\). The implied C effect is \(11/3\). Reconstructing the means gives \(28/3-10/3=6\), \(28/3-1/3=9\), and \(28/3+11/3=13\).
Step 4: Check residuals and the average distinction. The residuals are \(-1,1,0,-3,-1,4\), and the residual sum of squares is \(1+1+0+9+1+16=28\). The fitted values and residual sum of squares are the same under either coding. The sum-coded intercept \(28/3\) is approximately 9.333, whereas the sample mean is 10 because the sample has more C observations than A or B observations.
Method and mistake avoided. Compute the contrast-coded intercept from the model's level-effect constraint, not by automatically taking the overall sample mean. Here that shortcut would give 10 instead of the correct sum-to-zero intercept \(28/3\).
When Invariance Does Not Apply
The invariance result requires the codings to represent the same model—that is, the design matrices must have the same column space. With an intercept and a three-level factor, two independent contrast columns allow the model to represent any three fitted level means. A different choice of two independent contrast columns preserves that flexibility. But if a contrast is omitted, or a restriction is imposed that prevents some level-mean combinations, the model space may change and so may the fitted values.
Using an intercept together with all \(K\) level indicators is another case to handle carefully: those columns are linearly dependent, so the coefficients are not uniquely determined. This does not by itself mean the fitted values are nonunique, but it prevents a one-to-one interpretation of the coefficients without an additional constraint or a reduced coding.
Check Your Understanding
For each problem, identify the coding or model restriction before calculating.
- Three level means are 20 for A, 26 for B, and 17 for C. With A as reference, find the intercept and two coefficients. Then find the sum-to-zero intercept and the A and B effects. (Answer: Reference coding: intercept 20, B coefficient 6, C coefficient \(-3\); sum-to-zero coding: intercept 21, A effect \(-1\), B effect 5, implied C effect \(-4\).)
- Fitted means are 4, 10, and 16 for levels A, B, and C. Using contrast rows A \((1,1)\), B \((-1,1)\), and C \((0,-2)\), find the intercept and both coefficients and verify the C mean. (Answer: Intercept 10, first coefficient \(-3\), second coefficient \(-3\); the C fitted mean is \(10-2(-3)=16\).)
- A has observations 2 and 6, B has observations 8, 10, and 12, and C has observation 16. Find the reference-coded coefficients with A as reference, the sum-to-zero intercept, and the residual sum of squares. (Answer: Level means 4, 10, and 16; reference coefficients \(4,6,12\); sum-to-zero intercept 10; residual sum of squares 16.)
- Explain whether an invertible change from one set of contrast columns to another can alter the fitted values in an OLS model. State the condition that matters. (Answer: No, if the transformation is invertible and the design matrices span the same column space; the fitted vector is the same projection of the response onto that space.)
- A model with an intercept and two contrast columns omits one independent contrast for a three-level factor. Is fitted-value invariance with a full three-level factor model guaranteed? (Answer: No. The reduced model may have a smaller column space, so it need not represent all three level means or produce the same fitted values.)