Comparing Coding Schemes in a Three-Group Example

One Set of Group Means, Several Coefficient Stories

A categorical predictor with three groups can be represented in a regression equation in more than one way. The representation changes the coefficients and their interpretation, but it need not change the model’s fitted outcome for any group. Comparing the coding schemes on one dataset makes both facts concrete: coefficients depend on the coding, while fitted group means can remain identical.

We will use reference coding, sum coding, and cell-means coding for the same three groups. We will specify the contrast convention for each, calculate the coefficients, and check each group’s fitted mean. The arithmetic is descriptive: no hypothesis tests or confidence intervals are needed to compare the coding schemes.

Dataset and Model

Suppose, hypothetically, that an outcome is measured in points for individuals in groups A, B, and C. The observations are:

For a one-way group-means model, write the observation for individual \(i\) in group \(j\) as \(Y_{ij}=\mu_j+\epsilon_{ij}\), where \(\mu_j\) is that group’s mean outcome and \(\epsilon_{ij}\) is the individual’s deviation from the group mean. The usual linear-model assumptions are that the errors are independent, have mean zero, and have a common variance. Normality is additionally used for some exact small-sample inferences, but is not needed for the coefficient and fitted-mean calculations here.

The observed group means are \(10\), \(14\), and \(20\) points. We will use them to estimate the group means under all three coding schemes. The group sizes are unequal, which makes it especially useful to distinguish an unweighted average of group means from the overall sample mean.

Reference Coding: Differences from a Baseline Group

Take group A as the reference category. Define \(I_B=1\) for a group B observation and \(0\) otherwise; define \(I_C\) in the same way for group C. The reference-coded model is

For group A, both indicators equal zero, so its fitted mean is \(\beta_0\). For group B, \(I_B=1\) and \(I_C=0\), giving fitted mean \(\beta_0+\beta_B\). For group C, the fitted mean is \(\beta_0+\beta_C\). Therefore the coefficients are set by these three relationships:

Solving gives \(\beta_0=10\), \(\beta_B=14-10=4\), and \(\beta_C=20-10=10\), all in points. The intercept is the fitted mean for group A. The coefficient for B is the difference between B’s mean and A’s mean, and the coefficient for C is the difference between C’s mean and A’s mean.

Sum Coding: Deviations from an Unweighted Group Average

For sum coding, specify the contrast values rather than relying on the coding name alone. In this example, assign group A the pair \((1,0)\), group B the pair \((0,1)\), and group C the pair \((-1,-1)\). Let these two coded values be \(z_1\) and \(z_2\). The model is

Under this convention, the fitted means are \(\beta_0+\beta_1\) for A, \(\beta_0+\beta_2\) for B, and \(\beta_0-\beta_1-\beta_2\) for C. The three group means sum to \(3\beta_0\), because the contrast contributions cancel:

Consequently, \(\beta_0\) is the unweighted average of the three fitted group means. In this example it is \((10+14+20)/3=44/3\), or about \(14.67\) points. The coefficient \(\beta_1\) is A’s deviation from this average, and \(\beta_2\) is B’s deviation. The deviation for C is represented by \(-\beta_1-\beta_2\).

Cell-Means Coding: One Coefficient per Group

In cell-means coding, use one indicator for each of the three groups and omit the separate intercept. Let \(I_A\), \(I_B\), and \(I_C\) equal 1 for an observation in the named group and 0 otherwise. The model is

Exactly one indicator equals 1 for each observation. Therefore \(\gamma_A\), \(\gamma_B\), and \(\gamma_C\) are directly the fitted means for A, B, and C. For this dataset the estimates are \(10\), \(14\), and \(20\) points, respectively. Unlike reference coding, there is no single intercept that serves as a baseline; unlike the specified sum coding, the coefficients are not deviations from a common average.

Why All Three Schemes Give the Same Fitted Means

In this example, each coding scheme can represent any set of three group means. The model equations differ, but their coefficients can be translated so that the predictions agree group by group. With A as the reference, the means are \(m_A=\beta_0\), \(m_B=\beta_0+\beta_B\), and \(m_C=\beta_0+\beta_C\). Under the specified sum coding, they are \(m_A=\beta_0+\beta_1\), \(m_B=\beta_0+\beta_2\), and \(m_C=\beta_0-\beta_1-\beta_2\). In cell-means coding, they are \(m_A=\gamma_A\), \(m_B=\gamma_B\), and \(m_C=\gamma_C\).

A short least-squares argument explains why the estimated group means equal the observed group averages, regardless of which of these full three-mean representations is used. For any one group with observations \(y_1,\ldots,y_n\), the sum of squared residuals at a proposed fitted mean \(m\) is minimized at the sample mean \(\bar{y}\). To see this, write each residual as \((y_i-\bar{y})+(\bar{y}-m)\). The cross-product term sums to zero because \(\sum_i(y_i-\bar{y})=0\). Thus:

The first term does not depend on \(m\), and the second is nonnegative and is zero exactly when \(m=\bar{y}\). Since each of the three coding schemes can assign a separate fitted mean to each group, minimizing the total squared error gives the same three group averages in every scheme.

For a numerical check, the fitted means are A \(=10\), B \(=14\), and C \(=20\) points under all three schemes. They also give the same residuals: A has residuals \(-1,1\); B has \(-1,0,1\); and C has \(-2,0,2,0,0\). The residual sum of squares is \(2+2+8=12\) points squared in every scheme. Identical fitted values imply identical residuals for these same observations.

Interpreting the Intercept Carefully

The three intercept stories should not be confused:

Here the overall sample mean is \((20+42+100)/10=16.2\) points, not \(44/3\approx14.67\) points. The sample mean weights each group according to its size; the specified sum-coded intercept gives each group mean equal weight. Unequal group sizes make the distinction visible.

Common Mistake

• Wrong reasoning: “The sum-coded intercept is the overall sample mean, so it must be \(16.2\) points.” This uses the sample-size-weighted average, giving groups with more observations more weight.
• Correction: With the contrast values stated here, the sum-coded intercept is the unweighted average of the three fitted group means: \((10+14+20)/3=14.67\) points. Always check the actual coding values and model equation before interpreting an intercept.
• Another common error: Treating every coefficient as a group mean. That is true for the cell-means coefficients here, but not for the reference-coded differences or the sum-coded deviations.