A Worked Sum-Coding Example With Unequal Group Sizes

Why Unequal Group Sizes Make This Example Useful

With sum coding, the intercept represents the unweighted average of the group means. It is tempting to mistake this for the overall sample mean, especially when each group contributes a different number of observations. These quantities answer different questions: the unweighted average gives each group equal weight, while the pooled sample mean gives each observation equal weight.

This tutorial works through the calculation for three groups. The central checks are to reconstruct all three fitted means from the coefficients and to calculate the pooled mean separately using the group sizes.

The Three-Group Sum-Coded Model

Call the groups A, B, and C, and use the following sum-coding convention: A is coded \((1,0)\), B is coded \((0,1)\), and C is coded \((-1,-1)\). Let \(x_{1j}\) and \(x_{2j}\) be the two coding values for group \(j\). A regression model with an intercept and these two coded predictors is

Here \(Y_{ij}\) is the outcome for observation \(i\) in group \(j\); \(\beta_0,\beta_1,\beta_2\) are model coefficients; and \(\varepsilon_{ij}\) is the error. The fitted value for a group is the model prediction for that group. Write these three fitted group means as \(\hat{\mu}_A,\hat{\mu}_B,\hat{\mu}_C\).

For the algebraic calculations below, each group must have at least one observation. In a model with only a categorical group predictor, least squares chooses the fitted value for each group to minimize the sum of squared differences between observations and their group’s fitted value. Thus the fitted value for each group is its sample mean. This equality does not require normally distributed errors. Interpreting group means as estimates of population means requires a suitable sampling or study design; conventional inference additionally relies on assumptions such as independent errors and, for standard pooled-variance methods, equal error variances.

Translate the Coefficients Into Fitted Means

Substitute each group’s code values into the model. The A code keeps \(\beta_1\), the B code keeps \(\beta_2\), and the C code subtracts both coefficients:

Adding the three fitted means cancels both deviation coefficients:

Therefore the intercept is the unweighted average of those means. Solving the same equations for the coefficients gives

The third group’s deviation is \(\hat{\mu}_C-\beta_0=-\beta_1-\beta_2\). This is the “sum” in sum coding: the three deviations from the intercept add to zero. It does not say that the group sizes are equal, or that the three sample means are equal.

Worked Example With Unequal Group Sizes

Now consider hypothetical observations from three groups. The values below are in outcome units. Group A has one observation, group B has two, and group C has five. The group sizes are deliberately unequal.

Why the Least-Squares Fits Are the Group Means

The equality between a fitted group value and its sample mean can be justified directly. For any one group with observations \(y_1,\ldots,y_m\), let \(c\) be the fitted value assigned to that group. Its contribution to the residual sum of squares is

To find the minimizing value, differentiate with respect to \(c\) and set the derivative to zero:

The second derivative is \(d^2S(c)/dc^2=2m>0\), so this value minimizes the sum of squares. In a three-group model with two coded predictors and an intercept, each group has its own fitted value, and the coefficient formulas above can represent any three group means. Consequently, minimizing the total sum of squares fits each group’s sample mean. Unequal group sizes change how many squared residuals each group contributes, but they do not change this group-by-group minimizing value.

Common Mistake: Calling the Intercept the Overall Sample Mean

A common incorrect argument is: “The intercept is the average outcome, so it must equal the mean of all observations.” That reasoning overlooks what is being averaged. With this sum-coding convention, the intercept averages the three group means with equal weight, regardless of group sizes. The pooled mean instead weights each group mean by its sample size.

Keep the distinction tied to the question. If the aim is to report a typical group while giving each category equal importance, the unweighted average of group means may be relevant. If the aim is to describe the average outcome among all observed individuals, use the pooled sample mean. A coding choice changes coefficient meanings and values, but it does not change the observed group means or fitted predictions when the model represents the same three group means.