Why Recoding Leaves Predictions Unchanged

Why the Coefficients Can Change While the Fit Does Not

A reference category gives meaning to the coefficients in a regression model with a categorical predictor. If the reference group changes, the intercept and indicator coefficients change too. That change is a change of coordinates: the same group means can be described relative to a different starting group.

This distinction matters when reading or comparing regression output. A coefficient under reference coding is a difference from the chosen reference group, not the fitted mean for that category. Changing the reference group changes which differences are reported. It does not, by itself, change the model's fitted mean for any category.

The Model and Its Coefficients

Consider observations \(Y_i\) from individuals in one of \(K\) groups, denoted \(G_i\). For this tutorial, use a model with a categorical predictor only: each group has its own mean, and there are no other predictors. If group \(A\) is the reference, the model is

Here \(I(G_i=g)\) is 1 when observation \(i\) belongs to group \(g\), and 0 otherwise. The error \(\varepsilon_i\) is the difference between the observed outcome and its group mean in the model. The superscript \((A)\) labels the coding that uses \(A\) as the reference; it is not an exponent. The coefficient \(\beta_g^{(A)}\) exists for each nonreference group \(g\).

For an observation in group \(A\), all included indicators equal 0, so its fitted value is \(\beta_0^{(A)}\). For an observation in any other group \(g\), its fitted value is \(\beta_0^{(A)}+\beta_g^{(A)}\). It is useful to write \(m_g\) for the fitted mean of group \(g\). Thus \(m_A=\beta_0^{(A)}\) and \(m_g=\beta_0^{(A)}+\beta_g^{(A)}\) for \(g\ne A\).

To state the conversion compactly, define \(\beta_A^{(A)}=0\) by convention. This zero is not an estimated coefficient: it records that group \(A\) is the reference. Then every group mean under this coding can be written as \(m_g=\beta_0^{(A)}+\beta_g^{(A)}\).

Changing the Reference from A to B

Now change the reference group from \(A\) to a different group \(B\). The new intercept must equal the fitted mean for \(B\), because \(B\) is now the reference. Every new indicator coefficient must be the fitted mean for that indicator's group minus the fitted mean for \(B\). These requirements give the complete coefficient conversion.

The first line says that the new intercept is the old intercept plus the old coefficient for \(B\), which is the old fitted mean for \(B\). The second line expresses the old reference \(A\)'s mean relative to \(B\). The third line expresses each remaining group's mean relative to \(B\). In particular, these are differences between fitted means; they are not new group means on their own.

Why the Transformation Preserves Every Fitted Mean

The conversion follows by equating each group's fitted mean before and after recoding. For the new reference \(B\), the old model gives \(m_B=\beta_0^{(A)}+\beta_B^{(A)}\). Since the new intercept must equal \(m_B\), this gives the first line of the result. For the old reference \(A\), its mean is \(\beta_0^{(A)}\). Subtracting the new intercept gives its new coefficient:

For any group \(g\) other than \(A\) and \(B\), its old fitted mean is \(\beta_0^{(A)}+\beta_g^{(A)}\). Subtracting the new reference mean gives

These expressions also verify the fitted means directly. Under the new coding, group \(A\) has fitted mean \(\beta_0^{(B)}+\beta_A^{(B)}=m_B+(m_A-m_B)=m_A\). Group \(B\), now the reference, has fitted mean \(\beta_0^{(B)}=m_B\). Any other group \(g\) has fitted mean \(\beta_0^{(B)}+\beta_g^{(B)}=m_B+(m_g-m_B)=m_g\). Each category therefore retains exactly its previous fitted mean.

For ordinary least squares, this is more than an identity for a set of coefficients. Provided all \(K\) groups are represented in the data, either reference coding can describe any set of \(K\) group means: the intercept specifies the reference mean, and the \(K-1\) coefficients specify differences from it. The coding change is reversible, so the possible sets of fitted values are the same. Ordinary least squares minimizes the sum of squared residuals over those fitted values; minimizing over the same possibilities gives the same fitted values and residual sum of squares. No assumption of normally distributed errors is needed for this algebraic invariance. The usual regression assumptions matter for inference, but are not needed to show that recoding preserves predictions.

What Recoding Does and Does Not Change

The new coefficients answer different comparison questions, but the same model is being described. If the data, outcome, included groups, and fitting method remain fixed, changing only the reference group does not change the fitted value for any observation, its residual, or the residual sum of squares. The coefficient values and their direct interpretations do change because the comparisons are now made against a different group.

This invariance depends on changing only the coding of the same categorical predictor model. It does not say that predictions remain unchanged when a group is removed, a predictor is added, or a different model is fit. It also requires that the groups to be compared are represented in the data for their group means to be estimated from observed outcomes. A category with no observations does not have an empirically estimated group mean in this model.

Common Mistake

A common mistake is to see the coefficient for C change from \(-1.5\) to \(-5.5\) points in the three-group example and conclude that C's fitted score decreased by 4 points. That reasoning treats a difference-from-reference coefficient as though it were the group's fitted mean.

The correction is to reconstruct the mean using the intercept and the relevant coefficient. With A as reference, C's mean is \(12+(-1.5)=10.5\) points. With B as reference, it is \(16+(-5.5)=10.5\) points. The coefficient changed because its comparison group changed; C's fitted mean did not.