Changing Reference Levels Without Changing Fits

Why Changing the Reference Level Is a Reparameterization

A factor’s reference level determines how its coefficients are expressed: the intercept corresponds to the reference level, and each other level’s coefficient compares that level with the reference. Choosing a different reference changes those descriptions and the numerical coefficients. It need not change the model’s predictions. The essential distinction is between changing the coordinates used to describe a model and changing the model itself.

We derive the transformation for a single factor in a generalized linear model (GLM) with a common set of additional covariates. The factor enters through main effects only: this tutorial does not include factor interactions. The response distribution and link function remain fixed throughout.

In this parameterization, \(\beta_0\) is the predictor at level 1 when the covariates are zero. Each \(\alpha_j\) is the difference between the linear predictor at level \(j\) and the predictor at level 1, holding the covariates fixed. On a nonlinear link scale, this is a difference in linear predictors; it is not generally a difference in response means.

The Coefficient Transformation

Now choose level \(r\) as the reference, with \(r\ne 1\). The new intercept must give the same predictor as before at level \(r\), so it is the old intercept plus the old contrast for level \(r\). Each new factor coefficient must compare its level with \(r\). Because both the old and new parameters describe linear predictors, that comparison is a subtraction.

Proof. Consider an observation in factor level \(j\). Under the original parameterization, its factor contribution together with the intercept is \(\beta_0+\alpha_j\), using \(\alpha_1=0\). Under the new parameterization it is \(\beta_0^{*}+\alpha_j^{*}\). Substitution gives \(\beta_0^{*}+\alpha_j^{*}=(\beta_0+\alpha_r)+(\alpha_j-\alpha_r)=\beta_0+\alpha_j\). This calculation also applies when \(j=r\), because then \(\alpha_r^{*}=0\), and when \(j=1\), because \(\alpha_1=0\). Since \(\gamma^{*}=\gamma\), the covariate contribution is unchanged as well. Thus the linear predictor agrees for every factor level and every covariate vector. Applying the same inverse link \(g^{-1}\) gives the same fitted mean \(\mu_i\). \(\square\)

This transformation is reversible. In the new parameterization, \(\beta_0=\beta_0^{*}+\alpha_1^{*}\), \(\alpha_r=-\alpha_1^{*}\), and \(\alpha_j=\alpha_j^{*}-\alpha_1^{*}\) for \(j\ne 1,r\). The covariate coefficients remain \(\gamma=\gamma^{*}\). The two parameterizations therefore describe the same set of linear predictors, just with different coefficient coordinates.

What Happens to Estimates and Uncertainty?

The coefficient transformation applies to estimates as well as to parameter values. If an estimate under the old reference is \(\hat\beta_0,\hat\alpha_1,\ldots,\hat\alpha_K,\hat\gamma\), then the corresponding estimate under the new reference is obtained by applying the same formulas. This is a linear transformation of the coefficient vector, so its covariance matrix must also be transformed. In general, coefficient standard errors do not simply stay attached to the same coefficient label.

The identity follows directly from the definition of covariance. If \(m=E[X]\), then \(E[AX]=Am\), and \(\operatorname{Var}(AX)=E[(AX-Am)(AX-Am)^\mathsf{T}]=E[A(X-m)(X-m)^\mathsf{T}A^\mathsf{T}]=A\operatorname{Var}(X)A^\mathsf{T}\). The final equality uses that \(A\) is fixed, so it can be taken outside the expectation. \(\square\)

Fitting Invariance and Its Limits

Because the transformation is one-to-one, the two parameterizations assign the same linear predictor to every possible combination of factor level and covariate values. For a fixed GLM family, link, and any other parameters, they therefore give the same fitted means and the same likelihood for corresponding parameter values. If a maximum-likelihood estimate exists, a maximizer in one parameterization maps to a maximizer in the other: every candidate in either parameterization has a corresponding candidate with the same likelihood.

This statement concerns corresponding fits. If the design is rank deficient, the coefficients may not be uniquely determined; changing the reference does not repair that identifiability problem. The parameterization still represents the same possible linear predictors, but one should not infer unique coefficients merely from the fact that the reference-level transformation is reversible.

Common Mistake

A common error is to change the reference label while keeping all the old coefficient values. In the logistic example, retaining an intercept of \(-1\) and a coefficient of \(0.8\) after declaring \(B\) the reference would give predictor \(-1\) for \(B\) at \(x=0\). The original predictor for \(B\) at \(x=0\) was \(-1+0.8=-0.2\). The fitted probability would therefore change from \(1/(1+\exp(0.2))\approx0.4502\) to \(1/(1+\exp(1))\approx0.2689\). That is not a reference-level change alone; it is a different fitted model. The correction is to transform both the intercept and the factor coefficients, then verify the predictor level by level.