Reference-Category Dependence and Probability Invariance

Why the Reference Category Matters

A nominal response has categories with no required ordering, so a baseline-category model describes each nonreference category relative to one chosen reference. The reference is a coordinate choice: changing it changes the coefficient comparisons, even though it need not change the model’s fitted probabilities. Keeping those two facts separate is essential when interpreting or comparing results.

This tutorial considers a response $Y$ with $K$ categories and a predictor vector $x$. It focuses on the reparameterization of a baseline-category multinomial logit model. It does not change the response or the predictor values; it changes which category’s probability appears in the denominator of each logit.

Definition: Let $p_j(x)=P(Y=j\mid x)$, for categories $j=1,\ldots,K$, and choose category $b$ as the reference. The baseline-category logit model specifies $$ \log\!\left(\frac{p_j(x)}{p_b(x)}\right) =\eta_j^{(b)}(x) =\alpha_j^{(b)}+x^\mathsf{T}\beta_j^{(b)}, \qquad j\ne b. $$ Here $x$ is a $q$-dimensional predictor vector, $\alpha_j^{(b)}$ is an intercept, and $\beta_j^{(b)}$ is a $q$-dimensional slope vector. For convenient formulas, set $\eta_b^{(b)}(x)=0$, $\alpha_b^{(b)}=0$, and $\beta_b^{(b)}=0$.

The superscript $(b)$ records the reference category. It is not an exponent. In this parameterization, a slope component describes how a predictor changes the log odds of category $j$ relative to category $b$, holding the other predictors fixed. The reference category itself has no estimated logit in this representation; its zero linear predictor is a normalization.

The Exact Coefficient Transformation

Suppose the model is initially expressed relative to category $b$, and we want category $r$, with $r\ne b$, to be the reference instead. The new logit for category $j$ compares $j$ with $r$. For $j\ne r$, subtract the old logit for $r$ from the old logit for $j$:

\log\!\left(\frac{p_j(x)}{p_r(x)}\right) = \log\!\left(\frac{p_j(x)}{p_b(x)}\right) - \log\!\left(\frac{p_r(x)}{p_b(x)}\right) = \eta_j^{(b)}(x)-\eta_r^{(b)}(x).

Because each old linear predictor is linear in $x$, their difference is also linear in $x$. Define the new coefficient pair by subtracting the old coefficient pair for $r$ from that for $j$, treating the old reference pair as zero. In particular, the old reference $b$ gets a new logit too: its new predictor is $0-\eta_r^{(b)}(x)$.

Theorem: In an unrestricted baseline-category model, changing the reference from $b$ to $r$ gives, for every $j\ne r$, $$ \alpha_j^{(r)}=\alpha_j^{(b)}-\alpha_r^{(b)}, \qquad \beta_j^{(r)}=\beta_j^{(b)}-\beta_r^{(b)}. $$ These formulas use $\alpha_b^{(b)}=0$ and $\beta_b^{(b)}=0$. The transformed model represents exactly the same category probabilities for every $x$.

The first part of the theorem follows by matching the intercept and slope terms in the difference of linear predictors. The claim about probabilities is more than a statement that the logits can be manipulated: it says that using the transformed coefficients to recover the category probabilities gives the same result. Here is a direct proof.

Write the old probabilities.
With $\eta_b^{(b)}(x)=0$, the old baseline-category probabilities are $p_j(x)=\exp(\eta_j^{(b)}(x))/\sum_{k=1}^{K}\exp(\eta_k^{(b)}(x))$.

Express the new predictors.
For every category $j$, including $r$, the new predictor is $\eta_j^{(r)}(x)=\eta_j^{(b)}(x)-\eta_r^{(b)}(x)$. For $j=r$, this equals zero, as required for the new reference.

Recover the probabilities.
The new denominator is $\sum_k\exp(\eta_k^{(b)}(x)-\eta_r^{(b)}(x))=\exp(-\eta_r^{(b)}(x))\sum_k\exp(\eta_k^{(b)}(x))$. The numerator for category $j$ has the same factor $\exp(-\eta_r^{(b)}(x))$, so it cancels. Thus each new probability equals the old $p_j(x)$. $\square$

The transformation is reversible, which confirms that no category-specific information has been lost. Starting with reference $r$, the coefficients for the former reference $b$ are $\alpha_b^{(r)}=-\alpha_r^{(b)}$ and $\beta_b^{(r)}=-\beta_r^{(b)}$. For any $j\ne b,r$, subtracting the new coefficient for $b$ from that for $j$ returns $\alpha_j^{(b)}$ and $\beta_j^{(b)}$. Thus the two parameterizations are different coordinates for the same collection of probability functions.

Worked Coefficient Transformations

Worked Example: Re-basing a Three-Category Model

Let the categories be $A,B,C$, with $A$ initially the reference. Suppose $x$ is a single predictor and the fitted logits are $$ \log(p_B/p_A)=-0.4+0.8x,\qquad \log(p_C/p_A)=0.7-0.2x. $$ Change the reference to $C$.

For category $A$, the new logit is the negative of the old $C$-versus-$A$ logit: $$ \log(p_A/p_C)=-(0.7-0.2x)=-0.7+0.2x. $$ For category $B$, subtract the old $C$-versus-$A$ logit from the old $B$-versus-$A$ logit: $$ \log(p_B/p_C)=(-0.4+0.8x)-(0.7-0.2x)=-1.1+1.0x. $$ The transformed intercepts are $-0.7$ for $A$ and $-1.1$ for $B$; the transformed slopes are $0.2$ and $1.0$, respectively. For example, at $x=1$, the old logits are $0.4$ for $B$ versus $A$ and $0.5$ for $C$ versus $A$. The new logits are $-0.5$ for $A$ versus $C$ and $-0.1$ for $B$ versus $C$, exactly the corresponding differences.

Worked Example: Checking Probability Invariance

At a particular predictor value, suppose a three-category model uses $A$ as reference and gives $\eta_A^{(A)}=0$, $\eta_B^{(A)}=\log 2$, and $\eta_C^{(A)}=\log 4$. Find all fitted probabilities, then calculate them again using $C$ as reference.

With $A$ as reference, the unnormalized weights are $1,2,4$. Their sum is $7$, so $$ (p_A,p_B,p_C)=\left(\frac17,\frac27,\frac47\right). $$ With $C$ as reference, the new logits are $$ \eta_A^{(C)}=-\log 4,\qquad \eta_B^{(C)}=\log 2-\log 4=-\log 2,\qquad \eta_C^{(C)}=0. $$ The corresponding weights are $1/4,1/2,1$, with sum $7/4$. Dividing each weight by $7/4$ gives $1/7,2/7,4/7$, respectively. The probabilities agree category by category. The logits and weights look different because they are relative to a different reference, but normalization yields the same fitted distribution.

Which Interpretations Depend on the Reference?

An individual slope $\beta_{j\ell}^{(b)}$ is the change in $\log(p_j/p_b)$ for a one-unit increase in predictor $x_\ell$, with the other predictors held fixed. Therefore, its meaning names both the target category $j$ and the reference category $b$. Its exponent is the multiplicative change in the odds $p_j/p_b$ for that increase. Changing the reference changes which odds are being described, and generally changes the numerical coefficient.

A contrast between two nonreference categories can instead be written as a difference of baseline logits. Under reference $b$, $$ \log(p_j/p_k)=\eta_j^{(b)}(x)-\eta_k^{(b)}(x),\qquad \beta_j^{(b)}-\beta_k^{(b)} $$ is its slope vector. This category-versus-category contrast does not depend on which common reference was used to calculate it: subtracting the two new coefficients cancels the coefficient for the new reference. The individual baseline coefficients depend on the chosen reference; the pairwise contrast they imply does not.

Worked Example: A Reference-Dependent Slope and an Invariant Contrast

Suppose $A$ is the reference and a binary predictor $x$ has coefficients $$ \log(p_B/p_A)=-0.2+0.6x,\qquad \log(p_C/p_A)=0.4-0.1x. $$ A one-unit increase in $x$ multiplies the odds $p_B/p_A$ by $\exp(0.6)$, approximately $1.82$, and multiplies $p_C/p_A$ by $\exp(-0.1)$, approximately $0.90$. These are statements about comparisons with $A$.

Change the reference to $C$. The transformed logits are $$ \log(p_A/p_C)=-0.4+0.1x,\qquad \log(p_B/p_C)=-0.6+0.7x. $$ The slope for $B$ versus $C$ is $0.7$, not $0.6$. Its value is the old slope difference, $0.6-(-0.1)=0.7$, so a one-unit increase in $x$ multiplies $p_B/p_C$ by $\exp(0.7)$, approximately $2.01$. The value $0.7$ describes a different comparison from $0.6$, not a changed probability effect. In particular, the coefficient for $B$ versus $C$ can be recovered from the old fitted logits by subtracting the $C$-versus-$A$ slope from the $B$-versus-$A$ slope.

When Probability Invariance Applies

The reparameterization above assumes the category-specific intercepts and slopes are unrestricted: each nonreference category can have its own intercept and slope vector. Under that assumption, subtracting one category’s linear predictor from the others produces valid linear predictors in the same model class. For the same covariates and observations, changing only the reference is a one-to-one change of parameterization. Consequently, the set of possible fitted probability vectors is unchanged, and corresponding parameter values produce identical fitted probabilities.

A qualification matters in practice. If a model imposes restrictions on coefficients, such as equal slopes for selected categories, changing reference need not preserve those restrictions in the same form. For example, equality of two slopes relative to $b$ may not translate into equality of their slopes relative to $r$ unless the restriction is transformed as well. If a fitting procedure instead imposes a new, different restriction after changing the reference, it may fit a different model and give different probabilities. Invariance is guaranteed for a pure reparameterization, not for an altered set of constraints.

Common Mistake

Wrong reasoning: “The coefficient for $B$ changed after $C$ became the reference, so the model’s prediction for $B$ must have changed.” This treats a reference-dependent log-odds coefficient as if it were itself a predicted probability.
Correction: A baseline coefficient describes a log ratio, not a category probability. Recompute the probability using all category logits and the normalization denominator. Under the coefficient transformation, every category probability is unchanged.
Another trap: “The slope for $B$ is $0.6$, regardless of the reference.” A slope of $0.6$ in the example describes $B$ versus $A$; $B$ versus $C$ has slope $0.7$. Always state both categories in an interpretation.

Key takeaway: Changing the reference subtracts its old intercept and slope from every category’s old coefficients. In an unrestricted baseline-category model this changes the log-odds coordinates, not the fitted probabilities. Interpret each coefficient as a comparison with its stated reference.

Check Your Understanding

Use the coefficient transformation and distinguish the changed logits from the unchanged probabilities.

With $A$ as reference, suppose $\eta_B=-1+0.5x$ and $\eta_C=0.2+0.3x$. Write both logits when $B$ is the reference.
Explain why the coefficient of $x$ for $C$ versus $B$ is a difference of two slopes in the $A$-reference parameterization.
At one predictor value, the logits relative to $A$ are $0$, $\log 3$, and $\log 5$ for $A,B,C$. Compute the three probabilities. Would changing the reference alter them?
Which part of the interpretation of a baseline-category slope changes when the reference changes: the predictor, the response categories being compared, or the fitted probability for a category?
Why might refitting a model with a newly imposed slope constraint fail to preserve the original fitted probabilities?