Why the Reference Category Is a Choice of Coordinates
For a nominal variable with three or more categories, a multinomial-logit model compares each nonreference category with a chosen reference category. The choice determines how the coefficients are written and interpreted. It does not, by itself, determine different fitted probabilities for the categories.
This distinction matters in MICE because a nominal variable is imputed by drawing a category from its conditional probabilities. If the same multinomial model is merely expressed using another reference category, those probabilities should not change. The coefficient values will change, but the category probabilities used for imputation should agree.
The model assumes that each observation belongs to exactly one of the \(K\) categories and that the predictors are available for the cases being modelled. For the usual likelihood fit, observations are treated as independent conditional on their predictors, and the specified linear predictors are used for the log category-probability ratios. The model must also be estimable from the observed data. The algebraic invariance below does not require that the model fit well: it describes what happens when the same model is re-expressed with a different reference.
The Transformation When the Baseline Changes
Let \(b\) be the original reference and \(r\) a different reference. For category \(k\) other than \(b\) or \(r\), subtract the old logit for \(r\) from the old logit for \(k\):
For the old reference category itself, the new logit is \(\log\{P(Y=b\mid x)/P(Y=r\mid x)\}=-\eta_{rb}(x)\). These equations give every new linear predictor. In particular, if the old predictors have intercepts and slopes, the new intercepts and slopes are the corresponding differences; the old reference's new coefficients are the negatives of the old reference-category predictor.
A short justification establishes why this is a re-expression rather than a different probability model. Under the old baseline, the relative weights are \(w_b=1\) and \(w_k=\exp\{\eta_{kb}(x)\}\) for \(k\ne b\). The category probabilities are each weight divided by their sum. Changing the reference divides every weight by \(w_r\). Dividing every term in a sum by the same positive number does not change the normalized shares:
The new relative weight for \(r\) is 1, as required, and the other new weights are precisely the exponentials of the transformed logits. Thus, for every predictor value, all \(K\) fitted category probabilities are unchanged.
Worked Example: Re-expressing Three Category Logits
Suppose a hypothetical three-category response has categories A, B, and C. With A as the reference, let \(\eta_{BA}(x)=0.4+0.3x\) and \(\eta_{CA}(x)=-0.2+0.1x\). Find the probabilities at \(x=1\), then re-express the logits with C as the reference.
At \(x=1\), the old logits are \(\eta_{BA}(1)=0.7\) and \(\eta_{CA}(1)=-0.1\). The relative weights for A, B, and C are \(1\), \(e^{0.7}\), and \(e^{-0.1}\). Therefore the denominator is \(1+e^{0.7}+e^{-0.1}\approx 3.9186\). Using \(e^{0.7}\approx2.0138\) and \(e^{-0.1}\approx0.9048\), the probabilities are \(P(A\mid x=1)\approx0.2552\), \(P(B\mid x=1)\approx0.5139\), and \(P(C\mid x=1)\approx0.2309\).
With C as the reference, the new logits are \(\eta_{AC}(x)=-\eta_{CA}(x)=0.2-0.1x\) and \(\eta_{BC}(x)=\eta_{BA}(x)-\eta_{CA}(x)=0.6+0.2x\). At \(x=1\), these are \(0.1\) and \(0.8\). The new denominator is \(1+e^{0.1}+e^{0.8}\approx4.3307\). The resulting probabilities are \(P(C\mid x=1)\approx0.2309\), \(P(A\mid x=1)\approx0.2552\), and \(P(B\mid x=1)\approx0.5139\). The category probabilities agree, with only their order in the calculation changed.
Coefficients Change, Comparisons Change
A coefficient is a log ratio against the selected reference, not an absolute property of a category. With a predictor \(x\), a slope in \(\eta_{kb}(x)\) describes the change in the log ratio of category \(k\) to category \(b\) for a one-unit increase in \(x\), holding the other predictors fixed. Changing the reference changes the comparison being described. It does not imply that the underlying fitted category probabilities have changed.
For example, if the old model has \(\eta_{kb}(x)=a_k+c_kx\) and \(\eta_{rb}(x)=a_r+c_rx\), then the new coefficient for \(x\) in the logit comparing \(k\) with \(r\) is \(c_k-c_r\), and its intercept is \(a_k-a_r\). Reversing a comparison negates both its intercept and slopes. These are algebraic consequences of the log-ratio definition.
Worked Example: Transforming a Full Set of Coefficients
In a hypothetical model with categories A, B, C, and D, take A as the original reference. Suppose \(\eta_{BA}(x)=-0.6+0.5x\), \(\eta_{CA}(x)=0.2-0.3x\), and \(\eta_{DA}(x)=1.1+0.1x\). Re-express the model with D as the reference.
The logit for A relative to D is the negative of the old D-versus-A logit: \(\eta_{AD}(x)=-1.1-0.1x\). For B relative to D, subtract the D-versus-A predictor from the B-versus-A predictor: \(\eta_{BD}(x)=(-0.6+0.5x)-(1.1+0.1x)=-1.7+0.4x\). Likewise, \(\eta_{CD}(x)=(0.2-0.3x)-(1.1+0.1x)=-0.9-0.4x\).
At \(x=2\), the old logits for B, C, and D relative to A are \(0.4\), \(-0.4\), and \(1.3\). The old denominator is \(1+e^{0.4}+e^{-0.4}+e^{1.3}\approx6.8314\). Thus the probabilities for A, B, C, and D are approximately \(0.1464\), \(0.2184\), \(0.0981\), and \(0.5371\), respectively.
Under the D reference, the logits at \(x=2\) are \(-1.3\), \(-0.9\), and \(-1.7\) for A, B, and C. The new denominator is \(1+e^{-1.3}+e^{-0.9}+e^{-1.7}\approx1.8618\). The corresponding probabilities for D, A, B, and C are approximately \(0.5371\), \(0.1464\), \(0.2184\), and \(0.0981\). The rounded values agree with the first calculation.
Why the Fitted Probabilities Are Invariant
The coefficient transformation applies to any fixed set of logits, but it also explains the result of fitting the model. In an ordinary unpenalized multinomial-logit fit, changing the reference gives a one-to-one transformation between coefficient sets: the old predictors determine the new predictors by subtraction, and the original predictors can be recovered by reversing the same comparisons. The two parameterizations therefore describe the same set of possible category-probability vectors.
The multinomial likelihood for observed categories depends on the probabilities assigned to those observations. Because the re-expression leaves every probability unchanged and can represent the same probability vectors, the likelihood is unchanged at corresponding parameter values. Consequently, when both fits use the same observations, predictors, model structure, and unpenalized likelihood, their fitted probabilities agree, apart from possible numerical optimization or rounding error.
This conclusion is about fitted probabilities, not identical coefficient tables. Coefficients, standard errors, and coefficient-level tests are expressed for different category comparisons after a reference change. A contrast comparing the same two categories should be formed and interpreted consistently; a coefficient against one baseline cannot be compared directly with a coefficient against another baseline as if they represented the same contrast.
Worked Example: Checking an Imputation Probability and Draw
Suppose a hypothetical nominal variable has categories Red, Blue, and Green. For one case with its predictors fixed, a model using Red as the reference gives the logits \(\log\{P(Blue)/P(Red)\}=0.2\) and \(\log\{P(Green)/P(Red)\}=-0.5\). The relative weights are \(1\), \(e^{0.2}\), and \(e^{-0.5}\); the denominator is \(1+e^{0.2}+e^{-0.5}\approx2.8279\). Thus the conditional probabilities are approximately \(0.3536\) for Red, \(0.4319\) for Blue, and \(0.2145\) for Green.
Now use Green as the reference. The Red-versus-Green logit is \(0.5\), and the Blue-versus-Green logit is \(0.7\). The new denominator is \(1+e^{0.5}+e^{0.7}\approx4.6625\). The resulting probabilities are approximately \(0.2145\) for Green, \(0.3536\) for Red, and \(0.4319\) for Blue: the same probabilities assigned to the same categories.
For a categorical imputation draw, imagine generating \(U=0.30\) from a uniform distribution on \([0,1)\), and assigning categories in the order Red, Blue, Green. Under either representation, the cumulative intervals are approximately \([0,0.3536)\) for Red, \([0.3536,0.7855)\) for Blue, and \([0.7855,1)\) for Green. Since \(0.30\) falls in the Red interval, the draw is Red under both parameterizations. This illustrates why re-expressing the same probabilities does not alter the categorical draw when the category order and uniform draw are held fixed.
Practical Checks and Limitations
A reference change is a useful implementation check. Fit the same unpenalized model with two different reference categories, calculate fitted probabilities for the same predictor patterns, and compare probabilities category by category. Small discrepancies may result from numerical tolerance or rounding. Large discrepancies call for investigation: the fits may have used different records, predictor terms, category definitions, constraints, or estimation procedures.
The invariance claim has limits. If a fitting procedure imposes a penalty or prior that treats coefficients differently in one parameterization than another, its fitted estimates need not be invariant unless that penalty or prior is transformed appropriately. Likewise, a model with constraints may not allow the same probability vectors under both parameterizations. These are changes to the fitting specification, not counterexamples to the algebraic identity for an unchanged set of logits.
Changing the reference is also not a remedy for unstable estimation. If a category is poorly represented in the observed data, the coefficients may be imprecise or difficult to estimate no matter which category is named as baseline. The transformation can make a coefficient table easier to read, but it cannot create information or repair a poorly supported model.
Check Your Understanding
Answer each question, then check the solutions below.
- With A as reference, suppose \(\eta_{BA}(x)=1+0.2x\) and \(\eta_{CA}(x)=0.3-0.1x\). What is the logit for B relative to C after changing the reference to C?
- If \(\eta_{BA}(x)=-0.4+0.6x\) and \(\eta_{CA}(x)=0.5+0.1x\), what is the new intercept and slope for A relative to C?
- In one sentence, explain why changing the reference does not change the category probabilities.
- A model fit with a new reference produces noticeably different fitted probabilities. Name two features of the analysis to check.
Solutions. (1) Subtract the C-versus-A logit from the B-versus-A logit: \((1+0.2x)-(0.3-0.1x)=0.7+0.3x\). (2) The A-versus-C logit is the negative of the C-versus-A logit, so its intercept is \(-0.5\) and its slope is \(-0.1\). (3) Changing the reference divides every category's relative weight by the same positive weight, which leaves each normalized probability unchanged. (4) Check, for example, whether both fits used the same records and predictors, whether category coding and constraints match, and whether either fit used a penalty or prior that was not transformed equivalently.