Deriving Category Probabilities From Multinomial Logits

From Relative Logits to Category Probabilities

A multinomial logistic model for a nominal variable describes how each nonbaseline category compares with a chosen baseline. Its fitted logits are useful summaries of relative odds, but they are not themselves probabilities. To impute a missing value, the model must supply a probability for every possible category, with those probabilities adding to one. This tutorial derives that conversion and shows how to use the resulting distribution for a stochastic draw.

Suppose \(Y\) has \(K\) nominal categories, labelled \(1,\ldots,K\), and category \(K\) is the baseline. For a predictor vector \(x\), write \(p_k(x)=P(Y=k\mid x)\). The baseline-category logits are \(\eta_k(x)=\log\{p_k(x)/p_K(x)\}\) for \(k=1,\ldots,K-1\). In a fitted multinomial logistic regression, each \(\eta_k(x)\) is given by a linear predictor, such as an intercept plus predictor coefficients multiplied by the values in \(x\).

The baseline weight is one because its log-odds against itself are zero: \(\log\{p_K(x)/p_K(x)\}=0\). For the other categories, exponentiating a logit gives the odds relative to the baseline, \(p_k(x)/p_K(x)\). Dividing each weight by the sum of all weights turns these relative quantities into probabilities.

Why Normalization Gives the Correct Probabilities

The conversion can be derived directly from the definition of the logits. For \(k<K\), exponentiation gives \(p_k(x)=p_K(x)\exp\{\eta_k(x)\}\). Summing over all categories and using the fact that the probabilities must sum to one yields:

Solving for the baseline probability and substituting it back into the other category probabilities gives the softmax form:

This derivation assumes that the fitted logits are finite and that the categories in the model are mutually exclusive and collectively exhaustive for the variable being imputed. With finite logits, each weight is positive and finite, so every probability is positive and the denominator is positive. The probabilities sum to one because their numerators sum to the denominator. In practice, probabilities may be extremely close to zero or one, but rounding should not be used to conceal computational problems.

Turning Probabilities into a Stochastic Imputation

For a missing nominal value, a stochastic imputation selects one category at random using its conditional probabilities. A convenient way to implement this is with a uniform random number \(U\) on \([0,1)\) and cumulative probability intervals. If the category probabilities in order are \(p_1,\ldots,p_K\), assign category 1 when \(0\leq U<p_1\), category 2 when \(p_1\leq U<p_1+p_2\), and continue in this way, assigning category \(K\) to the remaining interval. The length of each interval is its category's probability, so the probability of assigning category \(k\) is \(p_k\).

In a MICE iteration, predictors for a missing case include its observed predictors and the current values of other variables that are incomplete. The fitted conditional model produces the logits for that case; normalization gives its category probabilities; then the categorical draw replaces the missing entry. Observed values of the variable being imputed are not redrawn in that update. A later visit to the variable in the chain can use updated values of other incomplete variables.

The category draw represents uncertainty about which outcome occurs for a case under the fitted conditional distribution. Multiple imputation also needs to reflect uncertainty in the model parameters: a typical proper-imputation procedure draws model parameters from an appropriate posterior or sampling distribution, then calculates probabilities and draws categories using that parameter draw. If the same fitted coefficients are held fixed throughout, categorical randomness remains, but parameter uncertainty is omitted. The exact parameter-drawing method depends on the model-fitting procedure; the probability normalization and category draw are the same once a set of logits has been obtained.

Checks, Alternatives, and Common Failure Modes

Several checks make this step of the imputation algorithm auditable. First, confirm that one logit has been calculated for every nonbaseline category and that the baseline weight is one. Second, check that all probabilities are between zero and one and that their sum is one within numerical tolerance. Third, verify that the cumulative intervals cover \([0,1)\) without gaps or overlaps. Finally, inspect the imputed category frequencies across cases and iterations alongside the observed data and model-predicted probabilities. Large discrepancies may be meaningful, but can also reveal a coding, category-order, or model-fitting problem.

For numerical stability, very large logits can cause direct exponentiation to overflow even when the final probabilities are well-defined. A standard calculation subtracts the largest of the \(K\) log-weights from every log-weight before exponentiation. Subtracting the same constant from all log-weights leaves the normalized probabilities unchanged, because the common factor cancels from numerator and denominator. This is a computational safeguard, not a change to the model.

A deterministic alternative is to assign the category with the largest probability. That produces a modal prediction, not a stochastic imputation: cases with identical predictors and probabilities always receive the same category. It suppresses variation in the completed data and can distort category proportions. Separately fitting binary models for each category does not automatically provide a valid categorical distribution, because the resulting probabilities need not sum to one. For a nominal outcome, the multinomial model directly represents the competing categories and its normalized probabilities support a coherent draw. Ordinal models, by contrast, use an ordering assumption that is not appropriate merely because a variable has several categories.

The conversion itself does not guarantee that the conditional model is appropriate. Its probabilities are only useful for imputation if the predictor specification and fitted model reasonably represent the conditional distribution of the nominal variable. Model diagnostics should therefore consider fit and stability as well as arithmetic correctness. A valid sum-to-one check cannot detect a misspecified relationship, omitted relevant predictors, or inadequate parameter uncertainty. Extremely large logits, unstable estimates, repeated imputed patterns, or implausible conditional distributions warrant investigation rather than automatic acceptance.

The central distinction is between predicting a probability distribution and imputing a category. Normalization supplies the distribution for a case; random sampling supplies a category while retaining the model's uncertainty about the outcome. Neither step should be replaced by rounding logits, treating odds as probabilities, or selecting the modal category for every missing value.