From a Fitted Model to an Imputed Category
When a variable has a small number of unordered categories, a multinomial imputation model can estimate the probability of each category conditional on other variables. The practical task is then to use those probabilities to draw a category for each missing value. A draw is not the same as assigning the most likely category: it preserves variation among records with similar predictors and is part of making an imputation stochastic.
Suppose the variable is service type, with categories primary care, specialist care, and emergency care. These categories have no natural order. In a MICE iteration, the other variables are treated as predictors in the current completed data, the model is fitted using records whose service type is observed, and a category is drawn for each record whose service type is missing. The observed service-type values are retained.
The Model and Its Conditions
For a response \(Y\) with \(K\) categories, choose one category as the baseline. For each nonbaseline category \(k\), a baseline-category multinomial logistic model specifies the log odds of category \(k\) versus the baseline as a linear function of predictors. With primary care as the baseline and predictors collected in a vector \(x\), the model is
Writing \(\eta_k=\alpha_k+x^\mathsf{T}\beta_k\), the category probabilities are
The model assumes that the response categories are mutually exclusive and collectively exhaustive, that records contribute independently to the multinomial likelihood conditional on predictors, and that the chosen predictor effects adequately describe the conditional log odds. For a continuous predictor, a linear term assumes a linear effect on each log-odds scale unless nonlinear terms are included. The design must also identify the coefficients: for example, predictors cannot be perfectly collinear, and the data must not exhibit complete separation that prevents finite maximum-likelihood estimates.
For use in multiple imputation, an additional missing-data condition is relevant: imputations based on observed data and included variables are justified under a suitable missing-at-random assumption, together with an adequately specified conditional model. Predictors should include variables in the substantive analysis that help explain the nominal variable or its missingness, as appropriate. These conditions do not guarantee a good model; they identify what the model’s predictions rely on.
In an iteration, the model is estimated from records with observed \(Y\), using their current predictor values. A parameter draw is then taken from an appropriate estimated sampling distribution or posterior distribution, and probabilities are calculated from that draw. Finally, each missing response is sampled from its category probabilities. The particular parameter-drawing method depends on the implementation, but simply using fixed fitted coefficients omits parameter uncertainty.
Worked Probability Calculations
Worked Example: Calculating Probabilities for a Referral Indicator
Hypothetical data. In a clinic dataset, service type has three categories: primary care (baseline), specialist care, and emergency care. The predictor \(R\) indicates whether the patient was referred (\(R=1\)) or not (\(R=0\)). Suppose a multinomial model fitted to the observed service types gives specialist-versus-primary coefficients \(\hat{\alpha}_S=0.20\), \(\hat{\beta}_S=0.80\), and emergency-versus-primary coefficients \(\hat{\alpha}_E=-0.40\), \(\hat{\beta}_E=1.10\).
For an unreferred patient, \(R=0\), so the fitted logits are \(\hat{\eta}_S=0.20+0.80(0)=0.20\) and \(\hat{\eta}_E=-0.40+1.10(0)=-0.40\). Thus \(\exp(\hat{\eta}_S)=1.221\) and \(\exp(\hat{\eta}_E)=0.6703\). The denominator is \(1+1.221+0.6703=2.891\), giving
The probabilities sum to \(0.3459+0.4223+0.2318=1.0000\), subject to rounding. For a referred patient, the logits are \(1.00\) and \(0.70\). Since \(\exp(1.00)=2.718\) and \(\exp(0.70)=2.014\), the denominator is \(1+2.718+2.014=5.732\). The probabilities are primary \(1/5.732=0.1745\), specialist \(2.718/5.732=0.4742\), and emergency \(2.014/5.732=0.3513\). Referral is associated with a larger fitted probability of specialist and emergency care in this hypothetical model; this is a conditional association, not a causal conclusion.
The model’s use here presumes the categories are exhaustive and unordered, the referral indicator is available, and the observed records provide information for both referral groups. In an actual analysis, separation, coding, and the adequacy of the predictor specification would need checking rather than being assumed from these estimates.
Worked Example: Drawing Categories for Two Missing Values
Use the referred-patient probabilities from the previous calculation: primary \(0.1745\), specialist \(0.4742\), and emergency \(0.3513\). To draw a category, generate \(U\) uniformly on \([0,1]\), and assign the first category whose cumulative probability is at least \(U\). The cumulative cut points are \(0.1745\) and \(0.1745+0.4742=0.6487\).
For the first missing service type, suppose the random number is \(U=0.60\). Since \(0.1745<0.60\leq0.6487\), draw specialist care. For a second missing service type with the same predictors, suppose \(U=0.90\). Since \(0.90>0.6487\), draw emergency care. Both records have the same model probabilities, but their stochastic draws differ.
The probabilities are not frequencies that must be reproduced exactly in a small set of imputations. Each draw is a random outcome with the specified conditional probabilities. Assigning specialist care to both records because it has the largest probability would instead be deterministic classification and would suppress uncertainty about their missing categories.
Why Parameter Uncertainty Matters
The probabilities above were calculated from fitted coefficients as though those coefficients were fixed. That calculation is useful for understanding the model, but a multiple-imputation procedure should also reflect uncertainty in the coefficients. Otherwise the imputed values can be too concentrated around a single fitted model, especially when the observed sample is limited or the fitted coefficients are uncertain.
Worked Example: A Parameter Draw Changes the Category Draw
Return to a referred patient. The fitted logits were \(1.00\) for specialist versus primary and \(0.70\) for emergency versus primary. Suppose a legitimate parameter draw in an imputation iteration instead gives logits \(0.80\) and \(0.90\). This is an illustrative draw, not a claim about the exact sampling distribution for the earlier data.
The exponentiated logits are \(\exp(0.80)=2.226\) and \(\exp(0.90)=2.460\); the denominator is \(1+2.226+2.460=5.686\). The resulting probabilities are primary \(1/5.686=0.1759\), specialist \(2.226/5.686=0.3915\), and emergency \(2.460/5.686=0.4326\), up to rounding. With a new \(U=0.60\), the cumulative primary-plus-specialist probability is \(0.1759+0.3915=0.5674\), so this parameter draw leads to an emergency-care imputation. Under the original fitted probabilities, the same \(U=0.60\) would have selected specialist care.
A proper implementation draws parameters according to its stated uncertainty procedure and then draws categories conditional on those parameters. This two-stage randomness represents uncertainty about both the model probabilities and the missing response. It does not mean that the parameter draw is arbitrary; it must come from the model’s estimated uncertainty distribution.
Support, Diagnostics, and Failure Modes
A multinomial model shares coefficients across predictor patterns. Therefore, a category’s absence in one exact predictor pattern does not, by itself, prevent the model from assigning that category a probability there. The prediction may use information from other patterns through the shared model. However, little or no support for a predictor pattern can make its predicted probabilities depend strongly on modeling assumptions and may make them unreliable. The key issue is the information supporting the model and how far the prediction extrapolates, not merely whether a category appeared in that exact pattern.
Worked Example: Predicting for a Thinly Supported Pattern
Suppose a model also includes centered age \(z=(\text{age}-50)/10\). Its fitted logits are \(\hat{\eta}_S=0.20+0.80R+0.30z\) and \(\hat{\eta}_E=-0.40+1.10R-0.20z\). Consider a hypothetical 70-year-old referred patient, so \(R=1\) and \(z=2\). The predicted logits are \(0.20+0.80+0.60=1.60\) and \(-0.40+1.10-0.40=0.30\). The exponentials are \(4.953\) and \(1.350\), and the denominator is \(1+4.953+1.350=7.303\). The resulting probabilities are primary \(0.1369\), specialist \(0.6782\), and emergency \(0.1848\), with a sum of \(0.9999\) due to rounding.
Suppose there are few or no observed records with this precise combination of age and referral status. The model can still produce these probabilities from its shared coefficients, but the data may give limited direct support for the prediction. Check the distribution of predictors among records with observed service type, compare missing and observed records, and inspect predicted probabilities for extreme or implausible values. A prediction outside the well-supported predictor range warrants caution; absence in the exact pattern alone is not proof that the estimate is unusable.
Practical checks include confirming category coding and the baseline, verifying that probabilities are nonnegative and sum to one, reviewing convergence and coefficient estimates, and looking for very large estimates or standard errors that may signal separation or weak information. Compare predictor distributions for records with observed and missing outcomes, and examine whether the imputed category proportions and predictor relationships are plausible across completed datasets. These checks can reveal problems but cannot prove that the conditional model is correct.
A multinomial model is appropriate when the categories are unordered. If categories have a defensible order, an ordinal model may use that structure, but it relies on its own assumptions. A binary logistic model is suitable only for a genuinely two-category outcome. Deterministic assignment to the modal category is not a substitute for stochastic imputation; it discards variation. Other approaches may be useful when the multinomial model is poorly supported or misspecified, but the choice must preserve the outcome’s categorical nature and represent uncertainty appropriately.
A Reliable Imputation Sequence
Confirm that the nominal categories are exhaustive and unordered, and identify predictors available in the current MICE iteration.
Use records with observed response values and check identifiability, convergence, and support for the predictors.
Draw coefficients using the method specified by the imputation procedure, rather than automatically treating point estimates as known.
Apply the multinomial probability formula for each missing record, then sample one category from its probability vector.
Confirm valid categories, plausible conditional patterns, and no unintended changes to observed responses.
The sequence is important: predicted probabilities are intermediate quantities, not the imputed values themselves. A missing response is categorical, so its imputation must be a valid category, sampled in a way that carries forward model and response uncertainty.
Common Mistakes and Limitations
- Choosing the most likely category every time. The modal category is a classification decision, not a stochastic imputation draw. Sample using the full probability vector.
- Using a probability vector that does not sum to one. Apply the baseline-category softmax denominator across all categories, including the baseline.
- Treating fixed coefficient estimates as certainty. Fitted probabilities are conditional on estimated parameters; proper multiple imputation should represent parameter uncertainty as well.
- Calling a category unestimable because it is absent in one exact predictor pattern. Shared coefficients can produce a probability for that pattern. Assess overall information and extrapolation instead.
- Ignoring model form and missingness assumptions. Stochastic draws cannot correct a badly specified conditional model or make an unsuitable missingness assumption valid.
A multinomial model can be limited by sparse data, separation, unstable estimates, nonlinear predictor relationships, or poor support in the patterns needing imputation. Predicted probabilities may appear numerically well behaved even when the model depends heavily on extrapolation. Diagnostics and substantive review remain necessary; no probability calculation alone establishes that the imputations are credible.
Check Your Understanding
Use the model and ideas in this tutorial to answer the questions. Short solutions follow the questions.
- For three categories, why must the baseline probability be included in the softmax denominator?
- A record has probabilities \(0.20\), \(0.50\), and \(0.30\) in category order. Which category is drawn when \(U=0.68\), using the cumulative-probability rule?
- Why is assigning the largest-probability category to every missing record not equivalent to multiple imputation?
- What should be concluded if one category is absent among observed records with a particular predictor pattern?
Solutions. (1) The baseline has relative weight 1, so the denominator is \(1+\sum_{k=2}^{K}\exp(\eta_k)\); omitting it would leave the category probabilities improperly normalized. (2) The cumulative probabilities are \(0.20\), \(0.70\), and \(1.00\), so \(0.20<0.68\leq0.70\) selects category 2. (3) Modal assignment ignores the nonmodal probabilities and removes variation among records with similar predictors; stochastic category draws retain that variation, while parameter draws can also represent uncertainty in the fitted model. (4) Its absence in that exact pattern alone does not establish non-estimability. A shared-coefficient model may predict the category there, but limited support can make the prediction assumption-dependent and unreliable, so assess the broader information and extrapolation.