Why categorical predictors need a compatible update
A binary predictor is often represented by a zero-one indicator, while a nominal predictor is represented by category contrasts. That coding can make the substantive model look simple, but it does not make the predictor’s imputation model automatically compatible with the analysis. If the outcome depends on category differently across values of a continuous covariate, an ordinary categorical imputation model may fail to preserve that relationship. SMC-FCS addresses this by weighting each possible category according to both its covariate-model probability and its likelihood under the substantive model.
For a binary predictor, a squared indicator is redundant because \(X^2=X\). Nonlinear structure instead commonly arises through interactions with a continuous predictor or through category-specific coefficients on nonlinear functions such as \(W^2\). For a nominal predictor, it is especially important not to interpret arbitrary numeric category labels as a continuous scale. The substantive model should encode categories using indicators or contrasts, and can allow category-specific effects of nonlinear terms.
The parameters \(\alpha\) and \(\beta\) may be current parameter values or draws used in the imputation procedure. The examples below use fixed values to show the probability calculation. In actual multiple imputation, parameter uncertainty should be represented according to the implementation’s procedure; repeatedly drawing categories with fixed parameters alone does not, by itself, account for that uncertainty.
Finite-category conditional distribution
Suppose \(X\) can take one of \(K\) values \(x_1,\ldots,x_K\). Given the observed \(Y=y\) and \(Z=z\), define the unnormalized weight for category \(x_k\) as
The conditional probability used to draw the missing value is
This follows from Bayes’ rule: the joint conditional density for \(Y\) and \(X=x_k\), given \(Z=z\), factors as \(f(y\mid x_k,z;\beta)g(x_k\mid z;\alpha)\). Summing that joint quantity over all possible values of \(X\) gives the normalizing density for \(Y\mid Z=z\). Dividing by this sum yields probabilities that are nonnegative and add to one.
For a binary outcome under a logistic substantive model, write \(q_k=P(Y=1\mid X=x_k,Z=z)\). The likelihood contribution is \(q_k^y(1-q_k)^{1-y}\), so the update weights are
Thus an observed \(Y=1\) favors categories with larger fitted event probabilities, while an observed \(Y=0\) favors categories with smaller fitted event probabilities. This direction is combined with, not substituted for, the covariate-model probabilities.
Worked updates
Worked Example: A Binary Predictor With a Nonlinear Interaction
Hypothetical setup. Let \(X\) be a binary predictor, \(W\) a fully observed continuous predictor, and \(Y\) a binary outcome. The substantive logistic model is
The \(XW^2\) term allows the contribution associated with \(X=1\) to vary with the squared value of \(W\). For a case with missing \(X\), suppose \(W=2\), \(Y=1\), and the covariate model gives \(P(X=1\mid W=2)=0.4\). The complementary probability is \(P(X=0\mid W=2)=0.6\). These are valid probabilities and both candidate categories have positive support.
For \(X=0\), the linear predictor is \(-1+0.2(2)-0.1(2^2)=-1\), giving \(q_0=\operatorname{expit}(-1)=0.268941\). For \(X=1\), it is \(-1+0.4-0.4+0.5+1.2=0.7\), giving \(q_1=\operatorname{expit}(0.7)=0.668188\). Since \(Y=1\), the likelihood contributions are \(q_0\) and \(q_1\). Therefore,
and the unrounded weights sum to \(0.428640\). The update probability of \(X=1\) is \(0.267275/0.428640=0.6235\), and that of \(X=0\) is \(0.3765\). A draw is therefore \(X=1\) with probability about \(0.6235\), not with probability \(0.4\): the outcome likelihood has updated the covariate-model probability.
Worked Example: Three Categories With Category-Specific Curvature
Hypothetical setup. Let \(X\) have nominal categories A, B, and C, and suppose the logistic substantive model has linear predictor
The category-specific \(W^2\) coefficients make the outcome relationship vary across categories. For a hypothetical case with \(W=2\) and \(Y=0\), suppose the covariate model gives probabilities \(0.2,0.5,0.3\) for A, B, and C. Each is positive and they sum to one. The linear predictors are \(\eta_A=-1.2\), \(\eta_B=0.2\), and \(\eta_C=0.5\). Thus the fitted event probabilities are \(0.231475\), \(0.549834\), and \(0.622459\).
Because the observed outcome is zero, the respective likelihoods are one minus those probabilities: \(0.768525\), \(0.450166\), and \(0.377541\). The unnormalized weights are
Their unrounded sum is \(0.492050\). Normalizing gives probabilities \(P(X=A\mid y,z)=0.3124\), \(P(X=B\mid y,z)=0.4574\), and \(P(X=C\mid y,z)=0.2302\). Category B remains most probable, but its probability falls from the covariate-model value \(0.5\) because its fitted event probability is relatively high while this case has \(Y=0\).
Worked Example: A Continuous Outcome With Two Categories
Hypothetical setup. Suppose \(Y\mid X,W\) is normally distributed with standard deviation \(\sigma=1.5\), and its mean has category-specific quadratic forms. At \(W=2\), let the specified means be \(\mu_A=2+0.5W+0.25W^2=4\) and \(\mu_B=1+0.75W+0.75W^2=5.5\). For a case with \(Y=5\), suppose \(P(X=A\mid W)=0.7\) and \(P(X=B\mid W)=0.3\). The normal outcome model uses the same positive standard deviation for both candidate categories.
The normal density for category \(k\) is proportional, across these candidates, to \(\exp\{-(y-\mu_k)^2/(2\sigma^2)\}\). The common factor \(1/(\sqrt{2\pi}\sigma)\) cancels when the weights are normalized. The likelihood kernels are \(\exp\{-1^2/(2(1.5)^2)\}=0.800737\) for A and \(\exp\{-(-0.5)^2/(2(1.5)^2)\}=0.945959\) for B. Hence,
The sum is \(0.844304\), giving probabilities \(P(X=A\mid y,w)=0.6639\) and \(P(X=B\mid y,w)=0.3361\). Although B has the larger outcome likelihood, A remains more probable because its covariate-model probability is substantially larger. The update balances both sources of information.
Assumptions and practical checks
The update is meaningful only relative to the models used to construct it. The substantive outcome distribution must represent the intended analysis, including its link, functional form, and category coding. The covariate model must adequately represent the distribution of the missing predictor given the variables included in that model. For the conditional probabilities to be estimable, candidate categories need support in the relevant covariate patterns, and the observed outcome must have a well-defined likelihood under each candidate. The missingness assumptions needed for the intended imputation analysis, commonly a suitable missing-at-random assumption conditional on included information, are separate from this probability calculation; SMC-FCS does not make an inappropriate missingness assumption valid.
With a finite number of categories, the update is directly computable: evaluate the substantive likelihood at each category, multiply by its covariate-model probability, normalize, and draw. Useful checks include:
- Confirm that the category probabilities from the covariate model are nonnegative and sum to one.
- Check that the substantive model is evaluated with the intended reference category and all relevant interactions or nonlinear terms.
- Verify that the normalized update probabilities sum to one and are not systematically concentrated on a category because of a coding or numerical error.
- Compare imputed category frequencies across chains and across important observed covariate or outcome groups. Implausible frequencies or persistent chain differences can signal poor support, model misspecification, or computational problems.
Extreme weights deserve investigation rather than automatic correction. They can arise from a genuinely strong outcome association, but can also indicate separation, sparse categories, or a poor substantive-model fit. If likelihoods are very small, calculations on the log scale can help: add the log covariate probability and log likelihood for each category, then normalize using a stable log-sum-exp calculation.
How this update differs from simpler alternatives
| Approach | How it updates a missing category | Important distinction |
|---|---|---|
| Covariate-model draw alone | Draws from \(g(X\mid Z;\alpha)\) | Does not use the observed outcome likelihood in the update. |
| SMC-FCS categorical update | Weights \(g(X\mid Z;\alpha)\) by \(f(Y\mid X,Z;\beta)\) | Uses the specified substantive model when assigning category probabilities. |
| Joint model | Specifies and estimates a joint distribution for the relevant variables | Can be a coherent alternative, but requires a suitable joint model and is not the same conditional-update construction. |
A categorical covariate model that includes the outcome can sometimes represent important associations, but merely adding outcome terms does not guarantee compatibility with a complicated substantive model. Conversely, SMC-FCS cannot repair a misspecified substantive model: its update is compatible with the model supplied, not with an unknown true relationship. A nominal predictor with many sparse levels can also yield unstable covariate probabilities and weakly supported conditional updates. Category grouping, if scientifically justified, changes the estimand and must not be done just to make computation easier.
Common mistakes
- Using the covariate-model probability as the final draw probability. That omits the outcome likelihood. Multiply the covariate probability by the substantive likelihood for each candidate category, then normalize.
- Treating nominal labels as an ordinal number. Category codes such as 1, 2, and 3 do not define a meaningful linear or quadratic effect unless the ordering and numerical scale are substantively justified. Use contrasts for nominal categories.
- Assuming a binary predictor needs a separate squared term. For a zero-one indicator, \(X^2=X\). Nonlinear contributions instead arise, for example, when the binary indicator interacts with \(W^2\).
- Ignoring the observed outcome in the likelihood calculation. For a binary outcome, use \(q_k\) when \(Y=1\) and \(1-q_k\) when \(Y=0\); choosing the wrong likelihood reverses the update’s direction.
- Interpreting compatibility as proof of correctness. The update respects the stated substantive model only under the modeling and support conditions. It does not establish that the substantive or covariate model is correctly specified.
Check Your Understanding
Answer each question before reading the brief solution immediately after it.
- A binary outcome model gives \(q_0=0.2\), \(q_1=0.6\), and the covariate model gives probabilities \(0.7\) and \(0.3\) for \(X=0\) and \(X=1\). If the observed outcome is \(Y=0\), what are the normalized update probabilities?
- Why is a squared term for a zero-one predictor not, by itself, a nonlinear effect?
- A nominal three-level predictor is coded 1, 2, and 3. What must be true before treating those values as a quantitative predictor in a polynomial model?
- In the three-category example above, why did the update probability for category B decrease from its covariate-model probability?
Solutions. (1) The weights are \(0.7(1-0.2)=0.56\) and \(0.3(1-0.6)=0.12\), with sum \(0.68\). Thus \(P(X=0\mid y,z)=0.56/0.68=0.8235\) and \(P(X=1\mid y,z)=0.12/0.68=0.1765\). (2) Since \(X\) is either zero or one, \(X^2=X\) for both possible values, so the squared term duplicates the original indicator. A varying effect can instead be represented by an interaction such as \(XW^2\). (3) The categories must have a substantively meaningful order and numerical spacing that justify arithmetic and a polynomial trend; otherwise use category contrasts. (4) Category B had a relatively high fitted event probability, \(0.549834\), so its likelihood for the observed \(Y=0\) was \(1-0.549834=0.450166\). Multiplying by its covariate probability \(0.5\) and normalizing reduced its probability to about \(0.4574\).