Imputing Categorical and Bounded Variables

Why the Imputation Model Must Respect the Variable

An imputed value should be possible for the variable it replaces. A binary response cannot be 0.37, a travel mode cannot be a fraction of a category, and a bounded measurement such as a percentage cannot exceed its defined range. These may sound like simple data-cleaning requirements, but they shape the statistical model used to generate missing values.

In multiple imputation, missing values are replaced by repeated draws from models for the variables with missing data. For categorical and bounded variables, those models need to describe both how values relate to observed information and which values are possible. A model that ignores the variable’s type can produce impossible values; a model that always selects the most likely value can erase uncertainty.

Definition: A support-respecting imputation model assigns values only within the variable’s possible set and represents its relevant distributional features. For a binary variable, that set is \(\{0,1\}\); for an unordered categorical variable, it is the listed categories; for a bounded continuous variable, it is the variable’s allowable interval, possibly including or excluding its endpoints.

Choosing a suitable model does not by itself guarantee valid conclusions. The conditional relationships must be modeled adequately, relevant observed variables must be included, and the assumptions used to justify the missing-data analysis still matter. In particular, the usual multiple-imputation rationale relies on an appropriate missingness assumption such as missing at random, together with a suitable imputation model. Respecting the possible values is essential, but it does not replace those requirements.

Binary Variables: Model a Probability, Then Draw

For a binary variable \(X\), code the two values as 0 and 1. A standard imputation model is logistic regression, which models the probability that \(X=1\) conditional on observed information \(Z\):

\log\left(\frac{P(X=1\mid Z)}{1-P(X=1\mid Z)}\right) =\beta_0+\beta_1 Z_1+\cdots+\beta_k Z_k.

If the linear predictor is \(\eta\), the fitted probability is \(p=1/(1+\exp(-\eta))\), which lies between 0 and 1. To impute a missing value, draw 1 with probability \(p\) and 0 with probability \(1-p\). In a full multiple-imputation procedure, uncertainty in the fitted model is also represented; a simple probability calculation with fixed coefficients illustrates only the value-drawing step.

This is different from assigning 1 whenever \(p>0.5\). That rule gives the most likely category, not a random draw from the modeled conditional distribution. Repeatedly using the most likely category tends to make imputed data too uniform and can weaken relationships between variables.

Worked Example: Imputing a Missing Binary Predictor

Suppose a hypothetical analysis has a binary predictor \(X\), indicating whether a participant attended a follow-up visit (1 = yes, 0 = no). For illustration, suppose the fitted logistic model is \(\operatorname{logit}\{P(X=1\mid A)\}=-1+0.04A\), where \(A\) is age in years. One participant is 30 years old and has a missing value of \(X\). Treat the displayed coefficients as fixed for this example.

Find the conditional probability. At \(A=30\), the linear predictor is \(\eta=-1+0.04(30)=0.20\). Therefore,

p=P(X=1\mid A=30) =\frac{\exp(0.20)}{1+\exp(0.20)} =\frac{1.2214}{2.2214} \approx 0.5498.

Draw the imputed value. Draw \(U\) uniformly from 0 to 1. If \(U<0.5498\), impute 1; otherwise, impute 0. For a hypothetical draw \(U=0.63\), the imputed value is 0, meaning “did not attend.” A different draw could produce 1, consistent with the estimated probability of about 0.55.

This example shows how the model prevents fractional visit indicators while retaining uncertainty. In an actual multiple-imputation analysis, the model would be fitted using the available data, its uncertainty would be incorporated, and the procedure would be repeated to create multiple completed data sets.

Unordered Categories: Multinomial Logistic Regression

A nominal variable has categories without a natural order, such as car, bus, and bicycle as travel modes. It should not be imputed by treating category labels as numerical measurements. A standard option is multinomial logistic regression. Choose one category as the reference; for each other category \(j\), model its log odds relative to that reference category.

\log\left(\frac{P(X=j\mid Z)}{P(X=\text{reference}\mid Z)}\right) =\eta_j,\qquad P(X=j\mid Z)=\frac{\exp(\eta_j)} {1+\sum_{\ell\ne\text{reference}}\exp(\eta_\ell)}.

The reference category has linear predictor 0, so its numerator in the probability calculation is 1. The resulting probabilities are nonnegative and sum to 1. Impute by drawing one category according to those probabilities, rather than always choosing the category with the largest probability.

Worked Example: Imputing an Unordered Travel-Mode Variable

Suppose a hypothetical data set has a missing travel-mode value with categories car, bus, and bicycle. Take car as the reference category. At the observed predictor values \(Z\), suppose the fitted log odds are \(\eta_{\text{bus}}=0.4\) and \(\eta_{\text{bicycle}}=-0.3\). These are illustrative model predictions, not estimates from a real study.

The unnormalized category weights are 1 for car, \(\exp(0.4)\) for bus, and \(\exp(-0.3)\) for bicycle. Their sum is \(1+\exp(0.4)+\exp(-0.3)\approx 3.2326429\). Dividing each weight by this denominator gives:

\begin{aligned} P(\text{car}\mid Z)&=\frac{1}{3.2326429}\approx 0.3093,\\ P(\text{bus}\mid Z)&=\frac{\exp(0.4)}{3.2326429}\approx 0.4615,\\ P(\text{bicycle}\mid Z)&=\frac{\exp(-0.3)}{3.2326429}\approx 0.2292. \end{aligned}

The probabilities sum to approximately 1. A single draw \(U\) can select the category using cumulative intervals: car from 0 to 0.3093, bus from 0.3093 to 0.7708, and bicycle from 0.7708 to 1, allowing for rounding. For example, \(U=0.70\) selects bus. The category is a valid mode, and the chance of selecting each mode reflects its modeled probability.

If categories have a meaningful order, such as mild, moderate, and severe, an ordinal model may be more suitable than a nominal model because it can use that ordering. But an ordinal model should not be imposed on categories that have no defensible order. The model’s assumptions should match the measurement scale, not just the way values happen to be coded.

Bounded Continuous Variables: Model Within the Range

For a continuous variable known to lie between lower bound \(L\) and upper bound \(U\), an ordinary regression imputation can predict values outside that interval. For example, a predicted mean near the upper boundary plus a positive residual draw can exceed the maximum. Such a result is not made valid merely because the regression fit is otherwise reasonable.

One option is a truncated predictive distribution: specify a distribution for the variable conditional on the observed information, then draw from that distribution restricted to \(L\leq X\leq U\). Another option, when values must be strictly between the bounds, is to transform them to an unbounded scale, model the transformed values, and transform the imputations back. Neither approach is universally best; the choice depends on how the variable is generated and whether exact endpoints can occur.

Key distinction: Truncation draws from a distribution conditional on the value being within specified bounds. Clipping first draws from an unrestricted distribution and then replaces an out-of-range draw by a boundary value. Clipping can create artificial piles at the bounds, so it is generally not equivalent to a correctly specified truncated model.

For a truncated normal model with conditional mean \(\mu\), standard deviation \(\sigma\), and bounds \(L\) and \(U\), define \(F(x)=\Phi((x-\mu)/\sigma)\), where \(\Phi\) is the standard normal cumulative distribution function. If \(V\) is uniform on 0 to 1, a draw from the normal distribution restricted to the interval can be generated as

X=\mu+\sigma\Phi^{-1}\left[ F(L)+V\{F(U)-F(L)\} \right].

The expression inside \(\Phi^{-1}\) lies between \(F(L)\) and \(F(U)\). Consequently, applying the inverse normal distribution yields a value between \(L\) and \(U\). This gives a direct justification for why the draw respects the bounds.

Worked Example: Drawing a Bounded Continuous Imputation

Suppose a hypothetical bounded measurement \(X\) must lie from 0 to 10 units. For a record with a missing measurement, assume the imputation model specifies a normal distribution with conditional mean \(\mu=9.5\) units and standard deviation \(\sigma=1.0\) unit, restricted to the allowed interval \([0,10]\). Use a hypothetical uniform draw \(V=0.80\).

The standardized bounds are \((0-9.5)/1=-9.5\) and \((10-9.5)/1=0.5\). Thus \(F(0)=\Phi(-9.5)\), which is effectively 0 at the precision needed here, and \(F(10)=\Phi(0.5)\approx0.6915\). The target probability for the inverse normal calculation is

\Phi(-9.5)+0.80\{\Phi(0.5)-\Phi(-9.5)\} \approx 0+0.80(0.6915)=0.5532.

The standard normal quantile at 0.5532 is approximately 0.134. Therefore the imputed value is \(X=9.5+1.0(0.134)\approx9.63\) units, which is within the allowed range. The corresponding unrestricted normal draw could exceed 10; the truncated draw instead samples from the model conditional on being within the bounds.

This model is appropriate only if a truncated normal distribution is a reasonable description of the measurement. If genuine observations can equal exactly 0 or 10 because there is meaningful probability mass at an endpoint, a continuous truncated normal alone cannot represent that feature. A model that explicitly accommodates endpoint values, such as a boundary-inflated model, may then be needed.

Choosing and Applying the Model

The imputation model should reflect the variable’s form as well as its relationships with other variables. Logistic regression is a common choice for binary variables; multinomial logistic regression is common for unordered categories; and bounded continuous variables may call for a suitable truncated or transformed model. The variables used as predictors in these models should be selected in light of the analysis and missing-data structure. Including relevant observed information helps the imputation model represent conditional relationships, but cannot compensate for a badly chosen distribution.

A logit transformation is one option for a variable strictly between \(L\) and \(U\). First scale it to \(Q=(X-L)/(U-L)\), so \(0<Q<1\); then transform using \(\log\{Q/(1-Q)\}\). After modeling and drawing on the transformed scale, apply the inverse transformation and rescale to return to the original range. Because the transformation is undefined at \(Q=0\) and \(Q=1\), this direct approach requires special treatment if the data include exact endpoints.

The key decisions can be summarized as follows:

Variable type	Possible model choice	What the imputation must respect
Binary	Logistic regression	Only values 0 and 1
Unordered categories	Multinomial logistic regression	Only listed categories, with probabilities summing to 1
Bounded continuous, no endpoint mass	Truncated model or suitable transformation	The allowable interval and a plausible conditional distribution
Bounded continuous with endpoint mass	Model that accommodates endpoints	The interval and the possibility of exact boundary values

Common Mistakes and Why They Matter

Using ordinary linear regression for a categorical value. A prediction such as 0.37 is not a possible binary observation. Model the probability and draw a category instead.
Always imputing the most likely category. The modal category may be possible, but repeating it ignores the other modeled probabilities and understates variation in completed data.
Treating category codes as measurements. Coding car as 1, bus as 2, and bicycle as 3 does not make the values quantitative or ordered. The numerical distance between those codes has no inherent meaning.
Clipping every out-of-range draw to a boundary. Replacing all values above the maximum by that maximum can create an artificial concentration of imputations at the maximum. Use a model whose draws respect the range.
Assuming valid support guarantees valid inference. A value can be possible yet implausible, and a support-respecting model can still miss important conditional relationships or rely on untenable missingness assumptions.

The practical aim is not simply to prevent impossible values. It is to represent uncertainty and conditional variation in a way that makes sense for the variable. When reviewing an imputation procedure, ask both “Could this value occur?” and “Does the model generate plausible values for records like this one?”

Check Your Understanding

Answer each question by identifying the variable’s structure and the implication for its imputation model.

A missing variable records whether a patient attended an appointment. What model could be used, and why is a predicted value of 0.4 not itself an imputed appointment status?
A nominal variable has categories red, green, and blue. Explain why choosing the category with the largest fitted probability for every missing record differs from drawing a category using the probabilities.
For the travel-mode example, which category is selected by a uniform draw of 0.25? Which is selected by a draw of 0.90?
A continuous score is defined between 0 and 100, but exact scores of 0 and 100 occur. Why might a transformed model that requires strictly interior values be insufficient without additional treatment?
Explain why clipping an unrestricted imputation at the upper bound can produce a different distribution from drawing from a distribution truncated at that bound.