Logistic Regression For Binary Imputation

Why Use Logistic Regression for a Binary Update?

In a MICE update, the variable being imputed may have only two possible values, such as Yes and No. The conditional model for that variable must respect those values: a predicted value such as 0.37 cannot itself be an imputed Yes/No response. Logistic regression provides a model for the conditional probability of one outcome, and a stochastic draw turns that probability into a plausible binary value.

Let $Y$ denote the binary variable being updated, coded $1$ for the event of interest and $0$ otherwise. Let $X_1,\ldots,X_q$ be predictors available for the cases being updated. At a particular MICE update, predictors may include other variables in the data set, using their currently available or imputed values. The model is fitted using cases for which $Y$ is observed, and its predicted probabilities are used for cases where $Y$ is missing.

Definition: A logistic regression conditional model for a binary variable specifies $$ P(Y=1\mid X_1,\ldots,X_q) = \frac{\exp(\eta)}{1+\exp(\eta)}, \qquad \eta=\beta_0+\beta_1X_1+\cdots+\beta_qX_q. $$ Equivalently, the log odds of $Y=1$ are linear in the predictors: $$ \log\left(\frac{P(Y=1\mid X_1,\ldots,X_q)} {1-P(Y=1\mid X_1,\ldots,X_q)}\right) =\eta. $$ The model assigns probability $1-P(Y=1\mid X)$ to $Y=0$.

Assumptions and Model Specification

Before calculating probabilities, define what counts as $Y=1$, identify the predictors, and specify their coding. A predictor with two categories can be represented by an indicator variable, for example $X=1$ for an intervention group and $X=0$ for a comparison group. A continuous predictor enters through a chosen numerical scale; centering it can make the intercept easier to interpret. Categorical predictors with more than two categories generally require multiple indicator variables and a stated reference category.

The standard logistic regression likelihood assumes that, conditional on the predictors, the observed binary responses are independent and follow Bernoulli distributions with probabilities given by the model. The linear-predictor assumption is also substantive: each continuous predictor is assumed to have a linear relationship with the log odds, unless the model includes transformations, polynomial terms, or other structure. The model must be estimable from the observed responses and predictor patterns. If predictors are missing, their current values in the MICE update must be available to calculate predictions.

These are assumptions about the conditional model, not a guarantee that the missingness process is adequately handled. The model also does not ensure that its conditional specification is compatible with every other model in an FCS system. Model compatibility and missing-data assumptions require separate consideration; a fitted logistic equation alone does not establish them.

Worked Example: Specifying the Conditional Model

Suppose a hypothetical data set contains a binary indicator $Y$ for attending a follow-up appointment, coded $1$ for attended and $0$ for did not attend. Predictors are age in years and assignment to an intervention, coded $1$ for intervention and $0$ for usual care. A fitted conditional model is

\log\left(\frac{P(Y=1\mid \text{age},\text{intervention})} {1-P(Y=1\mid \text{age},\text{intervention})}\right) = -1.20+0.080(\text{age}-60)+0.70(\text{intervention}).

The response coding makes the modeled event explicit: the probability is for attending, not for missing the appointment. Centering age at 60 means the intercept describes the log odds for a 60-year-old in usual care. Intervention is an indicator, so its coefficient compares intervention with usual care at the same age.

For example, a 70-year-old assigned to intervention has linear predictor $ -1.20+0.080(70-60)+0.70=0.30 $. The model therefore predicts a probability of attendance of $ \exp(0.30)/(1+\exp(0.30))=1.350/2.350=0.574 $, or about 57.4%. This is a conditional prediction under the specified model, not a claim that intervention caused attendance.

Interpreting Coefficients

A logistic coefficient is a change in log odds, not a change in probability. Exponentiating a coefficient gives a conditional odds ratio. For a one-unit increase in $X_j$, holding the other predictors fixed, the model multiplies the odds of $Y=1$ by $\exp(\beta_j)$, provided the model uses $X_j$ linearly in its predictor.

This interpretation follows directly from the model. If the other predictor values are held fixed and $X_j$ increases by one, the linear predictor changes from $\eta$ to $\eta+\beta_j$. Since the odds are $\exp(\eta)$, the ratio of the new odds to the old odds is

\frac{\exp(\eta+\beta_j)}{\exp(\eta)} =\exp(\beta_j).

An odds ratio above 1 corresponds to higher conditional odds of the event; one below 1 corresponds to lower conditional odds. Neither describes a constant change in probability. The probability change depends on the starting probability and the values of the other predictors.

Worked Example: Converting Coefficients to Conditional Odds Ratios

Use the fitted model from the previous example. The age coefficient is $0.080$ per year. Its conditional odds ratio for a one-year increase in age is $ \exp(0.080)=1.083 $. Thus, at the same intervention status and other modeled values, the odds of attendance are multiplied by about 1.083 for each additional year of age. This is an increase of about 8.3% in odds per year, not an increase of 8.3 percentage points in probability.

The intervention coefficient is $0.70$, so its conditional odds ratio is $ \exp(0.70)=2.014 $. At the same age, the fitted odds of attendance in the intervention group are about 2.01 times the fitted odds in usual care. This does not mean the probability is twice as large.

To see why the probability interpretation differs, consider a 55-year-old in usual care. The linear predictor is $ -1.20+0.080(55-60)=-1.60 $. The fitted probability is $ \exp(-1.60)/(1+\exp(-1.60))=0.2019/1.2019=0.168 $. For a 55-year-old in intervention, the linear predictor is $-0.90$, giving $ \exp(-0.90)/(1+\exp(-0.90))=0.4066/1.4066=0.289 $. Here the probability difference is about 12.1 percentage points, and the probability ratio is about 1.72, even though the odds ratio is 2.014. The odds ratio stays constant across age in this model; the probability contrast need not.

From Fitted Probabilities to Imputed Values

For a case with a missing $Y$, first calculate its linear predictor using the fitted coefficients and that case’s predictor values. Convert the result to a probability $p$ using the logistic formula. Then make a stochastic binary draw with probability $p$ of assigning 1 and probability $1-p$ of assigning 0. One convenient implementation draws $U$ uniformly between 0 and 1 and assigns 1 if $U<p$, otherwise 0. This rule is a way to generate the required binary draw; it is not a rule to classify cases at a probability threshold of 0.5.

In an actual multiple-imputation procedure, uncertainty in the model parameters should also be represented rather than treating a single set of estimated coefficients as known exactly. Software can implement an appropriate parameter-drawing procedure as part of logistic imputation. The numerical example below holds the coefficients fixed to make the conversion from probability to binary draw transparent.

Worked Example: Making Stochastic Imputations

For illustration, use the same model coefficients and suppose four hypothetical cases have missing appointment attendance. Their ages and intervention status are known. Compute each probability, then use the supplied uniform draw for that case.

Age	Intervention	Linear predictor $\eta$	Fitted probability $p$	Uniform draw $U$	Imputed $Y$
60	0	$-1.20$	0.231	0.17	1
60	1	$-0.50$	0.378	0.61	0
70	0	$-0.40$	0.401	0.39	1
70	1	$0.30$	0.574	0.88	0

For the first case, $\eta=-1.20$, so $p=\exp(-1.20)/(1+\exp(-1.20))=0.3010/1.3010=0.231$. Since $0.17<0.231$, the draw is $Y=1$. For the second case, $p=\exp(-0.50)/(1+\exp(-0.50))=0.6065/1.6065=0.378$; because $0.61$ is not below $0.378$, the imputation is $Y=0$. The third case has $p=0.401$, and $0.39<0.401$, so it receives $Y=1$. The fourth has $p=0.574$, but $0.88$ is greater than that probability, so it receives $Y=0$.

A case with a probability below 0.5 can therefore be imputed as 1, and a case with a probability above 0.5 can be imputed as 0. That variation is intentional: repeated stochastic imputations represent uncertainty about the missing binary value. Repeating the imputation can give different completed data sets, even when the predictor values are unchanged.

Diagnostics, Alternatives, and Limitations

A logistic update deserves inspection rather than automatic acceptance. Check that the response coding and reference categories match the intended interpretation; verify that all predictors used in a prediction are available; and examine whether fitted probabilities are plausible across important predictor patterns. Compare observed outcome frequencies with fitted probabilities among cases with observed $Y$, while remembering that this check cannot verify performance in predictor patterns with little or no observed data. Also look for unstable coefficient estimates, implausibly extreme fitted probabilities, and sensitivity to reasonable changes in model specification. These can signal sparse information, influential observations, or an inadequate predictor form. Specialized issues such as rare outcomes and separation require more focused methods.

The main alternatives answer different needs. Deterministic prediction assigns a fixed class or probability-based value and suppresses the variation needed for stochastic imputation. Replacing every missing value with the observed mode is even less responsive to predictors. A linear probability model is another way to model a binary outcome, but its fitted values can fall outside the interval from 0 to 1 and its error variance is not constant. Logistic regression keeps fitted probabilities within that interval and models conditional odds, but it can still be misspecified and may be unstable when the observed data provide weak information.

A frequent mistake is to impute 1 whenever $p\geq0.5$ and 0 otherwise. That creates a deterministic classification, not a stochastic imputation; it discards the uncertainty represented by intermediate probabilities and can distort the imputed distribution. The correction is to draw a binary value using the fitted probability, while also accounting for model-parameter uncertainty in the multiple-imputation procedure. Another mistake is to describe $\beta_j$ itself as a probability change. It is a log-odds change; $\exp(\beta_j)$ is the corresponding conditional odds ratio.

Key takeaway: Logistic regression specifies the conditional probability of a binary event through a linear predictor on the log-odds scale. Interpret exponentiated coefficients as conditional odds ratios, and use each case’s fitted probability to generate a stochastic binary imputation rather than a fixed threshold classification.

Check Your Understanding

Answer each question, then compare with the solutions below.

In a logistic model, what does $\exp(\beta_j)$ represent when $X_j$ increases by one unit and other predictors are held fixed?
A case has fitted probability $p=0.28$ and a uniform draw $U=0.20$. What binary value is imputed under the stated draw rule?
Why is assigning 1 whenever a fitted probability exceeds 0.5 not the same as stochastic binary imputation?
If a predictor coefficient is $0.40$, calculate its conditional odds ratio to three significant figures and interpret it.

Solutions. (1) It is the factor by which the conditional odds of $Y=1$ are multiplied for a one-unit increase in $X_j$. (2) Since $0.20<0.28$, impute $Y=1$. (3) Thresholding always assigns the same value for the same predictors and ignores the probability of the alternative value; stochastic imputation allows either outcome according to its modeled probability. (4) $\exp(0.40)=1.49$, so the conditional odds of $Y=1$ are multiplied by about 1.49 per one-unit increase in that predictor, holding the other predictors fixed.

Age	Intervention	Linear predictor \(\eta\)	Fitted probability \(p\)	Uniform draw \(U\)	Imputed \(Y\)
60	0	\(-1.20\)	0.231	0.17	1
60	1	\(-0.50\)	0.378	0.61	0
70	0	\(-0.40\)	0.401	0.39	1
70	1	\(0.30\)	0.574	0.88	0