Deriving Bernoulli Draws From Logistic Probabilities

From a Logistic Probability to a Binary Imputation

A logistic regression model supplies a conditional probability, not a completed value for a missing binary response. If a patient's recorded outcome is either 0 or 1, a fitted model might assign a probability of 0.73 to outcome 1 for that patient's predictor pattern. The imputation step must convert that probability into a value in the binary outcome's support while retaining the uncertainty represented by the probability. A Bernoulli random draw does this directly.

In a MICE update, the binary variable is modeled conditional on its predictors, which may include other variables and their current values in the chain. The model gives a probability for each row with a missing response. A new draw is made for each such row; observed responses are retained. The calculations below focus on the draw conditional on a specified logistic probability.

Definition: A binary random variable $Y$ has a Bernoulli distribution with success probability $p$, written $Y\mid x \sim \mathrm{Bernoulli}(p)$, when $Y$ takes values 0 and 1 and $$ P(Y=1\mid x)=p,\qquad P(Y=0\mid x)=1-p, $$ where $0\le p\le 1$. Its probability mass function is $$ P(Y=y\mid x)=p^y(1-p)^{1-y},\qquad y\in\{0,1\}. $$

Here, “success” is simply the conventional name for the value coded 1; it does not imply a favorable outcome. In logistic regression, a predictor pattern $x_i$ is assigned a probability $p_i$ for $Y_i=1$. For example, writing $\eta_i=x_i^\mathsf{T}\beta$, the probability is $p_i=\exp(\eta_i)/\{1+\exp(\eta_i)\}$. In an imputation update, the coefficients may be the current fitted values or a parameter draw supplied by the model-fitting step. Once those and the predictors are fixed for the update, the Bernoulli draw is made from $p_i$.

Deriving the Draw Rule

Let $U_i$ be a random number drawn from the uniform distribution on the interval from 0 to 1. Define the imputed value by comparing $U_i$ with $p_i$. Since the probability that a uniform random number falls in an interval from 0 to $p_i$ is $p_i$, the chance of assigning 1 is exactly the model probability:

Y_i^{\mathrm{imp}} = \begin{cases} 1, & U_i \le p_i,\\ 0, & U_i > p_i. \end{cases} \qquad P(Y_i^{\mathrm{imp}}=1\mid x_i)=P(U_i\le p_i)=p_i.

The remaining probability is $P(U_i>p_i)=1-p_i$, so the chance of assigning 0 is also the required Bernoulli probability. This establishes the full conditional distribution, not just its mean. The equality case has probability zero for a continuous uniform draw, so using $U_i<p_i$ instead makes no practical difference.

This is also an inverse-CDF construction. The Bernoulli cumulative distribution function jumps from $1-p_i$ at 0 to 1 at 1. Applying the inverse-CDF method to a uniform draw assigns 0 with probability $1-p_i$ and 1 with probability $p_i$. The threshold rule above uses an equivalent uniform-number convention and produces the same Bernoulli distribution.

Key result: Given a logistic probability $p_i$, draw $U_i\sim\mathrm{Uniform}(0,1)$, independently for each missing response conditional on the model and predictors, and set the imputed response to 1 when $U_i\le p_i$, otherwise to 0. Repeated draws for the same row need not match: each is a realization from the same conditional distribution.

Worked Draws for Predictor Patterns

Worked Example: Calculating Probabilities and Drawing Two Values

Suppose a hypothetical binary variable records whether a patient reports a symptom (1 = yes, 0 = no). For illustration, a logistic model has $\eta_i=-0.80+0.60x_i$, with $p_i=\exp(\eta_i)/\{1+\exp(\eta_i)\}$. Two patients have missing responses, predictor values $x_1=2$ and $x_2=0$, and the generated uniform numbers are $U_1=0.42$ and $U_2=0.64$. Calculate their probabilities and imputed values.

For patient 1, $\eta_1=-0.80+0.60(2)=0.40$. Thus $$ p_1=\frac{\exp(0.40)}{1+\exp(0.40)} =\frac{1.4918}{2.4918} \approx 0.5987. $$ Because $0.42\le 0.5987$, the imputed response is $Y_1^{\mathrm{imp}}=1$.

For patient 2, $\eta_2=-0.80+0.60(0)=-0.80$. Thus $$ p_2=\frac{\exp(-0.80)}{1+\exp(-0.80)} =\frac{0.4493}{1.4493} \approx 0.3100. $$ Because $0.64>0.3100$, the imputed response is $Y_2^{\mathrm{imp}}=0$. These are stochastic outcomes: the first patient had a 0.5987 conditional probability of a 1, not a guarantee of one.

Worked Example: Applying Probabilities to a Batch of Missing Responses

Consider four hypothetical records with missing binary responses. Suppose the fitted model gives probabilities $0.18,\ 0.47,\ 0.72,$ and $0.91$, respectively, and independent uniform draws are $0.33,\ 0.12,\ 0.83,$ and $0.90$. Find all four imputations.

Compare each draw with its corresponding probability. For record 1, $0.33>0.18$, so impute 0. For record 2, $0.12\le0.47$, so impute 1. For record 3, $0.83>0.72$, so impute 0. For record 4, $0.90\le0.91$, so impute 1. The imputed sequence is therefore $0,1,0,1$.

The expected number of ones in this batch, conditional on these probabilities, is $0.18+0.47+0.72+0.91=2.28$, but the realized number is 2. A realized batch need not contain exactly the expected number of ones; the expectation describes repeated draws under the same probabilities.

Worked Example: Same Predictor Pattern, Different Valid Draws

Two hypothetical records share the same predictor pattern, and the model assigns each $p=0.35$. Suppose their uniform draws are $U_1=0.08$ and $U_2=0.61$. The first response is imputed as 1 because $0.08\le0.35$; the second is imputed as 0 because $0.61>0.35$.

This difference is not an inconsistency. Conditional on the same predictor pattern and the same model probability, each response has a 0.35 probability of being 1 and a 0.65 probability of being 0. If instead both rows were always assigned the same value merely because their predictors match, the procedure would fail to represent the individual Bernoulli variation in the conditional model.

Conditions, Implementation Checks, and Failure Modes

The draw rule itself requires a valid probability between 0 and 1 and a uniform random number generated as specified. Its use as an imputation relies on additional modeling conditions. The outcome must truly be binary and coded consistently as 0 and 1; each predictor pattern must have the values needed to calculate its conditional probability; and the logistic conditional model should be an adequate description of the response given those predictors. For an imputation analysis to support inference under a missing-at-random assumption, the imputation model also needs relevant variables associated with missingness or the outcome and should be suitable for the substantive analysis.

Within a single update, independent uniform draws for different missing rows imply independent Bernoulli draws conditional on the probabilities. This is the usual row-wise sampling step. It does not mean the entire MICE process is independent across iterations: predictor values can change as other variables are updated, and parameter draws can also vary across updates. The draw described here is conditional on the model information for that particular update.

Several practical checks are useful:

Verify that each calculated $p_i$ is finite and between 0 and 1, and that the code compares each uniform draw with the probability from the same row.
Check that imputed values belong to $\{0,1\}$, that observed responses have not been overwritten, and that every missing response requiring an update received a draw.
For a sufficiently large group of rows with similar probabilities, compare the observed fraction of imputed ones with the average of their probabilities. They need not agree exactly in one update; large persistent discrepancies can signal a coding or random-number problem.
Inspect probabilities extremely close to 0 or 1. Such values can be legitimate, but can also signal weak overlap, sparse patterns, or model-fitting problems that require attention to the model rather than a change to the Bernoulli rule.

The final point is a diagnostic, not a reason to clip probabilities automatically. For probabilities near zero or one, a draw will usually match the more probable value, but the less probable value remains possible. Replacing all probabilities above a cutoff by 1 or below a cutoff by 0 changes the specified conditional distribution.

Why Not Insert the Probability?

A tempting shortcut is to fill a missing binary response with its predicted probability. That produces a value such as 0.35, which is not a possible value of a binary variable. More subtly, treating 0.35 as if it were a completed observation suppresses the Bernoulli variation: every such row contributes a fractional value rather than a sampled 0 or 1. Deterministic classification, such as assigning 1 whenever $p_i>0.5$, is also different. It discards the model's uncertainty and systematically turns probabilities into fixed decisions.

A Bernoulli draw is the appropriate row-level sampling step when the imputation model specifies $P(Y_i=1\mid x_i)=p_i$. It is not, by itself, a complete account of uncertainty in the fitted model. If model parameters are uncertain, a proper multiple-imputation procedure must also represent that uncertainty through its model-fitting step; repeatedly drawing outcomes from one permanently fixed estimate does not substitute for parameter uncertainty. Nor can a stochastic draw repair a misspecified conditional model, omitted predictors, or a failure of the assumptions needed for valid imputation.

Common pitfall: “The predicted probability is 0.35, so the imputed value should be 0.35,” or “because 0.35 is below 0.5, the imputed value must be 0.” The correction is to use a Bernoulli draw: generate $U\sim\mathrm{Uniform}(0,1)$, assign 1 if $U\le0.35$, and assign 0 otherwise. The result is binary and has probability 0.35 of being 1 over repeated draws.

Check Your Understanding

For each question, first identify the probability governing the draw. Solutions are provided so the calculation and interpretation can be checked.

A missing response has $p=0.62$ and a generated uniform value of 0.71. What is the imputation, and why?
For a group of missing responses, the conditional probabilities are 0.20 and 0.80. What is the expected number of imputed ones? Must a realized draw contain exactly that number?
For $p=0.40$, explain why inserting 0.40 and deterministically assigning 0 are both different from a Bernoulli imputation.
Two records have the same predictor values and each has probability 0.25. Can their imputed values differ in a valid update? Explain.

Solutions. (1) Impute 0 because $0.71>0.62$; the uniform draw did not fall in the interval that produces a 1. (2) The expected number is $0.20+0.80=1.00$. A realized draw need not contain exactly one 1, because an expectation is an average over repeated draws, not a required batch total. (3) The value 0.40 is not in the binary support $\{0,1\}$, while deterministic assignment of 0 removes the 0.40 probability of a 1. A Bernoulli draw returns 1 with probability 0.40 and 0 with probability 0.60. (4) Yes. Independent uniforms can fall on opposite sides of 0.25, so the resulting values can differ even though the conditional probabilities match.