Why Use Chained Equations?
A regression data set may have missing values in several variables, and those variables may have different types. A continuous measurement, a yes-or-no indicator, and a multi-category variable should not all be imputed by the same model. Chained equations provide a practical way to model each incomplete variable using a conditional model suited to its type, while using the other available variables to inform its missing values.
The method builds on the idea of multiple imputation as repeated plausible completion: instead of making one fixed replacement for each missing value, it creates multiple completed data sets with variation that reflects uncertainty. Its distinctive feature is that it specifies a separate conditional model for each variable with missing values, then cycles through those models, updating the missing cells in turn.
Suppose a data set contains an outcome and several predictors. Some predictor values are missing, and perhaps some outcomes are missing too. For each incomplete variable, its conditional model treats that variable as the response and uses the other relevant variables as predictors. The response model is chosen to reflect the variable's form: for example, linear regression for a continuous variable, logistic regression for a binary variable, and multinomial logistic regression for an unordered categorical variable.
The Models and Assumptions Behind an Update
Let \(X_1,\ldots,X_q\) denote variables that have at least some missing values, and let \(Z\) denote any other fully observed variables being used. For variable \(X_j\), a chained-equations model specifies a conditional distribution of \(X_j\) given the other variables, written \(P(X_j\mid X_{-j},Z)\), where \(X_{-j}\) means all the variables in \(X_1,\ldots,X_q\) except \(X_j\). A conditional model may be a regression model or another model suited to the response's type.
The key modeling assumption is that each conditional model adequately represents the distribution needed to draw plausible values for its missing cells. The imputation procedure also relies on assumptions about the missingness process. In particular, under a missing at random (MAR) assumption, missingness can depend on observed data included in the analysis but not on the missing value itself after conditioning on those observed data. Chained equations do not establish MAR, and they do not automatically correct for missing not at random (MNAR) mechanisms.
Observed values are retained; only missing cells are updated. When the method imputes a continuous variable, simply inserting a fitted mean is not enough: values need stochastic variation, including residual variation and uncertainty in the fitted model. For a binary or categorical variable, the procedure draws a category from estimated probabilities rather than assigning whichever category has the largest probability every time.
| Incomplete variable | Common conditional model | Type of imputation draw |
|---|---|---|
| Continuous | Linear regression | A numerical draw that includes residual and model uncertainty |
| Binary | Logistic regression | A Bernoulli draw using the estimated probability of one category |
| Unordered categorical | Multinomial logistic regression | A draw from the estimated probabilities of all categories |
| Ordered categorical | An ordinal regression model, when appropriate | A draw from the estimated category probabilities |
These are common choices, not rules that every data set must follow. The model should match the variable's scale and account for the structure of the data well enough to produce plausible conditional draws. Chained equations do not make an unsuitable model suitable merely by repeating it.
Worked Example: Imputing a Continuous Predictor
In a hypothetical data set, a patient's age is missing, while a continuous health score and an indicator for a particular exposure are observed. Suppose the conditional linear model for age, in years, is \( \text{age}=30+0.4(\text{health score})+5(\text{exposure indicator})+\varepsilon \), with residual standard deviation \(6\) years. For one missing age, the health score is \(50\) and the exposure indicator is \(1\). For illustration, suppose a model-uncertainty draw produces coefficients equal to the displayed values, and the predictive residual draw is \(-3\) years.
The fitted conditional mean is \(30+0.4(50)+5(1)=30+20+5=55\) years.
The imputed value for this illustrative draw is the fitted mean plus the residual draw: \(55+(-3)=52\) years. This is one plausible stochastic imputation, not a claim that the missing age is known to be 52. Another draw could produce a different age, reflecting uncertainty and residual variation. The numerical example assumes the linear model is suitable for these data and that the other variables used in it are available for this record.
Worked Example: Imputing a Binary Variable
Suppose a hypothetical binary predictor records whether a patient uses a particular medicine, coded \(1\) for yes and \(0\) for no. A logistic conditional model gives a probability of use based on other variables. For a record with predictor values that yield a linear predictor of \(-0.4\), the estimated probability of use is \(p=1/(1+\exp(-(-0.4)))=1/(1+\exp(0.4))\approx0.4013\).
For one imputation draw, generate a uniform random number \(U\) between 0 and 1. If \(U<p\), impute 1; otherwise, impute 0. If the illustrative draw is \(U=0.32\), then \(0.32<0.4013\), so this draw imputes 1 (use). If instead \(U=0.70\), the draw imputes 0. Both outcomes are possible under the model, with probability about \(0.4013\) assigned to 1.
The probability is not itself the imputed value: filling this cell with \(0.4013\) would create a value that is neither of the variable's two categories. The example assumes the logistic model is an adequate conditional model and that its predictor values are observed or currently available in the chain.
Worked Example: Imputing an Unordered Categorical Variable
Consider a hypothetical three-category variable describing a patient's transport to a clinic: walk, bus, or car. A multinomial logistic model, conditional on the other variables, gives probabilities of \(0.25\) for walk, \(0.50\) for bus, and \(0.25\) for car for one record. These probabilities sum to \(1\).
To make a categorical draw, use intervals whose lengths equal the probabilities: walk if \(0\leq U<0.25\), bus if \(0.25\leq U<0.75\), and car if \(0.75\leq U\leq1\). If the illustrative uniform draw is \(U=0.68\), it falls in the bus interval, so bus is imputed. A separate draw of \(U=0.81\) would impute car.
This procedure respects the categories while allowing uncertainty about which one belongs in the missing cell. It assumes that the categories are unordered; if their order has substantive meaning, an ordinal model may be more appropriate, provided its assumptions are reasonable.
How One Chained-Equations Cycle Works
A cycle updates the incomplete variables one at a time. At the first update, the other variables contain their observed values and any current initial imputations. After a variable is updated, its new imputations are available as predictors for the next update in the cycle. Thus, later conditional models use the most recently available values rather than leaving every variable fixed at its initial starting values.
Fill each incomplete variable's missing cells with reasonable starting values, such as simple provisional values. These are starting points, not the final imputation strategy.
Fit its conditional model using records with that variable observed, then draw new values for its missing cells. Keep its observed values unchanged.
Fit each variable's own conditional model and draw replacements for its missing cells, using the current values of the other variables.
Cycle through the conditional models repeatedly so that the current imputations can respond to updates in the other variables.
After an initial run-in period, retain completed data sets from separate stochastic runs or suitably separated iterations. Each retained data set has observed values plus one draw for each originally missing cell.
The initial run-in period, often called burn-in, gives the updates time to move away from arbitrary starting values. Stochastic model and value draws are essential: repeating deterministic predictions alone would tend to produce the same completed data and would understate uncertainty. The retained data sets are separate plausible completions, not different versions of the observed data.
Worked Example: Tracing a Short Update Cycle
Suppose a hypothetical data set has a continuous score \(S\), a binary indicator \(B\), and an unordered region variable \(R\), with missing cells in all three. For one record, \(S\) and \(R\) are observed but \(B\) is missing. For another record, \(B\) and \(R\) are observed but \(S\) is missing. For a third, \(S\) and \(B\) are observed but \(R\) is missing. Start with provisional initial values in all missing cells.
First, update missing \(S\) values using the continuous conditional model for \(S\), with current \(B\) and \(R\) values among its predictors. For the second record, suppose the model's fitted mean is \(72\) score points and the illustrative predictive residual draw is \(4\) points. The updated score is \(72+4=76\).
Next, update missing \(B\) values using its binary conditional model with the newly updated \(S\) and current \(R\). For the first record, suppose the estimated probability of \(B=1\) is \(0.60\), and the uniform draw is \(0.45\). Since \(0.45<0.60\), the updated value is \(B=1\).
Finally, update missing \(R\) values using its categorical conditional model and the current \(S\) and \(B\). Suppose the probabilities for the third record are \(0.20\) for North, \(0.50\) for Central, and \(0.30\) for South. A uniform draw of \(0.62\) falls between \(0.20\) and \(0.70\), so the updated region is Central. One cycle is complete. A later cycle uses these updated cells as current values, and further cycles produce the completed data sets retained for analysis.
This trace assumes the stated conditional models are suitable and that each record has the predictor values needed for its update. It illustrates the order of updates, not a guarantee that the imputations have stabilized after one cycle.
A Short Justification for Stochastic Updates
Why not fill each missing cell with its conditional mean or most likely category? Consider a binary cell with estimated probability \(p\) of being 1. A fixed probability replacement inserts \(p\), which is not a possible binary value unless \(p\) is 0 or 1. A stochastic draw instead takes value 1 with probability \(p\) and 0 with probability \(1-p\). Its expected value is
Thus repeated stochastic draws represent both the estimated probability and the fact that the missing category is uncertain. For continuous values, drawing a predictive value rather than substituting only a fitted conditional mean similarly retains residual variation. These draws do not make the models correct, but they avoid treating fitted predictions as if they were observed values known without uncertainty.
What the Method Does and Does Not Guarantee
Chained equations are useful when incomplete variables have mixed types because each variable can have a conditional model suited to its scale. They also provide a flexible computational recipe: specify conditional models, update missing cells in turn, and repeat. However, specifying several conditional distributions separately does not by itself guarantee that they correspond to one coherent joint probability distribution. Nor does the act of iterating prove that the resulting imputations are adequate for the scientific question.
The method depends on choices about model form and on assumptions about the missingness mechanism. It can generate implausible results if a model produces values outside the variable's meaningful range or categories, or if the conditional relationships are badly represented. Repetition alone cannot repair such problems. In practice, the analyst needs to examine whether the imputed values and the resulting analyses are sensible, while keeping the assumptions explicit.
Common Mistakes
- Using one model for every variable. A linear regression model can predict values below 0 or above 1 for a binary variable. Match the conditional model to the response type.
- Replacing draws with fitted predictions. A conditional mean or most likely category is not a stochastic imputation. Draw plausible values so that uncertainty and residual variation are represented.
- Updating observed cells. In ordinary imputation, observed values remain as recorded; the updates replace missing cells only.
- Assuming iterations remove bias. Iteration updates the imputations under the specified models. It does not establish MAR, fix a misspecified model, or solve an MNAR problem.
- Calling one completed data set multiple imputation. Multiple imputation retains several completed data sets, each reflecting a separate set of stochastic draws.
Check Your Understanding
Try these questions without looking back.
- Which conditional model would commonly be used for a missing continuous measurement, and why should its imputation include more than the fitted mean?
- A binary logistic model gives a probability of \(0.30\) for category 1. Describe how one stochastic imputation draw is made.
- In what order are variables updated during one chained-equations cycle, and what values can a later update use?
- Why is it inappropriate to fill an unordered categorical variable with its most probable category every time?
- Does repeating chained-equations updates establish that missingness is MAR? Explain.