Why Add an Indicator?
A predictor may be missing for some records even when the outcome is available. A tempting shortcut is to fill in the missing predictor values with a constant, add a variable marking which values were missing, and fit a regression using both. The indicator can distinguish records with recorded values from records that were filled in, but that distinction does not reveal the missing values themselves. As a result, it does not generally repair bias in the estimated relationship between the predictor and outcome.
The key is to ask what information the added indicator actually contains. It identifies a missingness group. If missingness is related to the outcome or to the unobserved predictor, that group may differ systematically from the group with observed predictor values. A group label alone does not account for those differences in the way the missing predictor would.
In a simple linear regression, this construction gives the model \(Y=\beta_0+\beta_1X^*+\gamma M+\varepsilon\). The coefficient \(\beta_1\) is the fitted slope for records with observed \(X\); \(\gamma\) allows the missing-\(X\) group to have a different fitted level. This is a description of the fitted model, not a guarantee that \(\beta_1\) estimates the slope that would have been obtained if all \(X\) values were observed.
The central limitation is that, among records with \(M=1\), every filled-in \(X^*\) equals \(c\). Those records provide no variation in \(X^*\) that could tell the regression how their outcomes change as their true \(X\) changes. The indicator can shift their fitted level, but it cannot supply the missing within-group predictor information.
What the Fitted Model Can Learn
For records with \(M=0\), the model reduces to \(Y=\beta_0+\beta_1X+\varepsilon\). For records with \(M=1\), it reduces to \(Y=\beta_0+\beta_1c+\gamma+\varepsilon\). The second expression gives every missing-\(X\) record the same fitted value, apart from any other predictors in the model. In particular, it does not use the actual, unobserved \(X\) value of any such record.
The indicator coefficient may capture a difference in average outcome between records with missing and observed predictor values, conditional on the model's other terms. That difference can be useful for prediction or description. But it is not automatically an adjustment for the missing values, nor does it show why they are missing. If the missingness pattern is associated with \(X\) or \(Y\), the observed records may represent a selected part of the data.
Why the Missing Group Does Not Identify the Slope
The limitation can be seen directly from least squares. Suppose the only predictors are \(X^*\) and \(M\), with an intercept. For fixed values of \(\beta_0\) and \(\beta_1\), the fitted value for every record with \(M=1\) is \(\beta_0+\beta_1c+\gamma\). The coefficient \(\gamma\) can move that common fitted value to the mean outcome in the missing group. After that adjustment, the missing group's contribution to the residual sum of squares is its variation around its own mean; it does not depend on \(\beta_1\).
Choosing \(\gamma=\bar{Y}_{M=1}-\beta_0-\beta_1c\) makes the fitted value for that group equal to its mean outcome, where \(\bar{Y}_{M=1}\) is the average outcome among records with missing \(X\). The resulting residual sum of squares for that group is
This expression has no \(\beta_1\) in it. In this simple model, the slope is therefore determined by the observed-\(X\) records, while the indicator lets the missing group have its own fitted level. If the observed records do not contain enough variation in \(X\), the slope may be unidentified or estimated imprecisely. With additional predictors or interactions, the details of the fit can change, but the indicator still does not supply the missing \(X\) values.
Worked Example: When the Slope Cannot Be Learned
Consider four hypothetical records with outcomes \(Y=2,2,5,5\). The predictor values are \(X=0,0,1,1\), but the two values of 1 are missing. Fill missing values with \(c=0\) and let \(M=1\) mark a missing value.
The regression inputs are then \((X^*,M)=(0,0)\) for the first two records and \((0,1)\) for the last two. Since \(X^*=0\) for every record, it has no variation beyond the intercept. The fitted model reduces to \(\hat{Y}=a+gM\). Least squares gives \(a=2\) for the observed group and \(a+g=5\) for the missing group, so \(g=3\). The coefficient of \(X^*\), however, cannot be estimated separately from the intercept: changing it does not change any fitted value.
In the hypothetical complete data, \(Y=2+3X\), so the complete-data slope is 3. The indicator fit does not recover that slope. It distinguishes the groups' outcomes, but the observed records contain no variation in \(X\) from which to estimate the relationship.
A Numerical Example of a Different Slope
Even when the observed predictor varies, the indicator model can give a slope that differs from the complete-data regression. The next example deliberately shows the complete values for teaching purposes; an analyst fitting the indicator model would not know the missing values.
Worked Example: A Group Shift Does Not Reconstruct Missing Values
Suppose the hypothetical complete data are \((X,Y)=(0,0),(1,1),(2,10),(3,10)\). The last two \(X\) values are missing. Fill them with the mean of the two observed \(X\) values, \(c=(0+1)/2=0.5\), and include \(M\).
Complete-data slope. The means are \(\bar{x}=1.5\) and \(\bar{y}=5.25\). The least-squares slope is
Indicator-model slope. Among the observed-\(X\) records, the points are \((0,0)\) and \((1,1)\), which lie on the line \(Y=X\). Thus their fitted intercept is 0 and their fitted slope is 1. The two missing-\(X\) records both have \(X^*=0.5\) and \(Y=10\). Setting \(\gamma=9.5\) gives their fitted value as \(0+1(0.5)+9.5=10\). The indicator model therefore fits all four outcomes exactly, but its coefficient of \(X^*\) is 1, not the complete-data slope of 3.9.
This comparison is not evidence that one coefficient is always the right answer and the other always the wrong one. It demonstrates that the indicator regression uses the observed records' \(X\)-to-\(Y\) pattern for its slope and uses \(\gamma\) to shift the missing group. The shift does not reconstruct how the outcomes of those records relate to their missing \(X\) values.
When an Indicator Has a Design-Based Meaning
Not every indicator associated with an unobserved predictor is an arbitrary label created after missingness occurs. In a planned measurement design, the indicator may record a known assignment or sampling rule. For example, a study might measure an expensive biomarker for a randomly selected subset of participants while recording the outcome for everyone. An indicator of whether the biomarker was scheduled to be measured then identifies a designed measurement group.
In that setting, the indicator has a genuine design-based interpretation: it records membership in a pre-specified group created by the sampling or measurement plan. If the subset is selected at random, the assignment rule can support comparisons between groups in expectation. The indicator can be useful for describing the design, defining analysis groups, or accounting for planned sampling structure. Its meaning comes from the design, not from a general property of missingness indicators.
Worked Example: Planned Random Measurement
A hypothetical study records an outcome for eight participants and randomly assigns four to have a costly biomarker measured. Let \(R=1\) denote assignment to measurement and \(R=0\) denote no measurement. Suppose the observed outcomes are 4, 5, 6, and 7 in the \(R=1\) group, and 3, 4, 6, and 7 in the \(R=0\) group.
The group means are \(\bar{Y}_{R=1}=(4+5+6+7)/4=5.5\) and \(\bar{Y}_{R=0}=(3+4+6+7)/4=5.0\). A regression of \(Y\) on an intercept and \(R\) gives an indicator coefficient of \(5.5-5.0=0.5\). Because measurement assignment was randomized, this observed difference is a difference between the randomly assigned groups; it is not, by itself, evidence that biomarker measurement caused a change in the outcome.
The indicator's design meaning is clear: it records random assignment to a measurement phase. But if the aim is to estimate the association between the biomarker and the outcome, \(R\) does not tell us the unmeasured biomarker values of the four participants with \(R=0\). Adding \(R\) to a regression after filling in those values would not, by itself, recover the biomarker-outcome relationship.
A missingness indicator can also be informative for a prediction task: if the fact that a value is missing is itself associated with future outcomes, the missingness pattern may help predict them. That is a different goal from estimating the relationship that would be found using the true predictor values for everyone. Predictive usefulness should not be presented as proof that missing-data bias has been removed.
Common Mistake
Wrong reasoning: “I filled in the missing predictor and added an indicator, so the model knows which values are artificial and the slope is unbiased.”
Correction: The model knows which records were filled in, but it still does not know their predictor values. In the basic additive model, the indicator permits a level shift for those records; it does not recover their within-group predictor variation or guarantee that the observed-\(X\) slope represents the full data. Any claim about bias requires conditions that justify the analysis, not merely the presence of an indicator.
Check Your Understanding
Try these questions without looking back.
- In the indicator method, what do \(X^*\) and \(M\) represent when \(X\) is missing?
- Why does an additive missingness indicator allow a group-level shift without revealing how outcomes vary with the missing \(X\) values?
- In the example with complete-data slope 3.9, what information determines the indicator-model slope?
- Why might a planned randomized measurement indicator have a clear design interpretation while still failing to recover unmeasured predictor values?
- Explain the difference between a missingness pattern that helps prediction and evidence that regression bias has been corrected.