Why Strategy Choice Depends on the Question
Missing values do not dictate a single automatic response. A strategy that is reasonable for a regression with a missing outcome may be unsuitable when a predictor is missing, and a method that estimates a relationship under one assumption may not answer a different analysis goal. The first decision is therefore not “Which procedure is easiest?” but “What quantity do we want to estimate, for which population, and what assumptions can we defend?”
The missingness pattern, the roles of the affected variables, and the information available in the observed data all help guide that decision. None of them, by itself, proves why values are missing. MCAR means missingness is unrelated to observed and unobserved data; MAR means it may depend on observed data but not on the missing value after conditioning on those data; MNAR means it may still depend on the missing value after conditioning on observed data. These are assumptions about the process, not labels that a pattern table can establish with certainty.
Start With the Target and the Variable Roles
State the analysis goal in concrete terms. Is the aim to estimate a conditional association between an outcome and predictors in the original study population? To describe the relationship among people with complete records? To make predictions for a population where some predictors will be unavailable? These goals can call for different decisions. In particular, the population represented by the complete records is not necessarily the population the analysis was intended to describe.
Next, identify which variables are incomplete and what role each plays. Missing outcomes often leave the predictor values available, so the central question is whether the observed outcomes represent the intended conditional relationship. Missing predictors are more complicated for a regression that needs those predictors: simply dropping every incomplete record may discard substantial information, while imputation requires a defensible model for the missing predictor values. If both outcomes and predictors are missing, the analysis must account for their roles together.
Available information matters too. Recorded variables associated with missingness may make an assumption such as MAR more plausible when they are included appropriately in the analysis or imputation model. But finding such variables does not prove MAR. If missingness may depend on the unseen value itself, assumptions beyond the observed data cannot generally be checked from those data alone. That is a strong reason to assess how conclusions depend on alternative assumptions.
| Strategy | When it may be suitable | What must be considered |
|---|---|---|
| Complete-case analysis | Few records are incomplete, or the regression target remains estimable from complete records under defensible conditions. | Who remains, whether the target relationship is preserved, and whether enough complete records cover the relevant predictor values. |
| Imputation | Incomplete records contain useful observed information and the missing values can be modeled plausibly from variables that are available. | Whether the imputation model represents the data and analysis adequately, and whether its assumptions are credible. |
| Sensitivity analysis | Key assumptions are uncertain, especially when departures from MAR are plausible or consequential. | Which alternative assumptions are plausible and whether they change the substantive conclusion. |
These choices are not always mutually exclusive. An analysis may use imputation as its primary strategy and also compare its conclusions with a complete-case analysis or other sensitivity checks. The comparison is informative only when its assumptions and target are made clear.
Why Complete Cases Can or Cannot Represent the Target
A short justification helps explain why complete-case analysis is not automatically biased, but also why it is not automatically safe. Consider a regression with a fully observed predictor vector \(X\), including an intercept, and an outcome \(Y\). Let \(R=1\) indicate that the outcome is observed and \(R=0\) that it is missing. Suppose the intended conditional regression is correctly represented by \(Y=X^\mathsf{T}\beta+\varepsilon\), with \(E(\varepsilon\mid X)=0\).
The target coefficient satisfies the population regression equation \(E[X(Y-X^\mathsf{T}\beta)]=0\). Complete-case regression instead uses records for which \(R=1\), so its corresponding equation is \(E[X(Y-X^\mathsf{T}\beta)\mid R=1]=0\). If response depends only on observed \(X\), then, under the stated model, the conditional mean residual remains zero among responders. The complete-case equation is therefore also satisfied by the same \(\beta\), provided response has positive probability across the predictor values needed for the target.
This is a sufficient justification for estimating the same conditional relationship when the predictors are observed and the outcome response process is adequately explained by them. It is not a universal rule for every regression. Missing predictors, a misspecified model, a different target such as a population-average projection, or response depending on unobserved outcome information can change the argument. The precise conditions depend on the analysis and on which variables are missing.
A Practical Decision Sequence
Name the population and the regression quantity to be estimated. Do not silently replace the original study population with the complete-record population.
Use the pattern of incomplete records to see which variables and combinations are affected, and identify whether the outcome, predictors, or both are missing.
Consider whether complete cases can estimate the target under defensible conditions. If not, ask whether the available variables can support a plausible imputation model.
If plausible alternatives about the missing values could materially affect the answer, plan sensitivity analyses and report what conclusions remain stable.
Worked Example: Missing Outcomes With Observed Predictors
Suppose a hypothetical follow-up study enrolled 240 adults. The regression outcome is a later health score, and all planned predictors, including age, are observed. The outcome is recorded for 72 of 80 younger adults and 120 of 160 older adults. Should a complete-case regression be considered?
The number of complete cases is \(72+120=192\), so the number with missing outcomes is \(240-192=48\). The response rates are
Response differs by age group, so the records are not missing completely at random with respect to this recorded grouping. That difference alone does not rule out complete-case regression. If age and the other predictors adequately account for outcome response, the conditional mean model is correctly specified, and the chance of observing the outcome is positive across relevant predictor values, complete cases may estimate the intended conditional regression relationship.
The decision is therefore conditional, not automatic: complete-case analysis is a defensible candidate if those conditions are plausible. The observed response rates do not prove that response depends only on recorded predictors, so the analyst should examine available information and acknowledge uncertainty. If outcome response also depends on unobserved health scores after accounting for predictors, the justification does not hold as stated.
Worked Example: A Predictor Is Missing in Some Records
In a hypothetical study of 300 patients, the outcome and other predictors are recorded, but a laboratory predictor is missing for 60 patients. It is missing for 30 of 100 patients at Clinic A and 30 of 200 patients at Clinic B. Age, clinic, and the outcome are observed for all 300 patients. Which strategy is worth considering?
The overall missing fraction for the laboratory predictor is
or 20%. The clinic-specific missing fractions are \(30/100=0.30\), or 30%, at Clinic A and \(30/200=0.15\), or 15%, at Clinic B. Clinic is therefore relevant information about which laboratory values are missing. Because the missing variable is a predictor needed for the regression, using only complete records would exclude 60 patients, including a disproportionate share from Clinic A.
Imputation is a reasonable candidate if a model using the observed information can plausibly represent the missing laboratory values for both clinics and the intended regression target. The available outcome may also carry relevant information for that modeling decision. However, observing a clinic difference does not establish that missingness is MAR, and a 20% missing fraction does not by itself determine whether imputation is valid. The analyst must judge the assumptions and whether patients with observed laboratory values provide adequate information for those with missing values. A complete-case result can also be useful as a comparison, but it does not automatically answer the original target question.
Worked Example: When Sensitivity Analysis Is Central
A hypothetical study has 200 participants with a recorded predictor and a follow-up outcome. The outcome is observed for 40 of 80 participants recruited remotely and for 110 of 120 recruited in person. Investigators also consider it plausible that participants with more severe unrecorded symptoms were less likely to complete follow-up, even after accounting for recruitment method. What should guide the strategy?
There are \(40+110=150\) observed outcomes and \(200-150=50\) missing outcomes. The missing fractions are \(40/80=0.50\), or 50%, for remote recruitment and \(10/120\approx0.0833\), or 8.33%, for in-person recruitment. These differences show that recruitment method is associated with outcome availability in this hypothetical data set.
If recruitment method and other observed variables adequately explained missingness, a MAR-based strategy might be considered. But the additional concern is that unrecorded severity may influence follow-up even after those observed variables are accounted for. The observed data cannot establish whether that concern is correct. A primary analysis under a stated assumption may still be useful, but it should be accompanied by sensitivity analysis that asks whether plausible departures from that assumption change the regression conclusion. The goal is not to declare the missing outcomes recovered; it is to show how dependent the answer is on what is assumed about them.
Common Mistakes and Why the Choice Matters
- Choosing by the percentage missing alone. A small fraction of missing data can still matter if it is concentrated in an important subgroup or related to the regression outcome. Conversely, a larger fraction does not, by itself, prove that complete-case analysis is unusable.
- Treating a visible pattern as proof of MAR. Recorded variables can help explain missingness, but they cannot reveal whether missingness also depends on the unseen values. State MAR as an assumption when relying on it.
- Calling imputation assumption-free. Imputation uses models and available information; it does not automatically remove bias. Its suitability depends on whether those models and assumptions are adequate for the analysis.
- Using sensitivity analysis as a substitute for a primary analysis. Sensitivity analysis assesses how conclusions change across assumptions. It should be connected to a clearly stated primary analysis and a clearly defined target.
- Assuming complete cases represent everyone enrolled. Complete cases are the records available for a particular analysis. Their composition and their relationship to the target must be considered.
A missing-data strategy affects both who contributes information and what assumptions support the resulting regression. Selecting it deliberately makes the analysis easier to interpret: readers can see the intended target, the role of incomplete variables, and how much confidence to place in conclusions that rely on untestable assumptions.
Check Your Understanding
For each question, explain how the analysis goal and missingness assumptions affect the decision.
- Why is the percentage of missing values not enough to determine whether complete-case analysis is appropriate?
- A regression outcome is missing for some records, but all predictors are observed. What condition on outcome response could support using complete cases for a conditional regression relationship?
- Why might missingness in a predictor make imputation worth considering, and what does that choice still require?
- What can a difference in missingness rates across two observed groups show, and what can it not prove about the missingness mechanism?
- How does sensitivity analysis contribute when missingness may depend on an unobserved value?