Why Audit Missing Covariates Before Model Building?
A covariate model can only be interpreted sensibly if its input data are understood. If baseline renal function is missing for many participants, for example, a selection procedure may use fewer people than expected, or the people retained may differ systematically from those excluded. Either way, the resulting covariate analysis may answer a different question from the one intended. An audit makes the amount, overlap, and plausible causes of missingness visible before candidate effects are assessed.
For context, write a typical NLME response model as \(y_{ij}=f(t_{ij},x_i,\eta_i;\theta)+\epsilon_{ij}\), where \(y_{ij}\) is the response for individual \(i\) at observation \(j\), \(t_{ij}\) is time, \(x_i\) is a vector of covariates, \(\eta_i\) is an individual random effect, \(\theta\) contains population parameters, and \(\epsilon_{ij}\) is residual error. A missing value in \(x_i\) is not the same as a missing response \(y_{ij}\): the response model is usually conditional on covariates, so an absent covariate cannot simply be treated as an absent response observation.
Describe the Missingness Before Choosing a Strategy
Start with counts and proportions for each candidate covariate, then examine how missing values overlap. Also summarize missingness by relevant observed features such as study site, visit schedule, dose group, or response availability. These summaries describe the data; by themselves, they do not establish why values are missing.
Worked Example: Quantifying Missingness and Overlap
Suppose a hypothetical NLME data set has 120 individuals with three baseline covariates: age, renal function, and albumin. The audit finds 78 individuals with all three values observed, 12 missing age only, 20 missing renal function only, 6 missing albumin only, and 4 missing both age and renal function.
The number missing for age is \(12+4=16\), or \(16/120=0.1333\), so age is missing for 13.3% of individuals. Renal function is missing for \(20+4=24\), or 20.0%. Albumin is missing for 6, or 5.0%. The number with at least one missing covariate is \(12+20+6+4=42\), or 35.0%; consequently, \(120-42=78\), or 65.0%, are complete on these three covariates.
The overlap matters: an analysis requiring all three covariates would retain 78 individuals, not the 120 available for some other analyses. These totals still do not reveal whether missingness is random or systematic. The next checks should describe the 42 affected individuals using observed information, and investigate data-collection records or operational explanations where available.
A useful conceptual classification is based on how the probability of missingness relates to the complete data. Let \(X_{\mathrm{mis}}\) and \(X_{\mathrm{obs}}\) denote missing and observed covariate values, and let \(Y\) denote observed response information.
- Missing completely at random (MCAR): missingness is independent of the covariates and responses, observed or unobserved. In notation, the distribution of \(R\) does not depend on \(X_{\mathrm{mis}}, X_{\mathrm{obs}}\), or \(Y\).
- Missing at random (MAR): conditional on observed data, missingness does not depend on the missing covariate values. For example, missingness may depend on observed site or observed response history, but not on the unobserved covariate after conditioning on those data.
- Missing not at random (MNAR): even after conditioning on observed data, missingness depends on the value that is missing. For example, very low renal function might be less likely to be recorded for a reason not captured by observed variables.
These are assumptions about a data-generating and recording process, not labels that can generally be proven by a test of the observed data alone. A difference in missingness rates by site is evidence that the MCAR story may be implausible; it does not, by itself, distinguish MAR from MNAR. A plausible explanation from study procedures can be as important as a statistical summary.
What the Missingness Assumption Changes
An ordinary conditional NLME likelihood uses the covariates supplied for each individual. If a covariate needed by the model is missing, that likelihood does not automatically provide a value for it. A complete-case fit discards affected individuals for that analysis; an imputation approach supplies uncertain values using an additional model; an indicator approach changes the covariate parameterization. Each approach therefore adds assumptions or changes the estimand.
A useful justification for complete-case analysis under MCAR is that selection then does not change the distribution of complete data. Let \(D_i\) denote all data for individual \(i\), and let \(C_i=1\) indicate that the individual is complete for the covariates required by the model. If MCAR holds, \(P(C_i=1\mid D_i)=P(C_i=1)\). Thus, for any integrable function \(g\),
The complete cases have the same data distribution as the full sample in expectation. This supports the usual complete-case analysis under MCAR when the NLME model is otherwise correctly specified and the estimator is valid for that population. Under MAR or MNAR, the equality need not hold: complete cases can represent a selected subset, and effects or population parameters may be biased. Even under MCAR, discarding observations usually reduces information and precision.
Compare the Main Strategies
| Strategy | What it does | Main conditions and risks |
|---|---|---|
| Complete-case analysis | Fits a given model using only individuals complete on its required covariates. | Simple to implement. MCAR supports representativeness; under other mechanisms selection can bias results. Loss of individuals reduces information. |
| Imputation | Replaces missing values with draws or estimates from a model for the covariates and relevant data. | Requires a credible imputation model. Multiple imputation propagates uncertainty; a single fixed replacement generally does not. Imputation and analysis models should be compatible. |
| Missingness indicator | For a covariate, fills missing values with a chosen constant and adds an indicator for whether the value was missing. | May be a pragmatic predictive parameterization, but does not recover the unknown covariate values or generally remove missing-data bias. Coefficients require careful interpretation. |
For multiple imputation, each completed data set should reflect uncertainty about missing values, rather than inserting one supposedly correct value. The imputation model should use relevant observed covariates and response information, and should respect important features of the data, including repeated observations and between-individual structure where relevant. For an NLME application, an imputation model that ignores clustering, nonlinear relationships, or interactions required by the analysis can produce incompatible imputations. Including outcome information can be important for preserving covariate–response relationships, but the model must be specified coherently; simply imputing each covariate from its marginal mean is not an adequate substitute.
Worked Example: Complete Cases and Precision
Suppose a hypothetical covariate analysis has 100 individuals, and 25 have a missing baseline covariate. Assume, for illustration, that the missingness is MCAR and that information is approximately proportional to sample size. A complete-case fit retains \(100-25=75\) individuals.
If the standard error for a covariate effect using all 100 individuals would be 0.120 on the model's coefficient scale, the approximate complete-case standard error is \(0.120\sqrt{100/75}=0.1386\), or 0.139. The factor \(\sqrt{100/75}=1.155\) reflects the loss of information under this simplified proportional-information approximation.
A hypothetical complete-case estimate of 0.42 with standard error 0.139 is not guaranteed to equal the estimate from the full data, even under MCAR. MCAR supports equality of the underlying data distributions in expectation, not identical estimates in a finite sample. The calculation illustrates a precision cost; it does not demonstrate that MCAR holds or prove that a particular NLME model has information exactly proportional to sample size.
Worked Example: Pooling Multiple-Imputation Estimates
Suppose five hypothetical imputed data sets produce estimates of a covariate effect on log apparent clearance per 10 kg of body weight: 0.48, 0.52, 0.50, 0.46, and 0.54. Assume each fit has estimated within-imputation variance \(U_m=0.040\), corresponding to a standard error of 0.200.
The pooled estimate is the mean of the five estimates: \(\bar{Q}=(0.48+0.52+0.50+0.46+0.54)/5=2.50/5=0.500\). The between-imputation variance is \(B=[(-0.02)^2+(0.02)^2+0^2+(-0.04)^2+(0.04)^2]/(5-1)=0.0040/4=0.00100\). The mean within-imputation variance is \(\bar{U}=0.040\).
The usual multiple-imputation total variance is \(T=\bar{U}+(1+1/m)B=0.040+(1+1/5)(0.00100)=0.0412\), so the pooled standard error is \(\sqrt{0.0412}=0.2030\). The pooled estimate is therefore 0.500 log-clearance units per 10 kg, with an approximate large-sample 95% interval \(0.500\pm1.96(0.2030)=[0.102,0.898]\). In an actual analysis, inference should use an appropriate finite-imputation degrees-of-freedom adjustment when needed. This calculation combines uncertainty within and between imputations; it does not make the result valid if the imputation model is inappropriate.
Worked Example: Coding a Missingness Indicator
Suppose baseline albumin \(A_i\), measured in g/L, is missing for some individuals. A proposed pragmatic model fills missing values with 40 g/L and defines \(M_i=1\) when albumin is missing and \(M_i=0\) otherwise. It then includes the terms \(\beta(A_i^*-40)+\gamma M_i\) in a model for log apparent clearance, where \(A_i^*\) is the filled-in albumin value.
For an individual with observed albumin of 36 g/L, the covariate contribution is \(\beta(36-40)+0\gamma=-4\beta\). For an individual with missing albumin, \(A_i^*=40\) and \(M_i=1\), so the contribution is \(\beta(40-40)+\gamma=\gamma\). Thus \(\gamma\) represents a fitted level shift for the missing-albumin group relative to an observed-albumin individual at 40 g/L, conditional on the rest of the model. It is not an estimate of the biological effect of albumin for those with missing values.
The coding is easy to calculate, but it does not reconstruct missing albumin or generally correct selection bias. Its interpretation depends on the chosen fill value and model form; it should not be treated as a universal replacement for imputation or a missing-data model.
Diagnostics, Failure Modes, and Practical Limits
Before covariate selection, document missingness counts, overlapping patterns, and the characteristics of individuals with and without each value. Check whether missingness aligns with known collection procedures, site, visit, or data-entry processes. Also establish whether the missingness concerns a baseline covariate, a time-varying covariate, or a whole individual's records; these situations may require different data structures and assumptions. Keep the audit separate from a claim that any one mechanism is established.
Warning signs include a large reduction in the analysis population under complete-case fitting; missingness concentrated in a site or subgroup; imputations outside plausible measurement ranges; imputed values with implausibly little variation; or substantial changes in estimates across defensible approaches. Such findings do not automatically identify the correct remedy, but they show that the covariate analysis may depend strongly on assumptions that need to be examined and reported.
- Complete cases as a default: treating the retained records as representative without checking the missingness process can induce selection bias.
- Mean substitution: replacing every missing value by the observed mean understates uncertainty and distorts covariate variation and relationships.
- Overconfident imputation: a complicated imputation model is not automatically a good one. It must be supported by observed information and preserve relevant NLME structure.
- Indicator as a cure: adding a missingness indicator does not generally make a biased analysis unbiased and does not estimate the missing covariate effect among those with missing values.
- Mechanism claimed from a test: observed associations with missingness can challenge MCAR, but observed data alone usually cannot distinguish MAR from MNAR.
No single strategy is best in every setting. Complete-case analysis is transparent and can be reasonable under defensible MCAR assumptions, but may waste information. Multiple imputation can retain individuals and reflect uncertainty under a suitable MAR-based imputation model, but relies on model quality. An indicator may be useful as a pragmatic predictive device in a prespecified model, but should not be presented as recovery of the absent biological measurement. When MNAR is plausible, conclusions may require explicit assumptions about the missingness process and sensitivity analysis rather than relying on a single default procedure.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In a data set of 200 individuals, 30 have a missing covariate and 20 have at least one missing value among three covariates. Can the overall number of complete cases be calculated from the first number alone?
- What does MCAR imply about the complete-case data distribution, and what does it not imply about a finite-sample estimate?
- Why is a single observed-mean replacement generally inadequate for uncertainty-aware NLME model building?
- In the albumin indicator model above, what does \(\gamma\) describe, and what does it not estimate?
- If missingness rates differ by site, has MAR rather than MNAR been established? Explain.
Solutions. (1) No. The number of individuals missing the specified covariate does not determine how many are missing any covariate; overlap among the three patterns is needed. (2) Under MCAR, selection into the complete-case subset does not change the complete-data distribution in expectation. A finite-sample estimate can still differ from an estimate using all data, and precision is generally reduced. (3) A fixed mean replacement treats imputed values as known, suppresses variation, and can distort relationships with responses and other covariates. (4) \(\gamma\) is the fitted level shift for the missing-albumin group relative to the observed group at the chosen reference value of 40 g/L, conditional on the model. It is not the biological albumin effect for individuals whose albumin is missing. (5) No. A site difference can make MCAR less plausible, but does not distinguish MAR from MNAR; the missingness may still depend on unobserved values after conditioning on observed data.