Reporting Missing Data and Data Processing

Why Missing-Data Reporting Matters

A reader cannot evaluate an observational comparison if the report says only that “missing values were handled appropriately.” Missingness can differ between treatment groups, affect particular covariates or outcomes, and change which individuals contribute to an analysis. Data processing choices can also alter the meaning or scale of measured variables. Reporting these details lets readers understand what data were available, what was changed, and which assumptions support the chosen handling procedure.

This tutorial focuses on transparent reporting, not on selecting a propensity-score model or presenting matching and weighting diagnostics. The essential practice is to describe missingness and processing decisions at the level of each relevant variable and analysis group, then explain how those decisions connect to the analysis sample.

A variable may be unavailable for a meaningful reason. For example, a laboratory test may not have been ordered, or a question may not apply to a particular participant. State whether such cases were coded as missing, treated as a separate category, or excluded from the denominator. Do not combine “not applicable,” “not measured,” “unknown,” and a genuine zero without explaining the coding: they can have different meanings.

Describe Missingness by Variable and Group

A useful summary reports counts and percentages for each important variable separately in the treatment and comparator groups. State the denominator, define the groups, and identify the time point or measurement window. An overall percentage alone can hide a group difference; a group-level summary alone can hide that one variable is much less complete than another.

For a variable intended for all group members, calculate the percentage as \(100 \times \text{number missing}/\text{group denominator}\). If intended measurement varies, report the eligible denominator as well as the missing count. Specify whether the table describes the eligible cohort, the analysis sample before missing-data handling, or another point in cohort construction. If missingness overlaps across variables, do not add variable-specific missing counts and call the sum the number of people excluded; a person may be missing more than one variable.

A short derivation clarifies why denominators matter. If \(M\) values are missing among \(N\) eligible individuals, the observed-value count is \(N-M\), and the missingness proportion is \(M/N\). Using only observed values as the denominator instead gives \(M/(N-M)\), which is not the fraction missing among eligible individuals. For example, 20 missing values among 100 eligible people means 20%, not \(20/80=25\%\). If only 80 people were eligible for measurement, however, the correct denominator would be 80; eligibility must therefore be defined, not guessed from the data table.

State the Handling Procedure and Its Assumptions

Reports should name the handling procedure and specify which variables and analyses it affected. With complete-case analysis, state the variables that had to be observed, the number retained and excluded, and the counts by group. The term “complete case” is otherwise ambiguous: a record may be complete for one model but incomplete for another.

Complete-case analysis does not automatically preserve an unbiased comparison. Let \(Y\) be a variable of interest and \(R=1\) indicate that it is observed. If missingness is completely at random, \(R\) is independent of the full data, including \(Y\). Then:

This follows because independence makes the distribution of \(Y\) among records with \(R=1\) the same as the distribution of \(Y\) overall. If selection into the complete-case sample depends on observed or unobserved data, that equality need not hold. In comparative analyses, the relevant conditions can involve treatment, outcome, and covariates; an overall complete-case percentage does not establish that the analysis is unbiased.

Multiple imputation is another common approach. Report the variables imputed, the number of imputed data sets, the broad imputation model specification, how treatment and outcome information were incorporated when appropriate, and how estimates and uncertainty were combined. State the assumptions supporting the method. In particular, missing at random (MAR) means that, conditional on observed information included in the missingness model, the probability of missingness does not depend on the unobserved value itself. MAR is an assumption, not a conclusion established by a test. Missing not at random (MNAR) means missingness can still depend on the unobserved value after conditioning on observed information. No routine check of observed data can rule out MNAR.

For any approach, distinguish the assumption from the evidence used to assess whether the procedure behaved plausibly. Useful checks may include comparing missingness patterns across groups and measured characteristics, checking imputed values against plausible ranges and observed distributions, and reviewing convergence or other algorithm diagnostics for the chosen imputation procedure. These checks can reveal problems; they cannot prove that the missingness assumption is true.

Document Coding and Transformations

Data processing is part of the analysis and should be reproducible from the report. Describe how raw values became analysis variables, especially when the choices affect interpretation or eligibility. Relevant details include category mappings, reference groups, units, measurement windows, derived-variable formulas, handling of implausible values, and whether a transformation was performed before or after imputation.

For example, report whether a recorded smoking code of 9 meant “unknown” and was converted to missing, rather than treated as a smoking category. If a laboratory value recorded in mg/dL was converted to mmol/L, give the conversion factor or formula and identify the units used in analysis. If negative values were set to missing because they were impossible under the measurement definition, state the rule and how many records it affected. Avoid vague statements such as “variables were cleaned” or “outliers were removed.”

Diagnostics, Alternatives, and Common Pitfalls

The principal alternatives are complete-case analysis, imputation, and other explicitly defined approaches such as using available observations for a particular summary. They answer to different assumptions and can yield different analysis populations. A report should not present them as interchangeable data-cleaning choices. If a study uses more than one approach, state which result is primary and describe the others as sensitivity or supplementary analyses only when that is how they were prespecified and interpreted.

Check for coding errors such as special numeric codes accidentally treated as real measurements, transformations applied in inconsistent units, and missingness percentages calculated with the wrong denominator. For imputation, inspect whether the procedure completed as intended, whether imputed values violate known constraints, and whether the imputed distributions are plausible in context. For complete-case analysis, compare the reported retained counts with the stated exclusion rule and check whether group-specific losses are visible.

A common mistake is to write, “A test found no significant difference in missingness, so the data were missing completely at random.” This is incorrect. A nonsignificant test does not establish equality of missingness patterns, and an observed-data comparison cannot test whether missingness depends on values that were never observed. The correction is to report the observed patterns and any checks, state the assumption used by the analysis, and acknowledge that the assumption cannot generally be confirmed from observed data alone.

Another mistake is to report only the total number of imputed cells or excluded records. That number conceals which variables and groups were affected. Report variable-by-group missingness and analysis-specific participant counts, then explain how overlapping missing values contributed to exclusions or were handled by imputation. Keep the account bounded: document missingness and processing here, while presenting propensity-score specifications, balance diagnostics, and final effect estimates in their appropriate analyses.