Why Reporting Decisions Matters
A regression result depends not only on the model fitted but also on how incomplete records were handled. Readers need enough information to understand who contributed to the analysis, which variables were incomplete, what assumptions supported the chosen strategy, and whether plausible alternatives changed the conclusion. A statement such as “missing data were handled using imputation” leaves these decisions largely hidden.
Transparent reporting does not require reproducing every software command in the main text. It does require stating the essential decisions precisely, with counts, model details, assumptions, diagnostics, and results that can be checked. The aim is to let readers distinguish what was observed in the data from what was assumed to address values that were not observed.
What to Report
Start with the analysis population and the missingness counts. Give the total number of eligible observations and, for each analysis variable, the number and percentage with missing values. State the denominator used for each percentage. If missingness in different variables overlaps, give either a pattern summary or the number with complete data for the variables in the regression. Marginal counts alone do not show how many records would be excluded by a complete-case analysis.
For example, let \(M_j\) be the set of observations missing variable \(j\), among \(N\) eligible observations. The count missing at least one of the variables is the size of the union of those sets, not necessarily the sum of their individual sizes. Overlap matters because the same observation can be missing more than one variable.
This identity explains why a report should not estimate the complete-case sample size by subtracting marginal missing counts unless the missingness sets are known to be disjoint. State how many observations were excluded, and why. If imputation was used, distinguish the records retained in the imputed analysis from any records excluded for other reasons, such as ineligibility or unavailable outcome information that was not imputed.
Next describe the chosen strategy at the level needed to understand and reproduce it. For multiple imputation, name the variables imputed, the variables included in the imputation models, the model types used for different variable types, and the number of imputed data sets. State the regression model fitted to each completed data set and how estimates were pooled. Explain the assumptions: in particular, the missing-at-random assumption is conditional on the observed information used in the analysis and imputation models. It is not established merely by including many predictors or by a favorable diagnostic.
Finally, summarize checks and sensitivity analyses. Useful diagnostic summaries can include whether monitored quantities stabilized across iterations and whether imputed values were plausible in range and distribution. State what was checked and what was found, rather than saying only that “diagnostics were satisfactory.” A sensitivity analysis should identify the alternative assumption or imputation scenario, report the resulting estimate, and compare it with the primary result. Diagnostics can reveal problems, but they cannot verify an untestable missingness assumption.
Report the eligible sample size, variable-specific missing counts and percentages, overlap or complete-case count, and any exclusions with reasons.
Identify the imputed variables, predictors and model forms in the imputation procedure, number of imputations, regression model, and pooling method.
State assumptions, diagnostics, sensitivity results, and limitations in language that distinguishes evidence from assumptions.
Worked Reporting Examples
Worked Example: Accounting for Missingness and Exclusions
Suppose a hypothetical observational study has 240 eligible participants. The regression variables are treatment group, age, baseline severity, and a continuous outcome measured at follow-up. Treatment group is observed for everyone. Age is missing for 16 participants, baseline severity for 42, and the outcome for 18. Among the 240, 180 have complete values for all four regression variables. How should the missingness be summarized?
First calculate each percentage using the eligible sample as the denominator. For age, \(16/240 \times 100 = 6.67\%\). For baseline severity, \(42/240 \times 100 = 17.5\%\). For the outcome, \(18/240 \times 100 = 7.50\%\). The complete-case count is 180, which is \(180/240 \times 100 = 75.0\%\). Thus, 60 participants have at least one missing regression variable.
The marginal counts sum to \(16+42+18=76\), not 60, because some participants are missing more than one variable. A transparent report can state the marginal counts and percentages, the complete-case count, and that missingness overlaps. If a pattern table is important to the analysis, it can give the distinct combinations of observed and missing variables. It should not imply that 76 separate participants were missing data.
A concise report might say: “Of 240 eligible participants, age was missing for 16 (6.7%), baseline severity for 42 (17.5%), and the follow-up outcome for 18 (7.5%); counts overlap. Complete-case records were available for 180 (75.0%). No participants were excluded for missingness before the imputed analysis.” That last sentence is appropriate only if no such exclusions actually occurred. Any additional exclusions should be counted and explained separately.
Worked Example: Describing an Imputation Analysis
Using the hypothetical study above, suppose the analysts create 40 multiply imputed data sets. They impute age, baseline severity, and the outcome, but not the fully observed treatment group. The imputation models also include study site, an observed auxiliary variable related to follow-up, and use predictive mean matching for age and severity and a linear regression model for the continuous outcome. In each completed data set, they regress the outcome on treatment group, age, and baseline severity. What information belongs in the report?
The report should identify all three imputed variables, the treatment and auxiliary variables used to predict missing values, the model types, and the 40 imputations. It should state that the treatment coefficient and its uncertainty were pooled across the fitted regressions using the multiple-imputation pooling procedure. These details clarify that the outcome was imputed as well as the predictors, and that the outcome and auxiliary information contributed to imputing incomplete predictors.
The assumption should be stated with its conditioning information: “The primary imputation analysis assumes missing at random conditional on treatment group, study site, the other analysis variables, and the auxiliary information included in the imputation models.” This does not establish that the assumption is true. It states the condition under which the primary analysis is interpreted.
Suppose the imputation diagnostics show no evident drift in monitored means over iterations, and the imputed age and severity values fall within plausible observed ranges. Report those checks specifically. They support the practical adequacy of the generated imputations but do not prove that the imputation models are correctly specified or that missingness is missing at random. The statement “all assumptions were verified” would therefore overstate what the checks show.
Worked Example: Reporting a Regression Result and Sensitivity Analysis
Suppose the pooled treatment coefficient in the primary imputed analysis is \(-2.40\) outcome units, with \(SE=0.82\), and a 95% confidence interval of \([-4.04,-0.76]\). A prespecified sensitivity analysis shifts imputed baseline severity values upward by 5 points relative to their primary imputation draws; the resulting treatment coefficient is \(-1.60\) outcome units with \(SE=0.85\). How can these results be reported without overstating them?
For the primary analysis, the interval is consistent with a negative adjusted association. In this hypothetical setting, the point estimate represents an estimated difference of 2.40 outcome units lower in the treatment group than in the comparison group, conditional on age and baseline severity in the stated regression model. Because the study is observational, this adjusted association alone does not establish a causal treatment effect.
The sensitivity result changes the point estimate by \(-1.60-(-2.40)=0.80\) outcome units toward zero. A report should state the imposed 5-point shift, the revised estimate and uncertainty, and how the interpretation compares with the primary analysis. It should not call this simply “robust” or “not robust” without explaining the scenario and the change. The shift is a specified alternative assumption, not evidence that the shifted values are the true missing values.
A compact summary might say: “Under the primary missing-at-random imputation, the adjusted treatment-group difference was \(-2.40\) outcome units (95% CI \(-4.04\) to \(-0.76\)). When imputed baseline severity values were shifted upward by 5 points, the estimate was \(-1.60\) units (\(SE=0.85\)). The direction remained negative, but the magnitude was smaller; conclusions depend in part on assumptions about unobserved severity.” The interval and standard error belong to their respective analyses and should not be interchanged.
Common Pitfalls and Limitations
A frequent reporting mistake is to say that missing data were “random” because observed characteristics did not differ between records with and without missing values. Such comparisons describe observed data; they cannot rule out dependence on unobserved values. Instead, name the assumption used, specify the observed information on which it is conditioned, and acknowledge that the assumption cannot be confirmed from the observed data alone.
Other common errors include reporting percentages without denominators, adding marginal missing counts as though they were distinct people, omitting exclusions, or giving a sensitivity result without defining the scenario. It is also misleading to present diagnostics as proof of the imputation assumptions. A sound report links each claim to what was actually checked or assumed.
No reporting template removes uncertainty about missing values that were never observed. A clear account does, however, help readers judge which decisions matter, whether the analysis is reproducible, and how strongly the substantive conclusion depends on those decisions. The most useful limitations are specific: for example, the analysis relied on missing at random conditional on measured variables, and a particular sensitivity shift changed the estimated association.
Check Your Understanding
Use precise wording and keep observed facts separate from assumptions.
- Why can the sum of variable-specific missing counts exceed the number of participants with any missing value?
- What counts and denominators would you report for 30 missing values among 300 eligible observations?
- Why should a multiple-imputation report identify variables included in the imputation models, not only variables in the final regression?
- What does it mean to assume missing at random conditional on observed information, and why do diagnostics not prove that assumption?
- What information should accompany a sensitivity-analysis estimate so readers can interpret its relevance?