Presenting Data-Quality Findings in Context

Why Data Quality Belongs Beside the Findings

A predictive result is only as informative as the data used to produce and assess it. A report that gives a performance figure but omits substantial missingness, unverified extreme values, or incomplete outcome records can leave decision-makers with an overly confident impression. The goal is not to list every data-cleaning step. It is to identify consequential limitations, state how they were handled, and explain what effect they could have on the reported findings.

A useful account is specific: name the affected variable or records, give the number or proportion affected, describe the handling decision, and state the plausible consequence. Distinguish what is known from what is possible. For example, “12% of records lacked income” is a measured fact; “the missing records may differ from the analyzed records” is a limitation to consider, not a demonstrated cause of bias.

Describe the Issue, the Handling, and the Implication

A compact report can organize each issue around three questions: How much data are affected? What was done about them? How might that decision affect the result? State the denominator as well as the count. “60 records were missing income” is less interpretable than “income was missing for 60 of 500 records (12%).” If different variables have different amounts of missing information, report them separately rather than implying that every record has the same completeness.

Missingness can affect findings when records without a value differ from records with a value. A simple identity shows why the observed mean alone may not describe the full group. Suppose a quantitative variable is observed for \(n_o\) records and missing for \(n_m\) records, with \(n=n_o+n_m\). If the full-group mean of the missing values would be \(\bar{x}_m\), and the observed mean is \(\bar{x}_o\), then the full-group mean is:

The second equality follows by writing \(n_o=n-n_m\), substituting, and collecting terms. It shows that the full-group mean equals the observed mean if the missing records have the same mean, or if no records are missing. Otherwise, the difference depends on both the missing fraction and the difference between the two group means. Since \(\bar{x}_m\) is not observed, the report generally cannot establish the direction or size of that difference from the observed records alone.

Separate Unusual Values From Confirmed Errors

An extreme or surprising value is a reason to investigate, not automatic proof that the value is wrong. A valid but unusual observation may be important to the population being studied. A confirmed data-entry error may warrant correction if a reliable source supports the correction; otherwise, it may need to remain unresolved. The report should distinguish these cases and state whether the displayed result includes the observation.

Sensitivity checks can show how much a finding changes under a stated alternative treatment. They do not determine which treatment is correct. In particular, showing a result with and without an extreme record is not evidence that the record is invalid. The reason for the alternative treatment must be explicit, and the primary result should use the chosen, justified handling.

Incomplete Outcomes Limit What Can Be Evaluated

A missing predictor value and an unavailable outcome are different reporting issues. Missing predictors may restrict which records can be used in an analysis requiring those predictors. Missing outcomes can make it impossible to determine whether an event occurred for certain records, limiting conclusions about event frequency or predictive performance. State how many outcomes are known and unknown, and avoid treating unobserved outcomes as negative outcomes.

When the outcome is binary and no assumptions are made about the unobserved outcomes, simple bounds can communicate the uncertainty in the overall event proportion. The lower bound counts every unknown outcome as no event; the upper bound counts every unknown outcome as an event. These bounds may be wide, but they make the information gap visible rather than concealing it.

Report Sensitivity Without Overstating It

A sensitivity result is useful when it answers a clear question: how much does a finding change under a plausible alternative handling decision? State the primary result, the alternative, and why the alternative is being shown. Keep the distinction between a diagnostic flag and a verified defect. If the alternative excludes a record only to illustrate influence, say that the exclusion is illustrative rather than a recommended correction.

For example, the root mean squared error is the square root of the mean of squared prediction errors. A large error can have a substantial effect because it is squared. That fact can motivate an influence check, but does not justify removing a record. Consider errors \(-2,1,1,2,9\). Using all five, the squared errors sum to \(4+1+1+4+81=91\), so the mean squared error is \(91/5=18.2\), and the root mean squared error is \(\sqrt{18.2}=4.266145\ldots\), or 4.266 to three decimals. Using only the first four as a sensitivity comparison gives a squared-error sum of 10, a mean of \(10/4=2.5\), and a root mean squared error of \(\sqrt{2.5}=1.581139\ldots\), or 1.581 to three decimals. The report should show the comparison without assuming that the error of 9 is invalid.

A common reporting trap is to remove an observation because it changes a summary substantially, then present the revised result as more trustworthy without evidence. The correction is to investigate the record, document what was verified, and report the primary handling transparently. If validity cannot be established, show an appropriate sensitivity comparison and retain the uncertainty in the interpretation.

What a Decision-Ready Data-Quality Note Includes

Include issues that could change how a reader interprets the population, comparison, or result. A minor formatting inconsistency with no effect on the analysis may not deserve space in the main report. A substantial missing-data pattern, an unresolved extreme record, or incomplete follow-up often does. The level of detail should be enough for a decision-maker to understand the limitation without implying that the analysis is unusable.

• Extent: give counts and denominators, and identify the affected variable or outcomes.
• Handling: say what was retained, excluded, corrected, or left unresolved, and why.
• Implication: identify which reported findings or population claims may be affected.
• Uncertainty: distinguish observed facts from plausible effects that the data cannot establish.

This approach is aligned with the SOA Exam PA (Predictive Analytics) syllabus because it connects analytical choices to responsible interpretation. A concise, contextual limitation is more useful than either silence or an unprioritized list of every data irregularity.