From Diagnostic Summaries to Narrative
Diagnostic plots and tables allow a reviewer to inspect model behavior. A narrative summary serves a different purpose: it gives the reader a concise, accurate account of what was computed and where the evidence is limited. The challenge is not to make the results sound decisive. It is to ensure that every sentence can be traced to a defined summary, population, and denominator.
A sentence such as “Prediction errors were small” is difficult to verify. It does not say which errors, how many records were used, what “small” means, or whether some values were unavailable. A data-linked alternative names the metric, its units, and its basis: “Among 112 records with both observed and predicted concentrations, the mean absolute prediction error was 0.450 mg/L.” This reports a computed result without turning it into a general claim about model performance.
Before writing, define the record being summarized. It might be an observation, a subject, a subgroup, or a diagnostic panel. These are not interchangeable units: 112 observation-level errors from 30 subjects do not represent 112 independent subjects. For descriptive reporting, the key requirement is to name the unit and avoid changing it mid-sentence.
A reliable summary also requires the source values to refer to the same analysis and data selection. The metric, count, and denominator should use compatible inclusion rules; units and transformations should match the diagnostic being described. If a quantity is unavailable, its status must remain distinct from a computed value of zero. These are conditions for accurate wording, not evidence that any model is adequate.
Build Sentences From Defined Quantities
A narrative can be assembled from a small set of fields: what was summarized, the eligible and valid counts, the metric and its value, the units, and any missing or sparse-data qualification. For a prediction error \(e_i\), define the sign convention before describing direction. In the examples below, \(e_i = \text{observed}_i - \text{predicted}_i\); positive mean error therefore indicates that observed values tend to exceed predictions in the summarized records.
Here \(n\) is the number of records with the values required to calculate the stated metric. If errors are concentrations, MPE, MAE, and RMSE are also in concentration units. A count or percentage must likewise identify its denominator. For example, “81.3% within 1 mg/L” is incomplete unless the reader can tell 81.3% of what.
For a threshold count, let \(k\) be the number of valid records meeting the stated condition and \(n\) the number of valid records assessed. The reported proportion is \(k/n\), usually expressed as a percentage. It describes these records; by itself, it does not establish a population-wide rate or a model-acceptance criterion.
The denominator is part of the result, not optional context. In particular, a percentage based on valid diagnostics must not silently use all nominal records as its denominator. If \(n=0\), the percentage is undefined, not zero. If some records have no computable diagnostic, state the eligible and valid counts where they matter.
Worked Example: Summarizing Prediction Errors With Units and Counts
In a hypothetical analysis, 120 observation records came from 30 subjects. Observed and predicted concentrations were both available for 112 records. For those records, the computed error summaries were \(\sum e_i = 14.0\) mg/L, \(\sum |e_i| = 50.4\) mg/L, and \(\sum e_i^2 = 31.36\) (mg/L)\(^2\). Of the 112 valid errors, 91 had absolute error no greater than 1 mg/L. Write a concise, supportable summary.
First state the valid denominator: \(n=112\), rather than 120, because eight records lack one or both values required to calculate an error. The MPE is \(14.0/112=0.125\) mg/L. Under the stated sign convention, its positive sign means the average observed-minus-predicted error is positive in these records.
For RMSE, divide the sum of squared errors by the valid count, then take the square root: \(\sqrt{31.36/112}=\sqrt{0.2800}=0.5292\) mg/L. The threshold percentage is \(100(91/112)=81.25\%\), or 81.3% to three significant figures.
A defensible narrative is: “Among 112 of 120 observation records with paired observed and predicted concentrations from 30 subjects, mean prediction error (observed minus predicted) was 0.125 mg/L, MAE was 0.450 mg/L, and RMSE was 0.529 mg/L. Absolute error was at most 1 mg/L for 91/112 records (81.3%).” This sentence gives the unit, sign convention, valid count, threshold denominator, and subject context. It does not claim that the model is adequate.
Keep Subgroup Summaries Comparable
Subgroup prose should report the subgroup-specific counts and valid denominators rather than presenting percentages alone. When valid counts differ, subgroup percentages can be compared descriptively, but the sentence should not imply that each percentage has equally precise support. Nor should a simple average of subgroup percentages be substituted for the overall percentage: the overall result uses the combined numerator divided by the combined denominator.
Worked Example: Reporting Unequal Subgroup Denominators
In a hypothetical diagnostic summary, subgroup A has 20 eligible records, of which 18 have valid errors; 15 of the valid errors are within a pre-specified absolute-error threshold of 2 mg/L. Subgroup B has 16 eligible records, 14 valid errors, and 10 within the same threshold. Prepare a subgroup and overall description.
For subgroup A, the valid-record percentage is \(100(15/18)=83.33\%\), or 83.3%. For subgroup B, it is \(100(10/14)=71.43\%\), or 71.4%. There are two and two unavailable errors, respectively. Overall, 25 of \(18+14=32\) valid records meet the threshold; \(100(25/32)=78.125\%\), or 78.1%. The eligible total is 36, so four records lack valid errors.
The overall percentage is not the unweighted mean of the subgroup percentages: \((83.33\%+71.43\%)/2=77.38\%\), which differs from 78.1% because the valid denominators differ. A suitable narrative is: “Within 2 mg/L of absolute error, 15/18 valid records (83.3%) in subgroup A and 10/14 (71.4%) in subgroup B met the threshold. Overall, 25/32 valid records (78.1%) met it; errors were unavailable for 4 of 36 eligible records.”
This wording reports the observed counts and acknowledges unavailable values. It does not call one subgroup better, attribute a difference to subgroup membership, or imply that the threshold defines acceptable model performance.
When a subgroup has few records, put the count next to the percentage and avoid emphatic comparative wording. A rate based on 3/4 records and one based on 30/40 records are both 75%, but they have very different amounts of underlying information. The count makes that distinction visible without requiring a separate inferential claim.
Missing Values, Sparse Data, and Claim Boundaries
Automated prose must distinguish at least three states: a quantity was computed and equals zero; the quantity was not computable; or the summary was not supplied. These states imply different sentences. “No records exceeded the threshold” is appropriate only when the threshold count was computed over a nonzero, defined set of valid records and the count is zero. “The exceedance count was unavailable” is appropriate when it could not be calculated. A missing value must not be translated into zero or omitted in a way that suggests complete coverage.
Worked Example: Describing a Threshold Count Without Overclaiming
A hypothetical subgroup contains 46 eligible standardized residuals from 12 subjects. Standardized residuals were computable for 39 records; six of those 39 had absolute value greater than 2. The threshold was specified for descriptive review, not as a pass/fail criterion. Write the result.
Seven of the 46 eligible residuals were unavailable, because \(46-39=7\). The observed exceedance percentage among valid residuals is \(100(6/39)=15.38\%\), or 15.4%. The remaining \(39-6=33\) valid residuals did not exceed the threshold. These arithmetic statements do not imply that missing residuals would have followed the same pattern.
A careful summary is: “Among 39 of 46 eligible standardized residuals from 12 subjects, 6 (15.4%) had absolute value greater than 2; 7 residuals were unavailable. The threshold count is descriptive and is not, on its own, a model-adequacy decision.” This reports the count and missingness, while making the threshold’s limited role explicit.
The same restraint applies to all diagnostic summaries. A descriptive statistic characterizes the selected records under its calculation rules. A plot pattern or metric may warrant review, but a sentence generated from that metric alone cannot establish why the pattern occurred, whether it will persist in another data set, or whether the model is fit for a particular decision. Avoid causal explanations, universal labels such as “good fit,” and pass/fail conclusions unless a separate, explicitly defined assessment supports them.
A Practical Construction and Review Sequence
A short, repeatable review can prevent common errors before narrative text is released. It is useful whether prose is drafted manually or assembled from computed fields. The review checks the link between the data and the wording; it is not a substitute for examining the diagnostics themselves.
Name the analysis, diagnostic, record unit, subgroup if applicable, and eligibility rule. Confirm that the values being summarized refer to the intended data selection.
Check each metric, unit, numerator, valid denominator, eligible count, and missing status. Confirm that threshold direction and error sign conventions are explicit.
Use a sentence structure that gives the quantity and its count or denominator. Add a limitation when data are sparse or unavailable, and do not turn a descriptive value into a model-adequacy verdict.
Recalculate percentages from the stated counts, verify units and rounding, and confirm that no sentence describes an absent value as zero or changes the population being summarized.
This sequence also clarifies how narrative relates to other diagnostic outputs. A table is preferable when readers need to compare many metrics or subgroups precisely; a plot supports visual inspection of distributions and patterns; concise prose points to a small set of results and qualifications. Narrative should complement, not replace, the underlying counts, metrics, and diagnostic displays.
A common mistake is to write “The diagnostics showed acceptable fit because 81% of errors were within 1 mg/L.” The percentage is a computed descriptive result, but “acceptable fit” does not follow from it unless an appropriate acceptance rule and its application have been specified. The correction is to report the result as a result: “Absolute error was within 1 mg/L for 91 of 112 valid records (81.3%).” Then leave the broader adequacy judgment to a review that considers the relevant evidence and criteria.
Check Your Understanding
Answer each question, then compare with the solutions below.
- There are 80 eligible records, 72 valid errors, and 54 valid errors within a stated threshold. What percentage should be reported, and how should unavailable errors be described?
- A signed mean error is negative under the convention observed minus predicted. What direction does that indicate for the average error in the summarized records?
- Subgroup A has 8/10 records meeting a threshold and subgroup B has 18/30. Calculate the overall percentage. Why is averaging the two subgroup percentages not the overall result?
- A diagnostic metric is unavailable for a subgroup. Why is reporting the metric as zero incorrect, and what is a suitable alternative?
Solutions. (1) Use the 72 valid errors: \(100(54/72)=75.0\%\). State that 8 of 80 eligible errors were unavailable; do not use 80 as the threshold denominator without saying so. (2) The average observed-minus-predicted error is negative, so observed values are lower than predicted values on average among the summarized records. This is descriptive and does not identify a cause. (3) The combined count is \(8+18=26\) of \(10+30=40\), so \(100(26/40)=65.0\%\). The unweighted mean of 80% and 60% is 70%, which gives the smaller subgroup the same weight as the larger one rather than using the combined record denominator. (4) Unavailable means the metric could not be computed or was not supplied; zero is a computed numeric value and would assert a result that is not known. State that the metric was unavailable and give the reason if it is known.