Workshop: Assemble a Complete Diagnostic Report

From Diagnostic Outputs to a Complete Report

A diagnostic report is more than a collection of plots. It is a set of connected evidence: the plots and tables must describe the same analysis population, the captions must identify the quantities shown, and the rendered document must preserve the intended content. A report can look polished and still mislead if, for example, its population plot uses one set of records while its summary table uses another.

This workshop turns standardized prediction and residual outputs into a complete diagnostic report. The focus is on assembling and checking the report, not on making a model decision from any single diagnostic. The earlier material on automated diagnostic summaries provides context for the numerical summaries used here.

Before assembling a report, establish what one row represents and which columns are required. For the examples below, each row is one included observation, DV and predictions are concentrations in mg/L, and CWRES is dimensionless. The raw residual convention is observed minus predicted. A report pipeline must preserve the subject and time identifiers as well as these quantities so that plots, summaries, and individual panels can be joined to the correct records.

The workflow also assumes that the predictions and residuals come from the same model run and analysis population, and that their records are aligned to the observations they describe. Numerical summaries below are descriptive and do not require normally distributed residuals. However, interpreting a standardized residual against a reference distribution does depend on how it was constructed and on the model and error assumptions behind that construction. State those assumptions rather than treating every standardized residual as interchangeable.

Prepare and Validate the Report Inputs

Use one standardized observation-level data set as the report’s source for plots and summaries. At a minimum, retain an analysis identifier, subject identifier, time, DV, PRED, IPRED, and any standardized residuals to be displayed. Keep units, transformations, prediction definitions, and residual sign conventions in report metadata rather than relying on a reader to infer them from column names.

Before plotting, check that the expected records are present, identifiers and times are usable, values have compatible units, and each output is matched to the intended observation. Count missing values by quantity: a missing CWRES need not remove a row from a DV-versus-IPRED plot, but it must not silently enter a CWRES summary as if it were observed. If different displays use different eligible record counts, state those counts.

Build Plots and Summaries From a Shared Source

A complete diagnostic report typically combines population-level views with subject-level views. A population plot can show DV against PRED, while residual plots can show a standardized residual against PRED or time. Individual panels commonly show DV and IPRED over time for each subject; if the report also shows PRED, distinguish its line or symbol clearly. The axes, units, transformations, and inclusion rules should be explicit. Plot limits should preserve relevant patterns rather than clipping observations.

Summary tables should identify the quantity, record count, units where applicable, and statistic definitions. For raw residuals \(e_i=DV_i-PRED_i\), the mean residual describes average signed discrepancy, while the root mean squared error (RMSE) summarizes squared discrepancies on the original measurement scale:

Here \(n\) is the number of eligible records for that particular quantity. RMSE is nonnegative and has the same units as the residual. It is not the sample standard deviation: it incorporates both spread and any nonzero mean residual.

A short identity explains why the two summaries answer different questions. Write each residual as \(e_i=(e_i-\bar{e})+\bar{e}\). On squaring and summing, the cross-product terms sum to zero because \(\sum_i(e_i-\bar{e})=0\). Dividing by \(n\) gives:

Thus the mean squared residual combines the average squared deviation around the mean with the squared mean itself. Taking its square root gives RMSE. This is why a report should not label RMSE as “residual variability” alone or substitute the signed mean for an overall error-size summary.

Assemble Captions, Tables, and Rendered Output

A caption should say what is plotted, the population or record set included, and any essential display choices. Captions generated from metadata are more reliable than hand-written captions copied between analyses, provided the metadata are correct. For example, “Observed concentration (DV, mg/L) versus population prediction (PRED, mg/L); six observations from three hypothetical subjects” identifies both variables, units, and scope. A corresponding CWRES caption should say that CWRES is dimensionless and identify its horizontal variable.

Organize report sections so readers can move from analysis context and record counts to population plots, individual plots, summary tables, and a concise data-linked narrative. The narrative should report what the generated summaries and displays show, without turning a descriptive pattern into a causal claim or an unsupported model conclusion. The report’s source data and metadata should drive the values and captions together. HTML and PDF may differ in pagination or navigation, but should not disagree in analysis content.

Diagnostics, Alternatives, and Limitations

Useful report-generation checks include reconciling the number of observations in the input, plots, and tables; confirming that each subject panel is present; checking for duplicated or unexpected subject-time records; and verifying that captions and axes describe the same variables and units. Examine warnings and missing-value handling explicitly. A plot with no visible points may be a valid result, an empty filtered subset, or a rendering failure; the report should not silently treat these possibilities as equivalent.

A common mistake is to see a successful HTML or PDF render and conclude that the report is complete and numerically correct. Rendering confirms that a document was produced, not that it contains every intended figure or that its summaries used the right rows. The correction is to add explicit checks—such as expected record and panel counts—and compare key table entries with calculations from the standardized input. Another frequent error is to compare a raw residual in mg/L with a standardized residual as if they were on the same scale. Keep their labels, units, and interpretations distinct.

A scripted, metadata-driven report is generally more reproducible than manually assembling plots and captions in a spreadsheet or document editor: it reduces repeated transcription and makes consistent updates easier. Manual assembly can be useful for a small exploratory review, but it is more vulnerable to stale captions, inconsistent filters, and copy-and-paste errors. HTML is convenient for navigation and screen viewing; PDF is useful for stable pagination and printing. Neither format guarantees content accuracy, so the choice of format does not replace input and render checks.

Automation also has limits. A report template cannot determine whether the model, analysis population, or residual definition is scientifically appropriate, and a summary table cannot replace inspection of the diagnostic plots. A misleading input can produce a consistent-looking but misleading report. Treat report generation as a controlled transformation of defined outputs, and preserve the definitions and eligible counts needed to interpret each display.