Why generate report outputs from saved results?
A table or figure in a report is part of the statistical record: it communicates particular results from a particular analysis. If it is assembled by hand, a later change to the saved analysis may not be reflected in the report. A number can be copied incorrectly, a plot can retain an old axis label, or an image can be replaced without updating its caption. Generating outputs with a script makes the connection between saved results and reported content explicit and repeatable.
The goal is not simply to produce the same-looking file twice. A sound workflow identifies the saved MICE result that supplies each output, extracts the intended values, applies declared formatting, and checks that the generated content agrees with the analysis being documented. Saving an image or spreadsheet is the final step, not a substitute for checking its contents.
This workflow assumes that the saved results are the intended results, can be read successfully, and contain the fields needed for the requested output. The analysis record should identify the relevant saved object and its provenance. The extraction and formatting rules must also suit the estimand: for example, a table of multiply imputed-data estimates should report the pooled result, not an arbitrary completed-data estimate. Output checks can reveal a mismatch between results and report, but cannot establish that the original analysis was appropriate.
Separate extraction, calculation, formatting, and export
A reliable script treats output creation as a small pipeline. Keep the source results unchanged; select the required fields; calculate summaries where needed; format labels, units, and rounding; then export. These stages make it easier to locate an error. For example, a wrong estimate might originate in the extraction step, whereas an incorrect number of decimal places belongs to formatting.
Specify the saved MICE result, the analysis or variable it represents, and the version or run identifier.
Select the intended estimates, diagnostics, or counts. Apply any required statistical calculation using the saved result, not values copied from an old report.
Set titles, labels, units, rounding, legends, and captions from declared rules. Keep these choices separate from the numerical source values.
Write the output to a known location, then check its content, dimensions, labels, and agreement with the documented analysis.
A saved object may contain several completed data sets, model fits, and diagnostics. The script should name the component it uses rather than relying on an implicit default or on whichever object happens to be available in the current session. Where an output depends on a setting such as the outcome, subgroup, iteration range, or interval level, record that setting as part of the output-generation process.
Worked examples
Worked Example: Building a pooled-estimate table
Suppose a hypothetical saved MICE result contains five estimates of a treatment coefficient, in score points: \(1.8, 2.0, 2.1, 1.9,\) and \(2.2\). Their corresponding within-imputation variances are all \(0.16\) score-points squared. The table is intended to show the pooled estimate and a 95% confidence interval. Assume the five estimates and variances are the saved results for the same specified model and coefficient.
First calculate the mean estimate, \(\bar{Q}\), and the mean within-imputation variance, \(\bar{U}\):
The between-imputation variance \(B\) is the sample variance of the five estimates. Their deviations from \(2.0\) are \(-0.2, 0, 0.1, -0.1,\) and \(0.2\), so
For \(m=5\), the total variance and its square-root standard error are
Using the usual Rubin degrees-of-freedom approximation, \(df=(m-1)\left[1+\bar{U}/\{(1+1/m)B\}\right]^2\), gives \(df=4[1+0.16/0.03]^2\approx160.4\). The two-sided 95% critical value is approximately \(1.975\), so the interval is \(2.0\pm1.975(0.4359)\), or approximately \((1.139,\ 2.861)\) score points. A generated table row can therefore report the coefficient as 2.00 score points, \(SE=0.436\), and 95% confidence interval 1.14 to 2.86 score points, with the model and coefficient identified in its labels.
The script should extract all five estimates and variances from the saved result, calculate these quantities, and then format the row. It should not read a manually typed estimate from an earlier table. The numerical result is conditional on the saved values and the stated pooling calculation; automating its display does not validate the model specification.
Worked Example: Plotting saved iteration-level diagnostics
A hypothetical saved diagnostic record contains the mean of an imputed BMI variable at three iterations for each of three chains. The values, in kg/m2, are:
| Iteration | Chain 1 | Chain 2 | Chain 3 |
|---|---|---|---|
| 1 | 26.1 | 25.8 | 26.2 |
| 2 | 26.4 | 26.0 | 26.1 |
| 3 | 26.2 | 26.3 | 26.0 |
The intended figure has iteration on the horizontal axis, mean imputed BMI in kg/m2 on the vertical axis, and one labelled line per chain. The script extracts three values per chain, so the plotted data contain \(3\times3=9\) points. It uses the saved iteration index rather than assigning new indices after sorting or filtering.
For a basic consistency check, calculate the mean across the three chain values at each iteration. At iteration 1, \((26.1+25.8+26.2)/3=78.1/3=26.03\) kg/m2. At iteration 2, \((26.4+26.0+26.1)/3=78.5/3=26.17\) kg/m2. At iteration 3, \((26.2+26.3+26.0)/3=78.5/3=26.17\) kg/m2. These summaries are useful for checking extraction, but the plotted lines should retain each chain’s values rather than plot only these averages.
The small example cannot establish that the MICE chains have mixed adequately or that the imputations are suitable. Its purpose is to show that the output can be rebuilt from saved values with the intended variable, iteration labels, units, and chain identity. A figure showing a different number of points, an unlabeled chain, or a different iteration range requires investigation before it is used.
Worked Example: Recalculating a missingness table
Suppose a hypothetical saved input record documents 80 participants and missing-value counts of 8 for age, 20 for a laboratory measure, and 12 for an outcome. A report table should show the count and percentage missing for each variable. Assume all three counts use the same denominator of 80 participants.
The percentage is \(100\times\text{missing count}/80\). For age, \(100(8/80)=10\%\); for the laboratory measure, \(100(20/80)=25\%\); and for the outcome, \(100(12/80)=15\%\). A script-generated table would show 8 (10%), 20 (25%), and 12 (15%), respectively, with the denominator and variable names made clear.
Imagine that an existing report image shows 24% missing for the laboratory measure. Rebuilding from the saved counts yields 25%, not 24%. The script should regenerate the output and flag or replace the discrepant image; it should not adjust the count to preserve the old figure. The check also depends on a valid denominator: if the analysis defines a different eligible population for a variable, that denominator must be represented explicitly instead of assuming 80.
Verification and failure modes
A regenerated output should be checked at more than one level. First check that the script read the intended saved result and that extraction succeeded: expected variables, coefficients, chains, or iterations should be present. Then check the statistical content: recalculate selected entries independently, confirm denominators and units, and compare numerical values with the documented analysis. Finally check presentation: titles, labels, rounding, axis ranges, legends, and captions should accurately describe the plotted or tabulated data.
For numerical tables, comparison can be made on unrounded values before display formatting. A useful check is to compare generated estimates and interval endpoints with the corresponding stored or independently recalculated values within a declared numerical tolerance. For figures, verify the data supplied to the plotting step as well as the rendered image. Two image files can differ at the pixel level because of font or graphics-device changes while representing the same data; conversely, two visually similar images can encode different values. Therefore, an exact image-file comparison is not a sufficient statistical check.
Common warning signs include a missing output file, unexpected row or point counts, duplicated labels, a variable absent from the saved result, an interval inconsistent with its estimate and standard error, or a report value that differs from a regenerated value. A script that runs without error is not necessarily correct: it may select the wrong coefficient, use the wrong completed data set, or apply an inappropriate denominator. Explicit checks should fail visibly when required fields are absent or values do not meet expectations, rather than silently producing an incomplete output.
Alternatives and limitations
Manual editing can be convenient for a one-off annotation, but it weakens the link between the MICE result and the displayed numbers. A safer alternative is to script both the numerical output and its formatting, then make any exceptional annotation explicit in the script. A second approach is to copy values into a spreadsheet and build the figure there; this can be appropriate when the transfer is controlled and checked, but it introduces another step where values, labels, or denominators can be altered. Generating directly from the saved results reduces these transfer risks and makes updates easier.
Automation has limits. It can faithfully reproduce a wrong model, misleading label, stale saved object, or incorrect selection rule. Saved results and the report script need enough provenance to identify which analysis they represent, and the output should be reviewed in context. Generated tables and figures support accurate reporting; they do not replace statistical judgment or diagnostic interpretation. These distinctions are important when comparing a polished output to the underlying analysis, as emphasized in the earlier material on Reporting MICE Diagnostics Reproducibly.
Check Your Understanding
For each question, consider both the numerical content and the output-generation process. Short solutions follow.
- A saved result has 4 estimates \(0.9, 1.0, 1.1,\) and \(1.0\). What is their mean, and what should a table-generating script do before rounding it?
- A figure contains 12 points, but the saved record has 4 chains and 4 iterations. What point count is expected if one value is plotted per chain per iteration, and what should be checked?
- A missingness table gives 6 missing values among 48 participants. What percentage should the script report?
- Why is comparing two exported image files byte for byte not a sufficient check that their statistical content agrees?
- A generated output matches a prior report, but the saved result identifier is different. Is that enough to establish that the output documents the current analysis? Explain.
Solutions. (1) The mean is \((0.9+1.0+1.1+1.0)/4=4.0/4=1.00\). Before rounding, the script should verify that it selected the intended four estimates and that the table’s model, coefficient, and units are correct. (2) The expected count is \(4\times4=16\). Check whether all chains and iterations were extracted, whether filtering was intended, and whether the plotted data use the correct iteration and chain labels. (3) \(100(6/48)=12.5\%\), assuming 48 is the appropriate denominator. (4) Export differences can arise from fonts or rendering settings without changing the data, while similar-looking images can contain different values; check the plotted data and labels as well as the image. (5) No. Agreement with an old report does not establish a match to the current saved result. Confirm the result identity, regenerate from it, and check the extracted values against the analysis being documented.