Handling Partial Report Generation Failures

Why Partial Failures Need Explicit Handling

A diagnostic report may contain several useful sections even when one plot, table, or calculation cannot be produced. For example, a subgroup diagnostic may have no eligible records, or a plotting function may fail while rendering a particular figure. If the workflow stops at the first local problem, it can discard valid results. If it silently carries on, readers may mistake a missing section for a diagnostic that was checked and found unremarkable. Reliable automation must do both: preserve outputs that remain valid and make incomplete work visible.

This tutorial concerns the workflow for representing and responding to failures during report generation. It does not change the diagnostic definitions or interpretations established by the earlier material on diagnostic metrics, subgroup summaries, and diagnostic plots. The key question here is whether an output can still be generated and trusted when another part of the report has a problem.

Section Statuses and Report Status

A report workflow should assign an explicit status to each planned section, rather than treating the presence or absence of a file as its only record. Use status labels whose meanings are stable across runs. The following distinctions are useful:

“Unavailable” and “failed” should not be merged. An empty, validly identified subgroup may make a subgroup diagnostic unavailable without indicating a software defect. A rendering exception is a failure even if the data needed for that figure are present. A skipped section is different again: it may be perfectly executable in isolation, but the workflow deliberately did not run it because doing so could create an invalid result.

At report level, describe whether all planned sections succeeded, whether some sections are incomplete, or whether the workflow could not produce a usable report. A practical vocabulary is complete, partial, and failed. Define the rule in advance. For instance, a report can be marked partial when at least one planned section is unavailable, failed, or skipped and other sections were successfully produced. Reserve failed for a run that cannot produce a usable report under the project’s stated criteria. Do not label a report complete merely because the generation program reached its final line.

Prerequisites Determine Whether Work Can Continue

Treat report generation as a set of tasks with explicit dependencies. A task may require input data, a validated transformation, a model output, or another section’s result. A dependency map makes it possible to distinguish a local failure from a failure that affects several downstream sections.

For example, suppose section A fails. Section B may still be attempted if B’s own prerequisites remain valid and B does not require A’s output, unless a valid fallback for A’s output has been explicitly defined. This is a necessary safety check, not a guarantee that B will succeed: B can still encounter its own error. Equally, the fact that A and B do not consume each other’s outputs is not enough to justify continuation. They may share a failed input or transformation, making both outputs invalid. Check shared prerequisites before continuing.

A compact validity rule makes the logic explicit. Let \(V_j\) mean that task \(j\)'s output is valid, \(E_j\) mean that its own execution and checks succeed, and \(P(j)\) be its required prerequisites. Then:

If a prerequisite is replaced by a fallback, count it as valid only if the fallback is appropriate for that task and its use is recorded. A task that runs successfully with invalid inputs does not have a valid output. This distinction is why catching an error and moving to the next line is not, by itself, robust failure handling.

A Practical Failure-Handling Sequence

Error handling should be specific enough to preserve diagnostic information. A broad handler that catches every error and returns an empty plot can make a software failure look like a legitimate plot with no data. Instead, retain an informative error message and identify the affected task. Avoid including sensitive data in logs; record what is needed to diagnose the failure without copying unnecessary subject-level information.

Shared-Prerequisite Failures and Dependency Propagation

A failure in a shared prerequisite can affect several sections even if those sections are otherwise independent. For example, if residuals and prediction errors both require a validated unit conversion, a failure in that conversion prevents both outputs from being trusted. A shrinkage summary derived from a separate, valid model-output file might remain possible, but only after confirming that it does not rely on the failed conversion.

This dependency check is especially important after data validation or transformation failures. An upstream issue may have a wider scope than the first task that reports it. Update the status of every affected task, and do not allow later tasks to reuse stale results from a previous run as if they belonged to the current run.

Make Partial Outputs Safe to Read

A technically successful run can still be misleading if the report hides its incomplete status. Make the state visible near the report’s start and at the affected sections. For each section, record its status and a short, specific reason when it is not successful. A reader should be able to distinguish “not attempted because a required input failed” from “attempted but the rendering operation failed” and from “no eligible data were available.”

Write outputs in a way that avoids exposing half-finished artifacts as complete. One useful pattern is to generate a section to a temporary location, run its checks, and make it available to the report only after those checks pass. If generation fails, retain the failure record and do not leave a partial artifact in the position normally used for a validated section. This does not require discarding other sections that passed their own checks.

Diagnostics, Alternatives, and Limitations

Useful checks include comparing the planned section list with the recorded outcomes, confirming that every non-success status has a reason, and verifying that successful sections have usable outputs. Check that a report marked complete has no failed, unavailable, or skipped planned sections under the project’s defined rule. Also check that no downstream section was generated from a failed prerequisite and that artifacts from earlier runs were not mistaken for current outputs.

Two tempting alternatives are unreliable. Stopping the entire run at the first error is simple, but can unnecessarily discard unaffected outputs. Silently suppressing errors and continuing is also simple, but can conceal invalid or missing diagnostics. A better approach uses scoped error handling and dependency-aware continuation: preserve valid work, record the problem, and stop or skip only what the failure actually affects. If a failure concerns a shared prerequisite or the validity of the report as a whole, broader interruption may be necessary.

Partial generation has limits. A partial report must not be used as if it were complete, and a successful section does not repair an invalid shared input elsewhere. Status labels and logged errors support interpretation and follow-up; they do not establish that the underlying model or diagnostic is scientifically adequate. Define in advance which failures make a report unusable and which allow a clearly labeled partial report.