Decision-First Report Architecture

Start With the Decision the Report Must Support

A predictive analytics report is useful only if its readers can connect the analysis to a decision. A technically accurate report can still fail if a busy reader cannot tell what action is being considered, what evidence supports it, or what limits the conclusion. Report architecture is the deliberate ordering and labeling of those elements.

A decision-first structure does not mean announcing a recommendation before explaining its basis. It means making the decision question easy to locate, then arranging the evidence, proposed action, and qualifications so that readers can follow the reasoning. Technical detail belongs in the report when it helps a reader evaluate the conclusion; it should not obscure the path from evidence to action.

Definition: A decision-first report architecture is an organized sequence of sections that makes four elements readily identifiable: the decision under consideration, the evidence relevant to that decision, the recommended action, and the caveats that limit or qualify the conclusion. The order and labels should help readers distinguish what the analysis found from what the report recommends.

A practical architecture often begins with a concise statement of the decision question and the conclusion being considered. It then presents the most decision-relevant evidence, explains the action proposed in light of that evidence, and states the limitations that affect how confidently or broadly the action should be taken. Supporting technical material can follow, or be placed in an appendix, provided the main report gives readers enough context to assess its claims.

Orient the reader.
State what decision is at issue and what conclusion the report reaches or evaluates. Do not make the reader infer the purpose from a model description.

Show the relevant evidence.
Present the results that bear on the decision, with enough context to interpret them and compare alternatives where appropriate.

State the recommended action.
Translate the evidence into an operational proposal. Distinguish a recommendation from a finding: a metric is a result, not itself an instruction.

Make caveats visible.
Explain what the evidence does not establish and what conditions could change the recommendation. Link each caveat to the claim it qualifies.

Provide supporting detail.
Include methods and additional results where they help readers verify or investigate the main argument. Use clear headings and avoid making essential qualifications available only in remote technical material.

Why the Order Matters

Decision-makers often read selectively. A reader may first scan headings, then inspect the conclusion and the evidence most relevant to their responsibility. If the report begins with algorithm terminology or a long sequence of exploratory results, the reader may not know which findings matter. Conversely, putting a recommendation on the first page without showing its evidential basis can make the report appear to ask for trust rather than scrutiny.

A reliable report makes the reasoning traceable: readers can identify the question, see the evidence used to answer it, understand the proposed action, and judge the qualifications. The order is not a rigid template. It is a navigation design. A short report may use a few labeled sections; a long report may use a main narrative plus appendices. In either case, the same four elements should be findable.

The relationship between evidence and action can be made explicit with a simple decision calculation. Suppose an action has a measurable cost under each possible outcome. Under the stated assumptions, its expected cost is the sum of each outcome's probability multiplied by the cost in that outcome. The calculation does not make the decision automatically; it shows what assumptions and trade-offs support it.

E[C\mid a] = \sum_{j=1}^{k} P(O_j\mid a)C_j

Here $a$ is a proposed action, $O_j$ are mutually exclusive outcomes under that action, and $C_j$ is the cost assigned to outcome $O_j$. The expression follows from the definition of an expected value for a discrete outcome: each possible cost is weighted by its probability, and the weighted costs are added. It applies when the outcomes and their probabilities and costs are specified. It does not capture unmodeled consequences, distributional concerns, or uncertainty in the inputs unless those are represented separately. In a report, showing the assumptions beside the calculation helps readers see why the evidence supports a particular action rather than treating a computed result as self-justifying.

Worked Example: Comparing Two Numeric Prediction Models

A hypothetical claims team is deciding which of two models to use in a limited planning pilot. On a four-claim validation sample, the observed amounts are $100, $120, $160, and $200. Model A predicts $110, $110, $150, and $190. Model B predicts $100, $130, $140, and $220. Use mean absolute error (MAE) as the comparison measure, then identify how the evidence, recommendation, and caveat should appear in the report.

Method. For each claim, calculate the absolute difference between the observed amount and the prediction. MAE is the sum of those absolute errors divided by the number of claims.

\operatorname{MAE}=\frac{1}{n}\sum_{i=1}^{n}|y_i-\hat{y}_i|

Model A. The absolute errors are $|100-110|=10$, $|120-110|=10$, $|160-150|=10$, and $|200-190|=10$. Their sum is $10+10+10+10=40$. With $n=4$, its MAE is $40/4=\$10$.

Model B. The absolute errors are $|100-100|=0$, $|120-130|=10$, $|160-140|=20$, and $|200-220|=20$. Their sum is $0+10+20+20=50$. With $n=4$, its MAE is $50/4=\$12.50$.

Report architecture. The decision statement should say that the team is choosing a model for a limited planning pilot. The evidence section should report both MAEs and the validation sample size: Model A has MAE $10, compared with $12.50 for Model B, on four claims. The recommendation can therefore be to use Model A in that limited pilot, because it had lower average absolute error on this sample. A nearby caveat should state that four claims are too few to establish that Model A will perform better in the intended broader use.

Check. Model A's error total is 40 across four claims, giving $10; Model B's error total is 50 across four claims, giving $12.50. These values are consistent with the stated observations and predictions. The common mistake avoided is reporting “Model A is better” without naming the measure, giving the comparison, or limiting the claim to the evidence available.

Make Evidence, Recommendation, and Caveat Distinct

These elements should work together without being blended into one ambiguous claim. Evidence describes what the analysis found. A recommendation specifies what action is proposed. A caveat identifies a boundary on interpretation or use. For example, “the model reduced the validation error” is an empirical claim; “use it to guide the pilot” is a recommendation; “the validation period does not cover an unusual operating condition” is a qualification.

Headings can make these roles explicit. A reader should not have to infer whether a sentence is a measured result, a judgment, or a proposed next step. Tables are useful when alternatives must be compared, but the report should still interpret the table in context. A table of scores without a statement of which differences matter for the decision leaves the reader to supply the reasoning.

Caveats are not a reason to bury a conclusion. They are part of the conclusion's proper scope. Put a qualification near the claim it limits, and repeat it in a conclusion if readers could otherwise mistake a narrow finding for a broad one. General disclaimers at the end are less helpful when they do not identify which result or action they affect.

Worked Example: Choosing a Review Threshold

A hypothetical service team is deciding whether to flag accounts for manual review. On a validation set of 1,000 accounts, 60 later required review under the team's definition and 940 did not. Threshold A flags 100 accounts: 40 of those required review and 60 did not; among the 900 not flagged, 20 required review and 880 did not. Threshold B flags 170 accounts: 50 of those required review and 120 did not; among the 830 not flagged, 10 required review and 820 did not.

The team assigns a cost of $500 to each missed account that required review and $50 to each flagged account that did not require review. For this comparison, assume these are the only costs being counted, and that the stated costs apply equally to each account in the relevant category. Compare the two thresholds using total cost.

Method. A false negative is an account that required review but was not flagged. A false positive is an account that was flagged but did not require review. The total cost under these assumptions is the sum of the false-negative count times its cost and the false-positive count times its cost.

C = 500(\mathrm{FN}) + 50(\mathrm{FP})

Threshold A. Its false-negative count is 20 and its false-positive count is 60. The missed-review cost is $20\times \$500=\$10{,}000$. The unnecessary-review cost is $60\times \$50=\$3{,}000$. Total cost is $\$10{,}000+\$3{,}000=\$13{,}000$.

Threshold B. Its false-negative count is 10 and its false-positive count is 120. The missed-review cost is $10\times \$500=\$5{,}000$. The unnecessary-review cost is $120\times \$50=\$6{,}000$. Total cost is $\$5{,}000+\$6{,}000=\$11{,}000$. Under the stated costs, B is lower by $\$13{,}000-\$11{,}000=\$2{,}000$.

Report architecture. Present the decision as a choice of review threshold. Show the two error counts, the cost assumptions, and the resulting totals before recommending that the team consider Threshold B. State the caveat alongside the recommendation: B creates 60 more false positives than A, and the comparison omits any capacity constraints or costs not included in the assumptions. The lower calculated cost supports consideration of B only under the specified cost model.

Check. The two thresholds' error counts agree with the data: A has $40+20=60$ accounts requiring review and $60+880=940$ not requiring review; B has $50+10=60$ and $120+820=940$. The cost calculations use only false negatives and false positives. A common mistake is to multiply the number flagged by the false-positive cost; flagged accounts include true positives as well as false positives, so only the 60 or 120 false positives receive that cost here.

Design the Main Narrative for Navigation

A report's architecture should let a reader move from the decision question to the evidence without searching through every technical detail. Use descriptive headings rather than generic labels such as “Results” when a more informative heading is possible. For example, “Validation errors by proposed model” tells the reader more about what follows than “Analysis.” Consistent section order across reports also helps recurring readers learn where to find results and caveats.

The main narrative should include evidence needed to evaluate the recommendation, not merely point to an appendix. Supporting detail can include additional tables, calculations, or methods, but the main report should state the central result and explain its relevance. A report that says “see appendix” without summarizing the decision-relevant finding may be complete in a formal sense but difficult to use.

The appropriate level of detail depends on the decision and audience. A compact report might use four sections—decision, evidence, recommendation, and limitations. A more extensive report can add sections for supporting analysis and technical detail. The architecture should remain stable even when the amount of material changes.

Worked Example: Reporting an Overall Result With a Subgroup Qualification

A hypothetical forecasting team compares a model with a benchmark for two customer groups on a validation sample. Group A contains 900 observations: the model's MAE is $10 and the benchmark's MAE is $12. Group B contains 100 observations: the model's MAE is $40 and the benchmark's MAE is $35. The team wants to decide whether to recommend the model for both groups.

Method. For this example, calculate the overall exposure-weighted MAE by multiplying each group's MAE by its number of observations, adding those products, and dividing by the total number of observations. This calculation assumes that each observation receives equal weight and that the two groups together comprise the validation sample.

For the model, Group A contributes $900\times \$10=\$9{,}000$, and Group B contributes $100\times \$40=\$4{,}000$. The total is $\$9{,}000+\$4{,}000=\$13{,}000$. The sample size is $900+100=1{,}000$, so the model's overall MAE is $\$13{,}000/1{,}000=\$13$.

For the benchmark, Group A contributes $900\times \$12=\$10{,}800$, and Group B contributes $100\times \$35=\$3{,}500$. The total is $\$10{,}800+\$3{,}500=\$14{,}300$. Its overall MAE is $\$14{,}300/1{,}000=\$14.30$. Thus the model's overall MAE is lower by $\$14.30-\$13.00=\$1.30$.

Report architecture. The evidence should show the overall comparison and the group-level results together, rather than using the overall value alone. The recommendation should not extend to both groups: the model's MAE is $2 lower than the benchmark in Group A but $5 higher in Group B. A defensible proposal is to consider the model for Group A while withholding a recommendation for Group B pending further assessment. State that the overall result is weighted toward Group A because it has nine times as many observations.

Check. The group counts sum to 1,000. The model's weighted error total is $13,000 and the benchmark's is $14,300, yielding the stated overall MAEs. The common mistake avoided is treating the lower overall MAE as evidence that the model performs better in every group. The caveat belongs next to the overall conclusion because the unequal group sizes explain how the aggregate can conceal a contrary result in Group B.

Common Architecture Traps

Opening with method rather than purpose. A report can briefly orient the reader to the analysis, but a long account of technical steps before the decision question makes the central purpose hard to find.
Turning a metric into an action without reasoning. A lower error score does not by itself determine whether a model should be used. The report must connect the evidence to the proposed use and acknowledge relevant trade-offs.
Separating caveats from the claims they qualify. Readers may encounter the headline conclusion but miss a limitation placed far away. Put important qualifications near the related evidence and recommendation.
Using aggregate evidence as if it applied uniformly. When group results differ, present enough detail to show whether an overall result supports the same action for each group.
Hiding essential evidence in supporting material. Appendices can hold detail, but the main narrative should contain the results required to assess its recommendation.

Key takeaway: Make the decision question, supporting evidence, recommended action, and caveats easy to distinguish and locate. A decision-first architecture does not replace careful analysis; it makes the reasoning and its limits accessible to the people who must use it.

Check Your Understanding

For each situation, identify the report-architecture choice that best helps a decision-maker assess the conclusion.

A report recommends a forecasting model because its validation MAE is $18 rather than a benchmark's $21. The comparison is based on 25 observations from one month. What evidence and caveat should appear near the recommendation? (Answer: Report both MAEs and the 25-observation, one-month basis; qualify the recommendation as limited evidence rather than a general performance guarantee.)
A team compares two review rules. Rule X has 8 false negatives and 30 false positives; Rule Y has 5 false negatives and 70 false positives. The assumed costs are $400 per false negative and $40 per false positive. Calculate the total cost for each rule and say what the report should recommend under only these assumptions. (Answer: X costs $8\times400+30\times40=3{,}200+1{,}200=\$4{,}400$; Y costs $5\times400+70\times40=2{,}000+2{,}800=\$4{,}800$; recommend X under these assumptions, while stating that omitted costs or constraints could affect the choice.)
A model has lower overall error than a benchmark, but the report shows that it performs worse in a small, operationally important group. Should the recommendation rely only on the overall result? (Answer: No. Present the overall and group-level evidence together, explain the group's relevance, and qualify or limit the recommendation accordingly.)
A long report includes a detailed method section, but its recommendation is stated only in the final paragraph and its main results appear in an appendix. Give one structural revision. (Answer: Put a clear decision-oriented opening and labeled evidence and recommendation sections in the main narrative; retain technical detail in supporting sections while summarizing decision-relevant results in the report.)
A report states, “The model reduced average error, so it should be adopted,” while its validation set excluded a major operating condition. Where should that limitation appear, and why? (Answer: Near the performance claim and recommendation, because it limits how broadly the validation result supports adoption.)

What You'll Learn