Why an Executive Summary Must Support a Decision
An executive summary is not simply a shorter version of a technical report. It is a compact account of what a decision-maker needs to know before deciding whether to proceed, pause, or ask for more evidence. A reader should be able to identify the decision at stake, the main result, how much confidence the evidence deserves, the recommended next step, and the most important limitation.
A decision-ready summary is direct without pretending the analysis proves more than it does. It does not hide uncertainty behind confident wording, and it does not leave a decision-maker to infer an action from a collection of statistics. The goal is a useful, appropriately qualified conclusion—not a promise that a model will perform perfectly.
The earlier material on decision-first report architecture establishes the value of leading with the decision rather than opening with a catalogue of methods. Here, the focus is the summary itself: how to include the essential elements and make each sentence serve the reader. “Decision” means the choice the reader needs to make in the situation described; a summary should name that choice in ordinary language.
The Five Elements of a Decision-Ready Summary
A useful summary usually covers five elements. They need not appear as five separate paragraphs, but each should be clear enough that a reader can find it.
| Element | What it tells the reader | A useful test |
|---|---|---|
| Decision | What choice or approval is being considered | Could the reader state what they are being asked to decide? |
| Main finding | The most relevant result, expressed in context | Does the sentence say what changed or what the analysis found? |
| Evidence strength | How firmly the result is supported by the available evaluation | Does the wording match the quality and consistency of the evidence? |
| Recommended action | What the organization should do next | Is there a specific, feasible next step? |
| Key caveat | The most important limitation on interpreting or applying the result | Could this limitation change the proposed action or how it is carried out? |
Evidence strength is not a decorative label such as “strong” or “promising.” It should be explained briefly by naming a relevant feature of the evidence: for example, whether the result was evaluated on data not used to fit the model, whether performance was consistent across the main groups considered, or whether a material limitation remains. A single favorable number is not, by itself, a reason to call the evidence strong.
The summary should also distinguish a predictive association from a causal effect. If an analysis finds that a model’s predictions are more accurate in an evaluation, that does not establish that using the model will cause better business outcomes. Avoid verbs such as “will reduce” or “will prevent” unless the evidence actually supports a causal claim.
Build the Summary in a Deliberate Order
A practical drafting process keeps the summary short without dropping essential information. First state the decision in the situation’s own terms. Then report the main finding, including the comparison that gives it meaning. Next explain why the evidence deserves a particular level of confidence. State the recommended action, and end with the caveat most likely to affect how that action should be used.
Say what approval, choice, or next step the reader is considering. Do not start with background or a list of analytical methods.
Explain what the analysis found and what it was compared with. Translate a number into a meaningful difference rather than presenting an isolated score.
Briefly say what supports the finding and what limits confidence. Use restrained terms such as “supports,” “suggests,” or “is not yet conclusive” when those terms fit.
Specify what to do next and the key condition or limitation the decision-maker should keep in view.
A short quantitative comparison can make the main finding clearer. If a model’s error falls from a baseline value \(B\) to a new value \(M\), the absolute reduction is \(B-M\), and the percentage reduction relative to the baseline is \((B-M)/B \times 100\%\), provided \(B>0\). The denominator matters: the percentage describes the reduction relative to the baseline, not relative to the new value.
This expression follows directly from the meaning of a percentage change. The absolute amount removed is \(B-M\). To express that amount as a proportion of the starting value, divide by \(B\); multiply by 100 to convert the proportion to a percentage. This calculation describes the observed comparison. It does not, by itself, establish that the difference is reliable in other settings or that using the model will cause a corresponding financial benefit.
Worked Examples
Worked Example: Recommending a Limited Forecasting Pilot
A hypothetical insurer is deciding whether to test a new method for forecasting monthly claim costs. On a held-out evaluation set of 2,400 policy-months, the average absolute error was $500 for the existing forecast and $420 for the new forecast. The new forecast was less accurate for one small but costly claim category. Draft a decision-ready summary and show the comparison arithmetic.
Method. Compare the new method with the existing forecast, calculate the absolute and relative error reductions, then match the recommendation to the observed limitation.
Step 1: Calculate the absolute reduction. The baseline error is $500 and the new error is $420:
The new forecast had $80 less average absolute error per policy-month in this evaluation.
Step 2: Calculate the relative reduction. Divide the $80 reduction by the baseline error of $500:
The average absolute error was 16% lower than the baseline on the evaluation set. This is a comparison of forecast errors, not evidence that claim costs themselves will fall by 16%.
Step 3: Calibrate the evidence and action. The held-out evaluation supports a favorable overall comparison, but the weaker result in a costly category argues against immediate broad use. A limited pilot with review of that category is more proportionate than an unconditional rollout.
Possible executive summary: “The decision is whether to test the new monthly claim-cost forecast. In a held-out evaluation of 2,400 policy-months, its average absolute error was $420, compared with $500 for the existing forecast—a reduction of $80, or 16%. This supports a limited pilot, rather than immediate broad use. The result was weaker for one small but costly claim category, so that category should be monitored during the pilot.”
Common mistake avoided. Reporting “the forecast is 16% more accurate” is less precise: it does not name the error measure or comparison. Saying “claim costs will fall by 16%” changes an error comparison into an unsupported claim about outcomes.
Worked Example: Describing a Review Flag Without Overclaiming
A hypothetical service team is considering whether to use a model’s flag to prioritize cases for manual review. In a test of 150 flagged cases, reviewers found that 90 needed follow-up. The existing process identified 60 cases needing follow-up among 120 cases it flagged. Draft a summary that compares the proportions, recommends a suitable next step, and names a meaningful caveat.
Method. Calculate the follow-up proportion for each process using its own flagged cases as the denominator, then describe the result as a prioritization signal rather than proof of future savings.
Step 1: Calculate the new flag’s follow-up proportion.
Step 2: Calculate the existing process’s follow-up proportion.
Step 3: Compare the proportions. The difference is \(60\%-50\%=10\) percentage points. The new flag therefore yielded a higher proportion of cases needing follow-up among the cases flagged in this test.
Step 4: Choose a proportionate action. The result may support a monitored trial to help prioritize review. It does not show how many cases needing follow-up were missed, since the problem gives no count of unflagged cases needing follow-up. That missing information matters if the team’s decision depends on avoiding missed cases.
Possible executive summary: “The decision is whether to trial the new flag for prioritizing manual review. In the test, 60% of flagged cases needed follow-up, compared with 50% under the existing process, a difference of 10 percentage points. This supports a monitored trial as a way to prioritize reviewers’ attention, not a claim that outcomes or costs will improve. The test results provided here do not show how many cases needing follow-up were missed, so missed cases should be assessed before relying on the flag as the sole route to review.”
Common mistake avoided. The denominators are the cases flagged by each process, not all cases reviewed or all cases in the population. Also, the 10-point increase should not be described as “10% more” without clarifying the comparison; the two proportions differ by 10 percentage points.
Worked Example: Making an Aggregate Improvement Conditional on Group Results
A hypothetical organization compares average prediction error in two groups. Group A has 800 observations and Group B has 200. The new method’s average errors are $100 in Group A and $300 in Group B. The baseline errors are $120 and $250, respectively. The decision-maker wants to know whether the overall result supports adoption. Calculate the exposure-weighted average error for each method, identify the group-level trade-off, and write a qualified recommendation.
Method. Multiply each group’s average error by its observation count, add the results, and divide by the total count. This gives each observation equal weight in the overall average.
Step 1: Find the total number of observations.
Step 2: Calculate the new method’s overall average error. Group A contributes \(800\times\$100=\$80{,}000\). Group B contributes \(200\times\$300=\$60{,}000\). Total error is \(\$80{,}000+\$60{,}000=\$140{,}000\), so:
Step 3: Calculate the baseline overall average error. Group A contributes \(800\times\$120=\$96{,}000\). Group B contributes \(200\times\$250=\$50{,}000\). Total error is \(\$96{,}000+\$50{,}000=\$146{,}000\), so:
Step 4: Compare overall and group results. Overall, error falls by \(\$146-\$140=\$6\), or \((\$6/\$146)\times100\%=4.1095\ldots\%\), which is 4.11% to two decimal places. But Group A improves by \(\$120-\$100=\$20\), while Group B worsens by \(\$300-\$250=\$50\). The aggregate improvement does not describe every group.
Possible executive summary: “The new method has a lower observation-weighted average error in this evaluation: $140 compared with $146 for the baseline, a reduction of $6, or about 4.11%. The improvement is not consistent across groups: error falls by $20 in Group A and rises by $50 in Group B. Consider a limited adoption only if the Group B result is reviewed and addressed; the overall average alone is not sufficient to support use across both groups.”
Common mistake avoided. Taking the simple mean of the two group errors would give \((\$100+\$300)/2=\$200\) for the new method, which incorrectly gives the 200-observation group the same weight as the 800-observation group. The recommendation also avoids letting a favorable aggregate hide a material worsening in one group.
Common Traps in Summary Writing
The hardest part of a short summary is often deciding what to leave out. Keep information that changes the reader’s interpretation or action; leave technical detail for the body of the report. Avoid these frequent errors:
- Starting with methods instead of the decision. “We fitted three candidate models” may be relevant later, but it does not tell an executive what choice is at hand.
- Giving a number without a comparator. A score is difficult to interpret unless the reader knows what it is being compared with and what a lower or higher value means in context.
- Using jargon as if it were self-explanatory. Replace an unexplained metric or technical label with a short plain-language description of what was measured.
- Claiming certainty from one favorable result. An observed improvement on an evaluation set supports a comparison in that setting; it is not a guarantee of future performance.
- Turning prediction into causation. A model that identifies cases with higher observed risk has not necessarily been shown to reduce that risk when used.
- Adding a generic caveat that does no work. “More research is needed” does not explain what is uncertain. State the limitation that could affect the recommendation.
- Overloading the summary with every caveat. Include the most decision-relevant limitation, and direct the reader to the report for additional detail.
A useful final check is to read the summary without the rest of the report. Can the reader state what decision is being considered, what the analysis found, why the evidence is or is not persuasive, what action is proposed, and what limitation should shape that action? If one answer is missing, revise before polishing the wording.
Check Your Understanding
For each situation, identify the appropriate summary point or calculate the requested comparison before choosing a recommendation.
- A hypothetical routing model has an average error of $270, compared with $300 for the current method, on a separate evaluation set. Calculate the absolute and percentage reduction relative to the current method. (Answer: $30 lower; 10% lower.)
- A hypothetical review flag identifies 48 cases needing action among 80 flagged cases. The current process identifies 42 such cases among 70 flagged cases. What are the two proportions, and what is the difference in percentage points? (Answer: 60% and 60%; 0 percentage points.)
- A new method has lower overall error in an evaluation, but the result is worse in a group that makes up a small share of observations. What should an executive summary do with this fact? (Answer: State the overall result and the group-level worsening, then make any recommendation conditional on investigating or managing that group result.)
- A summary says, “The model will prevent 12% of costly events,” based only on an evaluation showing that its prediction error was 12% lower than the baseline. What is the main flaw, and how should the claim be corrected? (Answer: It converts an error comparison into an unsupported causal claim; say that the model’s measured prediction error was 12% lower than the baseline in the evaluation.)
- A hypothetical method has average errors of $80 for 900 observations in Group A and $200 for 100 observations in Group B. Calculate its observation-weighted overall average error. (Answer: \((900\times\$80+100\times\$200)/1{,}000=(\$72{,}000+\$20{,}000)/1{,}000=\$92\).)