From Model Results to a Decision
A report can present accurate metrics and still leave a decision-maker unsure what to do. A recommendation closes that gap. It identifies a decision, explains which reported evidence bears on it, makes relevant practical trade-offs explicit, and proposes a next action. Its strength depends on keeping the analysis distinct from the assumptions and priorities used to choose among actions.
For example, a model may identify cases with higher predicted risk. That finding alone does not establish that an intervention will reduce risk, that the organization has capacity to offer it to every identified case, or that its cost is worthwhile. A well-formed recommendation says what the model supports, what additional judgment is being applied, and what action can test or implement the proposal.
A useful recommendation can usually be read as a short chain: decision → evidence → implication → trade-off → next action. The evidence should be concrete enough to check against the report, while the action should be specific enough to assign or approve. Avoid presenting a preference as if it were a model result.
Specify the choice, such as whether to pilot a review process, how to allocate a fixed number of staff hours, or whether to use a forecast as a planning input.
Report the result that bears on that choice, with its population, period, and operating condition clear enough for the reader to interpret it.
Translate the result into expected workload, captured cases, prediction error, or another consequence relevant to the decision.
Say what the proposed option gains and gives up, then identify a proportionate action, owner, or review point.
Separate Evidence, Assumptions, and Judgment
Three kinds of statements often appear close together in a recommendation, but they are not interchangeable. Evidence is what the analysis reports under its stated evaluation setup. An assumption supplies a quantity or condition not established by that analysis, such as the cost of a review or the benefit of a successful intervention. Judgment applies priorities to the evidence and assumptions, such as deciding that current staffing capacity is more important than capturing every possible case.
This distinction is especially important when a recommendation moves from prediction to action. A model may estimate who is more likely to experience an outcome; that is not, by itself, evidence that contacting those people will change the outcome. If an economic calculation uses an assumed intervention effect, label it as an assumption and do not attribute it to the model.
One simple way to make the distinction precise is to calculate an illustrative incremental value under stated assumptions. Suppose each selected case costs \(c\) to act on, there are \(n\) selected cases, and each correctly selected case produces an assumed benefit \(b\). If the action benefits only correctly selected cases and every selected case incurs the same cost, then:
The expression follows by adding the benefit across the correctly selected cases and subtracting the action cost across all selected cases. It does not prove that the assumed benefit will occur. If costs differ, benefits extend to other cases, or the action creates other consequences, this simple expression is not sufficient; the relevant terms must be specified before it is used.
Worked Example: Recommend a Limited Outreach Pilot
A hypothetical insurer considers offering a loss-prevention call to policyholders flagged by a risk model. On a validation set of 1,000 policies, the proposed operating threshold flags 120. Of those 120, 72 experienced the defined outcome and 48 did not. Among the 880 unflagged policies, 28 experienced the outcome and 852 did not. The organization has capacity for 120 calls. For a planning calculation only, managers assume that each correctly targeted call would produce an incremental benefit of $80 and that every call costs $20. Form a recommendation and show what the evidence does and does not establish.
Method. First calculate the relevant classification results from the supplied counts, then calculate the illustrative value using the explicitly supplied economic assumptions. Finally, choose an action that does not treat the assumed benefit as a model finding.
The number of outcome cases is \(72 + 28 = 100\). The number without the outcome is \(48 + 852 = 900\). The flagged count is \(72 + 48 = 120\), which exactly matches the stated call capacity.
The proportion of flagged policies that experienced the outcome is \(72/120 = 0.6000\), or 60.00%. The proportion of outcome cases flagged is \(72/100 = 0.7200\), or 72.00%. These are different quantities: one describes the flagged group, while the other describes how many observed outcome cases were captured.
Under the planning assumptions, the benefit for correctly targeted calls is \(72 \times \$80 = \$5{,}760\). The cost of all calls is \(120 \times \$20 = \$2{,}400\). The illustrative net value is \(\$5{,}760 - \$2{,}400 = \$3{,}360\).
Recommendation. Consider a controlled outreach pilot at this threshold, using the available capacity, and evaluate its operational and outcome results before expanding. The validation counts show that the rule identified 72 of the 100 observed outcome cases in this evaluation and fits the stated call capacity. The positive illustrative value depends on the assumed $80 benefit; the model evaluation does not establish that calls will prevent losses. Treating $3,360 as a demonstrated saving would be the common mistake this calculation is designed to avoid.
Match the Recommendation to the Decision Constraint
The same analytical result can support different actions under different constraints. A decision-maker may be limited by staff capacity, budget, acceptable missed-case levels, or the cost of acting on an incorrect prediction. State the constraint that matters rather than implying that one metric determines the decision for every organization.
A useful recommendation makes the trade-off visible in ordinary operational terms. Instead of saying that one option is “better,” explain what it yields and what it requires. Where the trade-off depends on a chosen priority—for example, capturing more cases at the cost of additional reviews—identify that priority as a decision, not a statistical conclusion.
Worked Example: Choose Between Two Review Workloads
A hypothetical public-benefit agency evaluates two model-based queues on the same 500 historical cases. There were 40 cases with the outcome of interest and 460 without it. Queue A flags 50 cases: 30 had the outcome and 20 did not. Queue B flags 80: 36 had the outcome and 44 did not. The agency can review at most 60 cases per period. Compare the queues and recommend an action under this capacity limit.
Method. Compare captured outcome cases and review workload, then check feasibility against the stated capacity. Do not choose a queue from its capture rate alone.
Queue A captures \(30/40 = 0.75\), or 75% of the outcome cases, and requires 50 reviews. Queue B captures \(36/40 = 0.90\), or 90%, and requires 80 reviews. Queue A fits the 60-review limit; Queue B exceeds it by \(80 - 60 = 20\) reviews.
Moving from A to B would capture \(36 - 30 = 6\) additional outcome cases while requiring \(80 - 50 = 30\) additional reviews. That is \(30/6 = 5\) additional reviews per additional captured case in this comparison. This ratio describes the observed difference between these queues; it does not establish the value of a review or the consequences of a missed case.
Recommendation. If the 60-review limit is fixed, pilot Queue A as the feasible option and report that it captured 30 of 40 outcome cases in this evaluation. Queue B captured six more but did not meet current capacity. The common mistake is to recommend B solely because its capture rate is higher, without checking whether the organization can perform the required 80 reviews. If capacity changes, the choice can be reconsidered using the decision-maker’s priorities and the new constraint.
Use Prediction Errors to Set the Scope of a Recommendation
A recommendation about a numeric forecast should reflect both its performance and the way the forecast will be used. If a forecast is being considered for aggregate planning, an aggregate result may be relevant; if it will allocate resources among locations, local errors also matter. A favorable overall total does not automatically justify every use of the individual predictions.
For actual values \(y_j\) and predictions \(\hat{y}_j\) across \(n\) observations, mean absolute error is the average of the absolute errors, and root mean squared error is the square root of the average squared error. Absolute error records the typical size of deviations in the outcome’s units. Squaring gives larger deviations more influence in RMSE. Neither measure, by itself, specifies what error is acceptable for an operational decision.
Worked Example: Use a Forecast Without Overstating Its Local Accuracy
A hypothetical service plans staffing in four districts. For a held-out evaluation period, observed demand was 100, 120, 80, and 100 service units; the forecasts were 110, 115, 70, and 105 units. The planning team is considering whether to use the forecasts as a district-level allocation or only as an aggregate planning reference. Calculate the errors and form a recommendation.
Method. Calculate each forecast error as prediction minus observed value, then calculate MAE and RMSE. Compare the district-level results with the aggregate totals before recommending a scope of use.
The errors are \(110-100=10\), \(115-120=-5\), \(70-80=-10\), and \(105-100=5\) units. Their absolute values are 10, 5, 10, and 5, so the total absolute error is \(10+5+10+5=30\). Thus, MAE is \(30/4=7.5\) units.
The squared errors are \(10^2=100\), \((-5)^2=25\), \((-10)^2=100\), and \(5^2=25\). Their sum is \(100+25+100+25=250\). The mean squared error is \(250/4=62.5\), so RMSE is \(\sqrt{62.5}\), approximately 7.9057 units. Rounded to one decimal place, RMSE is 7.9 units.
The observed total is \(100+120+80+100=400\) units. The forecast total is \(110+115+70+105=400\) units, so the aggregate totals match in this evaluation even though the district errors are not zero.
Recommendation. The results support considering the forecast as an aggregate planning reference, while testing district-level allocations before relying on them operationally. The total matches in this evaluation, but the four district errors range from 5 to 10 units in absolute value. The common mistake is to treat a matching total as proof that the district forecasts are accurate; the calculations show that errors can offset when aggregated.
Write the Recommendation So It Can Be Acted On
A decision-maker should not have to reconstruct the logic from a set of charts. State the proposed action near the beginning, then give the evidence that supports it and the practical trade-off that informs it. Identify the next step in concrete terms: for example, approve a bounded pilot, assign a team to test a workflow, or use a forecast for a specified planning purpose. If an approval, capacity, or assumption is needed first, say so.
The action should be proportionate to what the evidence establishes. A pilot may be appropriate when a result is promising but an operational effect is still assumed. A bounded use may be appropriate when aggregate performance is more persuasive than local performance. A comparison may be inconclusive when the preferred option depends on a value judgment that has not been made explicit.
This discipline supports recommendations aligned with the SOA Exam PA (Predictive Analytics) syllabus: connect the analytical result to a decision, keep interpretation faithful to the reported evidence, and make the practical reasoning visible. A concise recommendation is not one that omits the reasoning; it is one in which the reasoning is easy to follow.
Check Your Understanding
For each case, connect the reported evidence to a decision and distinguish the analytical result from any operational assumption.
- A hypothetical outreach model flags 40 of 240 cases. Of the flagged cases, 24 experienced the outcome; there were 30 outcome cases in total. Calculate the flagged-group outcome proportion and the proportion of outcome cases captured. If capacity is 40 contacts, what workload conclusion follows? (Answer: The flagged-group proportion is \(24/40=0.6000\); the captured proportion is \(24/30=0.8000\); the queue fits capacity exactly.)
- For three districts, actual demand is 50, 60, and 40 units, while forecasts are 55, 54, and 41. Calculate MAE, RMSE to two decimals, and the two aggregate totals. (Answer: Errors are 5, \(-6\), and 1; MAE is \(12/3=4\) units; RMSE is \(\sqrt{62/3}\approx4.55\) units; both totals are 150 units.)
- A review queue captures 28 of 35 outcome cases and flags 40 cases. A second queue captures 31 of 35 and flags 52. Capacity is 45 reviews. Which queue is feasible, and how many additional reviews does the second queue require for its additional captured cases? (Answer: The first queue is feasible; the second exceeds capacity by 7. It captures 3 additional cases at the cost of \(12/3=4\) additional reviews per additional captured case.)
- A hypothetical intervention plan assumes each correctly targeted case yields $70 in incremental benefit. It proposes 50 contacts at $15 each and, based on the evaluation counts, expects 18 correctly targeted cases. Calculate illustrative net value and state one essential distinction for the recommendation. (Answer: Assumed benefit is \(18\times\$70=\$1{,}260\); contact cost is \(50\times\$15=\$750\); illustrative net value is $510. The $70 benefit is an assumption, not a benefit established by the predictive evaluation.)
- A forecast has equal predicted and observed totals across several locations, but individual location errors are 8, \(-8\), 6, and \(-6\) units. What does the matching total establish, and what does it not establish? (Answer: The total forecast equals the total observed value for these locations; it does not establish that the location-level forecasts are accurate, because the nonzero errors offset.)