Choosing a Target That Matches the Decision

Why Target Choice Matters

A prediction is useful only in relation to a decision. If an organization wants to prevent equipment failures, a model that predicts which machines will receive maintenance may be accurate at forecasting past maintenance practices yet fail to identify which machines are at risk of failure. The predicted label and the decision objective are not automatically the same thing.

Earlier material on translating business questions into predictive tasks and connecting predictions to intended actions provides the decision context. This tutorial focuses on a further choice: which observable outcome should the model predict? The answer should represent the result the decision is meant to improve, not merely a variable that is easy to collect or strongly associated with that result.

Definition: A decision objective is the result a decision-maker wants to achieve or avoid. A target is the observable outcome the predictive task is designed to predict. A proxy is a different observable measure used as an indirect stand-in for the objective or target. A proxy may be informative, but it is not automatically equivalent to the outcome of interest.

The target does not have to be identical to the objective. An objective may be expressed in terms of cost, service quality, or another broader goal, while the target is a measurable event that is closely connected to it. The key question is whether differences in the target correspond to differences that matter for the decision. If not, a model can perform well at predicting its target while offering poor guidance for the intended decision.

A Decision-Based Test for Target Alignment

A practical way to assess target alignment is to ask what would happen if the model predicted the target perfectly. Would that prediction help the decision-maker choose an action that advances the stated objective? If not, the target may be a proxy that is too remote from the decision.

This can be made precise using expected loss. Suppose the decision-maker chooses an action $a$, the relevant outcome is $Y$, and $L(a,Y)$ is the loss associated with taking action $a$ when outcome $Y$ occurs. Assume the available actions and outcomes are defined, the conditional probabilities of outcomes are meaningful for the decision, and the loss values adequately represent the decision-maker’s priorities. Given information $X$, the action with the smallest conditional expected loss is preferred:

a^*(X)=\arg\min_a E[L(a,Y)\mid X] =\arg\min_a\sum_y L(a,y)P(Y=y\mid X).

The formula follows from the definition of expected loss: for each possible outcome $y$, multiply its loss under action $a$ by the probability of that outcome, then add across outcomes. The action with the smallest sum has the lowest average loss under the stated probabilities and loss values. If a model predicts a proxy $Z$ rather than $Y$, the proxy is decision-sufficient only when it preserves the information needed to choose among actions. Mere association between $Z$ and $Y$ does not guarantee that.

Worked Example: Predicting Pump Failure Rather Than a Vibration Flag

A hypothetical water utility must decide whether to replace a pump now or defer replacement until the next scheduled service. For this example, assume replacement costs $2,000, a failure during the operating cycle costs a fixed $10,000, and deferring replacement costs nothing if the pump does not fail. A vibration indicator is available, and estimated failure probabilities are 0.30 for pumps with a high reading and 0.05 for pumps with a low reading. Identify the decision-aligned target and choose the lower-cost action for each group.

Step 1: State the objective and target. The objective is to minimize expected cost. The directly relevant outcome is whether the pump fails during the operating cycle. The vibration reading is a possible proxy: it may help predict failure, but it is not the failure outcome itself.

Step 2: Calculate expected costs for a high-reading pump. Replacing costs $2,000 regardless of whether failure would otherwise occur. Deferring has expected cost $0.30\times \$10{,}000 + 0.70\times \$0 = \$3{,}000$. Since $2,000 is less than $3,000, replace the pump under these assumptions.

Step 3: Calculate expected costs for a low-reading pump. Replacing still costs $2,000. Deferring has expected cost $0.05\times \$10{,}000 + 0.95\times \$0 = \$500$. Since $500 is less than $2,000, defer replacement under these assumptions.

Check. The expected costs are $2,000 versus $3,000 for the high-reading group and $2,000 versus $500 for the low-reading group. The choices minimize the stated expected costs. The conclusion depends on the assumed failure probabilities and fixed failure cost; it would need reconsideration if those assumptions changed.

Method and trap avoided. Compare the expected losses of the available actions using failure as the target. A common mistake is to treat “high vibration” as the outcome to predict simply because it is easy to record. That would answer which pumps have a high vibration reading, not which pumps are likely to fail.

When a Proxy Helps—and When It Misleads

Directly measuring the objective may be difficult, delayed, or expensive. A proxy can still be useful when it is available more readily and provides reliable information about the outcome relevant to the decision. But the proxy’s usefulness must be judged against the decision, not only by whether it is associated with the objective.

Ask what the decision is intended to change. Identify the result that would count as success or failure from the decision-maker’s perspective.
Identify the observable outcome closest to that result. The target should reflect the relevant outcome as directly as the available data allow.
Check what the proxy actually measures. A process measure, staff action, or preliminary signal may reflect how an organization operates rather than the underlying outcome.
Look for important omissions. A proxy may fail to represent severity, costs, or cases that do not pass through a particular process.
State the limitation clearly. If a proxy is used, describe it as a proxy and explain what additional evidence would be needed to establish its decision value.

Target alignment also depends on the decision-maker’s priorities. For example, a model intended to reduce operating cost may need a target connected to cost, while a model intended to improve a service experience may need a different target. There is no universally correct label independent of the decision context.

Worked Example: Prioritizing Claims for an Audit

A hypothetical insurer can audit 20 claims this month. An audit costs $100 per claim, and the insurer expects to prevent or recover $1,000 for each claim with a material overpayment. Historical audit records show 20 complex claims, of which 8 had a material overpayment, and 80 simple claims, of which 12 had a material overpayment. A referral flag is often attached to complex claims. Identify the appropriate target and assess the two groups using expected net value per audit.

Step 1: Identify the decision and objective. The decision is which claims to audit. Under the stated assumptions, the objective is to prioritize audits with the greatest expected net value. The direct target is whether an audit confirms a material overpayment. The referral flag is a proxy for that outcome, not proof that an overpayment occurred.

Step 2: Calculate the observed overpayment rates. Among complex claims, the rate is $8/20=0.40$. Among simple claims, it is $12/80=0.15$. There are $8+12=20$ confirmed overpayments among $20+80=100$ claims overall, an overall rate of $20/100=0.20$.

Step 3: Calculate expected net value per audit. For a complex claim, expected recovery is $0.40\times \$1{,}000=\$400$. Subtract the $100 audit cost to obtain expected net value of $300. For a simple claim, expected recovery is $0.15\times \$1{,}000=\$150$. Subtract the $100 cost to obtain expected net value of $50.

Step 4: Apply the capacity limit. There are exactly 20 complex claims and capacity for 20 audits. Using the observed group rates and the stated assumptions, auditing those 20 claims has expected recovery $20\times \$400=\$8{,}000$, audit cost $20\times \$100=\$2{,}000$, and expected net value of $6,000. This is higher than the $1,000 expected net value from auditing 20 simple claims at $50 each.

Check. The group rates reproduce the total: $20\times 0.40+80\times 0.15=8+12=20$ expected overpayments, matching the 20 observed in the data. The result is based on the given historical rates; it does not establish that every complex claim has an overpayment.

Method and trap avoided. Use confirmed overpayment as the target, then compare expected net value. Treating the referral flag itself as the target would make the model predict referral practice rather than the outcome the audit decision is meant to find. Also, a historical rate does not by itself show that the flag causes an overpayment.

Worked Example: Using Satisfaction as a Proxy for Repeat Contact

A hypothetical service team wants to reduce repeat calls after issues have been resolved. In a review of 200 resolved calls, 50 received a low satisfaction rating and 150 received a high rating. Among the low-rated calls, 20 were followed by a repeat call; among the high-rated calls, 30 were followed by a repeat call. The team can follow up on 50 cases. Assess satisfaction as a proxy for repeat contact and calculate how many repeat callers the low-rating group contains.

Step 1: Choose the target. The stated objective concerns repeat contact, so whether a repeat call occurs is the target. Satisfaction is a potentially useful proxy, but it measures a different outcome.

Step 2: Calculate repeat-contact rates. The low-rating group has $20/50=0.40$ repeat contacts. The high-rating group has $30/150=0.20$. Overall there are $20+30=50$ repeat contacts among 200 calls, or $50/200=0.25$.

Step 3: Assess the 50-case follow-up rule. If the team selects all 50 low-rated cases, it includes 20 repeat callers and misses the other 30. Thus, it captures $20/50=0.40$, or 40% of the observed repeat callers. A simple random selection of 50 cases would contain $50\times 0.25=12.5$ repeat callers on average, so the proxy identifies a group with a higher observed rate than the overall rate.

Check. The repeat counts add to $20+30=50$, and the group sizes add to $50+150=200$. The proxy identifies more repeat callers than random selection on average under these data, but it still misses 60% of them.

Method and trap avoided. Compare the proxy groups against the direct target and quantify both the rate and the missed cases. A common mistake is to call low satisfaction “repeat contact” or to assume that following up on low-rated cases will prevent repeat calls. The data show an association, not that the follow-up action causes an outcome to change.

Common Target-Selection Traps

Choosing what is easiest to extract. Ease of measurement can make a proxy tempting, but convenience alone does not make it decision-aligned.
Confusing a process measure with an outcome. A referral, review, or intervention can reflect an existing workflow rather than the result the decision-maker cares about.
Assuming correlation is enough. A proxy can be associated with the objective and still miss cases, rank cases poorly for the decision, or reflect factors unrelated to the objective.
Ignoring the consequences of errors. If the decision has different costs for different outcomes, a target that omits those distinctions may not support the intended choice.
Claiming more than the target supports. Predicting an outcome does not, by itself, show that taking an action will cause that outcome to improve.

For a target to be defensible, describe it in plain language and connect it explicitly to the decision objective. If an indirect proxy must be used, say what it measures, what it omits, and why it is still expected to help with the decision. This discipline is aligned with the SOA Exam PA (Predictive Analytics) syllabus and supports clear communication about what a model can—and cannot—answer.

Key takeaway: Select the target by starting with the intended decision objective and asking which observable outcome most directly represents it. A proxy may help, but its association with the objective is not evidence that the two are interchangeable.

Check Your Understanding

For each situation, identify the target that best represents the stated objective and use the given figures where needed.

A transit agency wants to reduce missed trips. It can record whether a bus departed late and whether a passenger complaint was filed. Which is the more direct target for the stated objective? (Answer: Whether a scheduled trip was missed; a complaint is a proxy.)
A hypothetical clinic wants to reduce unplanned return visits. Of 40 patients with a low follow-up satisfaction score, 12 returned; of 160 with a high score, 24 returned. What are the return rates in the two groups, and what should be the target? (Answer: 12/40 = 0.3000 and 24/160 = 0.1500; the target is whether an unplanned return visit occurs.)
A hypothetical repair service pays $80 to inspect a device and avoids an expected $500 loss if a defect is found. A group has a 0.25 defect rate. What is the expected net value per inspection, and is defect status a more direct target than the number of inspection requests? (Answer: $0.25\times \$500-\$80=\$45$; yes, defect status is more direct.)
A hypothetical team can contact 30 customers. There are 60 customers rated “at risk,” of whom 18 later cancel, and 140 others, of whom 14 later cancel. How many cancellations are in the full data, and what fraction would selecting all 30 at-risk customers capture if selected customers were representative of that group? (Answer: 32 cancellations total; $30\times 18/60=9$, which is $9/32=0.2813$ of cancellations.)
A hypothetical lender wants to reduce loan losses. A model predicts whether a loan officer referred an application for review, and referrals are strongly associated with subsequent loss. What is the more decision-aligned target, and why is the referral label not automatically equivalent? (Answer: A clearly defined loan-loss outcome; referral records a review process and may be associated with loss without being the loss itself.)

What You'll Learn