Why a Score Needs an Interpretation
A model can return a number for each case, but that number is useful only if its meaning is clear. A value of 0.12 might be a predicted probability, a numeric forecast in a particular unit, or a score used only to order cases. Those interpretations are not interchangeable. In a business setting, misunderstanding the scale can lead someone to treat a model output as a certainty or as an instruction.
A score should be read in the context of the predictive task: what outcome is being predicted, for which unit, using information available at what time, and over what horizon. Those details give the score its meaning. Even with that context, a prediction describes an outcome under the model; it does not by itself establish why the outcome will occur or what action should follow.
Predicted Probabilities Describe Uncertain Events
Suppose a model predicts whether a binary event will occur for a particular case during a defined future period. If the model output is a probability, write it as \(p_i\) for case \(i\). Its intended meaning is the model's estimate of the chance that the defined event occurs for that case, conditional on the information supplied to the model. For example, a predicted probability of 0.1200 means a model-estimated chance of 12.00% for the specified event and horizon.
A probability is not a promise about an individual case. The event either occurs or does not occur, while the score represents uncertainty before the outcome is known. If many comparable cases receive probabilities near 0.1200, those scores can be understood as indicating that the model estimates a frequency near 12 events per 100 such cases, on average, if its estimates are appropriate for that group and setting. It does not follow that exactly 12 of every 100 cases must experience the event.
The event definition matters. “Claim” might mean at least one claim during a specified year, rather than the number of claims or the dollar amount paid. A score for one definition cannot be silently reinterpreted as a score for another. Similarly, a probability for a one-year horizon does not automatically describe the probability over a different horizon.
Worked Example: Reading a Predicted Event Probability
A hypothetical service team scores 250 customer accounts for whether each account will close during the next 12 months. The model returns a predicted probability of 0.1200 for every account. Interpret the individual score and calculate the expected number of closures across the 250 accounts. Also calculate the standard deviation of the total number, assuming the account outcomes are independent Bernoulli variables with the stated probabilities.
Method. Treat each account's closure indicator as a Bernoulli random variable. For each account, the probability of closure is \(p=0.1200\), and the probability of no closure is \(1-p=0.8800\). The model's individual score is a probability estimate for the defined 12-month event, not a prediction that 12% of an account will close.
Expected count. Let \(I_j=1\) if account \(j\) closes during the period and \(I_j=0\) otherwise. The total number of closures is \(N=\sum_{j=1}^{250} I_j\). The expected count is \(250(0.1200)=30\). Thus, the model-implied expected number of closures is 30 accounts.
Variability under the stated independence assumption. A Bernoulli variable with probability \(p\) has variance \(p(1-p)\). Therefore, \(\operatorname{Var}(N)=250(0.1200)(0.8800)=26.4\). The standard deviation is \(\sqrt{26.4}=5.138\), or 5.14 accounts to two decimals.
Interpretation and check. The expected count is 30, but it is not a guarantee that exactly 30 accounts will close. The variance calculation uses the additional independence assumption; the expected count does not require independence. The common mistake is to report “30 accounts will close.” The correction is “the model-implied expected count is 30, with outcomes still uncertain.”
Predicted Amounts Depend on the Target's Meaning
A numeric prediction also needs a precise interpretation. If a model predicts a mean amount, the output is an estimate of the average target value for cases with the supplied information, under the model. It is not necessarily the amount that will be observed for any one case. A realized amount can be higher or lower, and the model's target may be conditional on an event.
For example, a severity model may predict the expected payment conditional on a claim occurring. That is different from predicting expected payment per policy, including policies with no claim. If \(C\) indicates whether a claim occurs and \(Z\) is the payment if it occurs, the payment per policy can be represented as \(L=CZ\), with \(L=0\) when no claim occurs. Under the assumptions that at most one claim occurs per policy and that \(Z\) is defined conditional on a claim, the expected payment is:
This formula does not say that the expected payment is what every policy will cost. It combines an event probability with a conditional mean amount to obtain an average over the full policy population. If the amount target instead already includes zero-payment cases, multiplying that prediction by the event probability again would count the event probability twice.
Worked Example: Combining an Event Probability and a Conditional Amount
A hypothetical insurer models one-year claim costs for a group of 400 policies. For each policy, the estimated probability of at least one claim in the year is 0.0600. The estimated mean payment conditional on a claim is $1,800.00. Assume at most one claim per policy, no payment without a claim, and that the conditional mean applies to this group. Find the expected payment per policy and the expected total payment for the group.
Method. The probability is for the occurrence of a claim; the $1,800.00 figure is the mean payment conditional on a claim. Apply the event-probability-times-conditional-mean formula to obtain the mean payment per policy.
Per-policy calculation. The probability of no claim is \(1-0.0600=0.9400\). The payment is $0.00 in that case. The expected payment per policy is \(0.0600\times \$1{,}800.00=\$108.00\).
Group calculation. For 400 policies, the expected total payment is \(400\times \$108.00=\$43{,}200.00\). Equivalently, the expected number of claims is \(400\times0.0600=24\), and \(24\times\$1{,}800.00=\$43{,}200.00\).
Interpretation and check. The group expected payment is $43,200.00, not a guaranteed total. The equivalent calculation agrees, which checks the arithmetic. A common mistake is to call $1,800.00 the expected payment per policy. It is conditional on a claim; including policies with no claim reduces the expected payment per policy to $108.00.
A Ranking Score Gives Relative Order
Some models return a score intended to sort cases rather than estimate an outcome on a probability or dollar scale. A larger score may mean that the model places a case earlier in an ordering according to its prediction. Unless the score is specifically defined and validated as a probability or amount, its value should not be given those interpretations.
Ranking is about relative position. If scores are 88 and 72, the model orders the first case above the second; the difference of 16 score points does not necessarily represent a 16-percentage-point difference in event probability or a fixed change in expected cost. A transformation that preserves order can change the numerical distances without changing the ranking. Ties also need to be recognized: equal scores do not establish an ordering between the tied cases.
Worked Example: Interpreting a Relative Priority Score
A hypothetical review team receives a model output on a 0-to-100 priority scale for five cases. The scores are: Cedar, 72; Maple, 88; Birch, 88; Aspen, 64; and Elm, 79. The team wants the cases listed from highest to lowest model priority. State the ordering, identify any tie, and explain what cannot be inferred from the scores.
Method. Treat the values as ranking scores, as specified, and sort them in descending numerical order. Do not convert them into probabilities.
Ordering. The two highest scores are Maple and Birch, both at 88. Elm follows at 79, Cedar at 72, and Aspen at 64. Using competition ranks, Maple and Birch share rank 1; Elm is rank 3, Cedar is rank 4, and Aspen is rank 5.
Interpretation and check. The model places Maple and Birch jointly ahead of Elm, Cedar, and Aspen, but it does not distinguish between the tied cases. A score of 88 on this scale does not mean an 88% chance of an event. Nor does the gap between 88 and 79 establish how much more likely an outcome is for Maple or Birch than for Elm. The common mistake is to read the score's numerical appearance as a probability. The correction is to use only the stated ranking interpretation.
From Score to Decision Input
A score can inform a decision process, but it is not itself a decision. A probability may help a decision-maker consider how likely an event is; an amount forecast may help quantify an expected scale; and a ranking score may order cases for consideration. What action is appropriate depends on the decision context and any applicable rules, costs, constraints, and human review. A score alone does not supply all of that information.
Before interpreting or communicating a score, check the following:
- Target: What exact event or numeric quantity does the model predict?
- Unit and horizon: Is the score for a policy, claim, person, or another unit, and what period does it cover?
- Scale: Is the value a probability, a predicted amount, or a relative ranking?
- Conditioning: Does an amount apply to all cases or only to cases where an event occurs?
- Decision role: What question can the score inform, and what other information is needed before an action is chosen?
A useful communication states both what a score means and what it does not mean. For example: “The model estimates a 0.0600 probability of at least one claim during the next year; this is an uncertain forecast, not a claim decision.” That wording preserves the event definition, horizon, and uncertainty. For a ranking output, say that a case is ordered above another by the model, rather than describing it as a specific probability unless that is truly how the score is defined.
Check Your Understanding
For each hypothetical case, identify the score's meaning and complete the requested calculation. Keep the interpretation tied to the stated target and assumptions.
- A model assigns each of 80 accounts a 0.0250 probability of cancellation in the next quarter. Calculate the expected number of cancellations and state whether that count is guaranteed. (Answer: Expected count = 2; it is not guaranteed.)
- A risk model estimates a 0.0400 probability of an event during one year and a conditional mean payment of $3,000.00 if it occurs. Assuming no payment without the event, calculate expected payment per exposure. (Answer: $120.00.)
- A triage model produces ranking scores of 70, 95, 85, and 95 for cases K, L, M, and N, respectively. Give the descending order and state what the tie means. (Answer: L and N tie at the highest score, followed by M, then K; the score does not order L relative to N.)
- Two hypothetical cases have event probability and conditional mean payment as follows: Case A has probability 0.1000 and mean payment $500.00; Case B has probability 0.0400 and mean payment $900.00. Assuming payment is zero without the event, calculate expected payment for each and identify which has the larger expected payment. Does that alone determine an action? (Answer: A = $50.00; B = $36.00; A has the larger expected payment, but this alone does not determine an action.)
- For 160 independent cases, each with a predicted event probability of 0.1500, calculate the model-implied expected event count, variance, and standard deviation. State whether the expected count is a certain outcome. (Answer: Expected count = 24; variance = 20.4; standard deviation is approximately 4.52; the count is not certain.)