Why the Meaning of an Output Needs Context
A model might assign a policy a predicted probability of 0.12, or give an applicant a score of 72. Neither number explains itself. A useful interpretation must say what outcome is being predicted, over what period, for which population, and whether the output is a probability or a score. Without that context, a reader may mistake an estimate for a promise, or a ranking number for a percentage.
In predictive work, a probability describes an estimated likelihood under a specified model and context. It does not say that the event will—or will not—occur for a particular person. A model score may indicate relative position or be an input to a probability conversion, but a score is not automatically a probability.
For example, “a 12% predicted probability” is incomplete unless the event and period are known. “The model estimates a 12% probability of a lapse during the next 12 months for a policy with these recorded characteristics” is more informative. It identifies a target event and horizon, although it should also be clear which policy population the model is intended to describe.
Interpreting an Individual Predicted Probability
For an individual case, a predicted probability summarizes the model’s estimate given the available recorded characteristics. It is not a statement that the model knows the future outcome for that case. If the model assigns 0.20, the careful interpretation is that it estimates a 20% likelihood in the stated context—not that the individual “will have” the event with 20% certainty, or that exactly 20% of that individual’s future will contain the event.
A single case produces one observed outcome for a binary event: it occurs or it does not. The probability is meaningful as an estimate across comparable cases or under repeated situations described by the same model context. Whether those estimates are reliable for a particular use depends on the model and whether the case belongs to the population and conditions for which the model is intended. A probability should not be detached from those conditions.
Worked Example: Explaining a Single Policy Probability
A hypothetical model estimates the probability that an active policy will lapse during the next 12 months. A policyholder has a predicted probability of 0.18. The model is intended for active individual policies in the service area used for model development, with the same lapse definition and 12-month horizon. Write a plain-language interpretation and identify what the number does not establish.
Step 1: State the event and time horizon. The event is a lapse, and the period is the next 12 months. Do not describe the output merely as “risk” without naming the outcome.
Step 2: State the reference context. The intended population is active individual policies in the specified service area. The estimate is for a policy with the recorded characteristics supplied to the model, under the stated definition and period.
Step 3: Translate the decimal. Since \(0.18 \times 100=18\), the model estimates an 18% probability of lapse during the next 12 months for this policy in that context.
Plain-language statement: “For an active policy with these recorded characteristics, the model estimates an 18% probability of lapse within the next 12 months, using the model’s stated policy population and lapse definition.”
What it does not establish: It does not establish that this policy will lapse, that it will remain in force, or that the policyholder has a fixed 18% chance independent of how well the model applies. The outcome for this one policy is not determined by the probability statement. This interpretation avoids the common mistake of treating an estimated likelihood as an individual guarantee.
From Individual Probabilities to an Expected Count
When probabilities are available for a group, they can be added to obtain an expected number of events, provided each probability refers to the same defined event and compatible observation period. The result is an average under the model’s probability assumptions; it is not a promised count for that group.
The justification is linearity of expectation: the expected value of a sum is the sum of the expected values. Because \(Y_i\) is an indicator, it equals 1 on the event and 0 otherwise, so \(E[Y_i]=1\cdot p_i+0\cdot(1-p_i)=p_i\). This derivation does not require independence. Independence would matter for other calculations, such as the variance of the total, but not for this expected count.
Worked Example: Summing Probabilities for a Small Portfolio
A hypothetical insurer uses a model to estimate the probability of a claim during the next year for five comparable policies. The predicted probabilities are 0.08, 0.12, 0.20, 0.25, and 0.35. Find the expected number of claims and explain it in plain language.
Method. Add the individual probabilities, since each is a probability of the same event over the same one-year period.
Step 1: Add the probabilities. \(0.08+0.12=0.20\); \(0.20+0.20=0.40\); \(0.40+0.25=0.65\); \(0.65+0.35=1.00\).
Step 2: Interpret the sum. The model-based expected number of claims is \(1.00\) claim. The mean predicted probability across the five policies is \(1.00/5=0.20\), or 20%.
Plain-language statement: “Across these five policies, the model’s probabilities sum to an expected one claim over the next year.” This is not a prediction that exactly one claim will occur. The observed count must be a whole number, and it could be zero, one, or more than one.
Check and common mistake avoided. The sum is 1.00 and the average is 0.20; these are different quantities. Reporting the average probability as the expected claim count would confuse a rate per policy with a total for five policies.
What a Model Score Does—and Does Not—Say
A model score is a numerical output whose meaning depends on how the model defines it. Some scores are used to order cases from lower to higher estimated risk. Others are transformed by a specified rule into probabilities. A score of 72 is not automatically a 72% probability, and the score’s direction should not be assumed: for one system a higher score may indicate a higher estimated likelihood of the event, while another system may use a higher score for a more favorable outcome.
To interpret a score, identify its scale, direction, and any documented conversion. If the conversion is not supplied, do not invent one. You may be able to say that one case has a higher score than another, if the scoring convention says that higher scores indicate higher event risk. You cannot infer the size of the probability difference from that ordering alone.
| Output | What can be said | What should not be assumed |
|---|---|---|
| Predicted probability | An estimated likelihood for a defined event, period, and applicable population | That the event is certain or impossible for an individual case |
| Raw score with a documented direction | A case’s position on the score scale, such as higher or lower than another case | That the score is a percentage or that a score difference is a probability difference |
| Score with a specified conversion rule | The probability produced by applying that rule, subject to the model’s intended context | That the number is meaningful outside the rule’s defined outcome and population |
Worked Example: Converting a Score Only When the Rule Is Given
A hypothetical scoring system supplies the conversion \(p=1/(1+\exp(-s))\), where \(s\) is its raw score and \(p\) is the resulting predicted probability for a defined binary event. Two cases have scores \(s=-\ln(4)\) and \(s=\ln(4)\). Calculate both probabilities and explain why the raw scores should not be read as percentages.
Method. Apply the supplied conversion to each score. Here \(\exp(\ln(4))=4\) and \(\exp(-\ln(4))=1/4\).
Case 1. For \(s=-\ln(4)\), \(-s=\ln(4)\), so \(\exp(-s)=4\). Therefore, \(p=1/(1+4)=1/5=0.2000\), or 20.00%.
Case 2. For \(s=\ln(4)\), \(-s=-\ln(4)\), so \(\exp(-s)=1/4\). Therefore, \(p=1/(1+1/4)=1/(5/4)=4/5=0.8000\), or 80.00%.
Interpretation. Under this specified conversion, the first case has a predicted probability of 20.00% and the second has a predicted probability of 80.00%, for the event and context defined by the scoring system. The raw scores themselves are not percentages; the probabilities come from applying the stated rule.
Check and common mistake avoided. The two converted probabilities sum to 1.0000, which is consistent with the symmetric scores in this example. Treating \(-\ln(4)\) and \(\ln(4)\) as probabilities, or reporting them as percentages, would ignore the conversion rule and give meaningless probability statements.
Make the Reference Population Explicit
A probability is not fully described by the characteristics of one case. It also has a reference context: the population and conditions in which the model is intended to be used. For example, a model developed for active policies in one region, with a particular outcome definition and observation period, does not automatically describe new policies in another region or events over a different period.
When explaining an output, name the relevant context in ordinary language. Include the event, period, and intended population when available. If the intended population is unclear, say so rather than quietly extending the interpretation. A probability estimate for a case outside that context may still be computable, but its meaning and reliability have not thereby been established.
Worked Example: Translating Probabilities for a Defined Cohort
A hypothetical model is intended to estimate whether an active policy in a specified service area will lapse in the next 12 months. For a cohort of 100 such policies, it assigns probability 0.04 to 25 policies, 0.08 to 25 policies, and 0.14 to 50 policies. Calculate the expected number of lapses and the average predicted probability, then write a statement that preserves the reference context.
Step 1: Calculate each group’s contribution. For the first 25 policies, \(25\times0.04=1\) expected lapse. For the next 25, \(25\times0.08=2\). For the 50 policies assigned 0.14, \(50\times0.14=7\).
Step 2: Add expected counts. \(1+2+7=10\), so the model-based expected count is 10 lapses during the next 12 months.
Step 3: Calculate the cohort’s average probability. The average is \(10/100=0.10\), or 10%. Equivalently, the weighted calculation is \((25\times0.04+25\times0.08+50\times0.14)/100=10/100=0.10\). A simple unweighted average of the three group probabilities would be inappropriate because the groups have different sizes.
Plain-language statement: “For this cohort of 100 active policies in the model’s specified service area, the model’s probabilities imply an expected 10 lapses over the next 12 months, equivalent to an average predicted probability of 10%.” The statement does not promise that exactly 10 policies will lapse.
Check and common mistake avoided. The group sizes add to \(25+25+50=100\), and the probability-weighted expected count is \(1+2+7=10\). Averaging 0.04, 0.08, and 0.14 without weights gives about 8.67%, which does not represent the 100-policy cohort because the probability groups are not equally sized.
Common Communication Traps
- Turning likelihood into certainty. “The model predicts this person will lapse” overstates a probability output. State that the model estimates a probability of lapse within the stated period.
- Calling a score a percentage. A score of 72 is not 72% unless its definition and conversion explicitly make it so.
- Omitting the event or period. “The probability is 10%” leaves the reader unable to tell what event is being predicted or when it may occur.
- Omitting the reference population. A model output should not be presented as automatically applicable to cases outside the model’s intended population and conditions.
- Confusing an average with a total. The mean predicted probability describes an average per case; the sum of individual probabilities is the expected count for a group when the outcomes and periods align.
- Reporting an expected count as an observed count. An expected count can be fractional and is a model-based average, not a claim about how many events actually occurred.
Check Your Understanding
For each question, identify the output’s proper interpretation and avoid treating an estimate or score as more than the information supports.
- A model estimates a 0.07 probability of a defined event for each of 20 cases over the same year. What is the expected number of events? (Answer: \(20\times0.07=1.4\) expected events.)
- A system gives two applicants scores of 41 and 58. The only documentation says that higher scores indicate higher estimated event risk. What can be concluded, and what cannot be concluded about their probabilities? (Answer: The score of 58 indicates higher relative event risk under the stated direction; the probability values and their difference cannot be determined without a probability scale or conversion.)
- A score conversion is \(p=1/(1+\exp(-s))\). For \(s=-\ln(3)\), calculate \(p\). (Answer: \(p=1/(1+3)=0.2500\), or 25.00%.)
- A model gives probability 0.06 to each of 60 comparable units for the same one-year event. State the expected count and average predicted probability. (Answer: \(60\times0.06=3.6\) expected events; the average predicted probability is 6%.)
- A model was intended for active policies in a specified region and predicts a 0.11 probability of a 12-month event for a new policy outside that region. What context should be included in an explanation, and what limitation should be noted? (Answer: State the event and 12-month horizon, and identify that the intended reference population is active policies in the specified region; note that applicability to the new policy outside that region is not established by the prediction alone.)