Why Frame the Case Before Choosing a Model?
An actuarial team may ask for a model to “identify high-risk policies” or “flag claims for review.” Those phrases sound precise, but they do not yet specify who will use the prediction, what event counts as high risk, or what decision the result is meant to support. Without those details, it is easy to build a technically sound model for the wrong task.
A framing checklist turns a broad business request into a prediction task that can later guide data design and evaluation. It does not select an algorithm. It makes the intended prediction and its operating context explicit enough that an analyst can determine what information is needed and whether the task is feasible.
The Seven Questions
The checklist is most useful when answered in order. Each question resolves a different ambiguity; an answer to one should not be substituted for an answer to another.
Name the person or team that will receive or use the prediction, such as a renewal specialist, claims manager, or underwriting team. If the prediction is only for analysis and no operational user has been identified, say so rather than implying there is one.
Specify the outcome to be predicted in observable terms. Define what counts as the event, the relevant measurement, and any conditions needed to distinguish the target from similar events. “Risk” by itself is not a target.
State what one prediction describes: for example, a policy renewal opportunity, an open claim, or an application. The unit is the subject of the prediction, not automatically the person who will use it. Precise decisions about how records represent that unit are a separate data-design task.
Give the decision point at which the prediction must be available, such as 30 days after a claim is reported or 60 days before a policy expires. The prediction time determines what information could be available for use.
State the period after the prediction time during which the target is observed. A target without a horizon can refer to materially different outcomes, such as an event within 30 days or at any point during the next year.
Describe what the organization expects to do with the prediction: send outreach, request review, or route a case to a specialist, for example. The prediction describes an outcome; an action is a subsequent operational choice.
Check deadlines, information availability, staffing or processing capacity, system requirements, permitted data use, governance, and any relevant fairness or customer-impact considerations. A prediction that arrives too late or cannot be acted on does not meet the business need.
The checklist separates three ideas that are often blurred. The target is what the model is asked to predict; the unit is what receives a prediction; and the action is what a user may do after seeing it. For example, a model might estimate the probability that an open claim exceeds a specified cost threshold within a stated period. A claims manager might use that estimate to select claims for review. The estimate is not itself a recommendation that a particular claim should be reviewed.
A Compact Specification and Why Its Timing Matters
A useful case statement can be written in words or represented with a small amount of notation. Let \(t_0\) denote the prediction time, \(H\) the stated horizon, and \(Y\) the defined target. Let \(X(t_0)\) denote the information that is available by \(t_0\). The prediction task concerns the outcome over the period after that prediction time, using information available then.
This notation is a framing aid, not a claim that a particular model or probability scale has already been chosen. It makes two timing conditions visible: the target refers to a specified future interval, and the predictors are restricted to information available by the time the prediction is made.
The horizon must also match the decision. A specialist who needs a referral list by the next morning has a different timing requirement from a team planning a quarterly review. A target may be measurable over a long period, but that does not make the result useful for an action that must happen sooner. Similarly, when an action can affect the eventual outcome, observed outcomes may reflect the organization’s existing actions. The case statement should describe the target and the operating context clearly; it should not quietly equate outcome risk with the value of taking a particular action.
Worked Applications
Worked Example: Renewal Outreach for Small Commercial Policies
A hypothetical insurer wants to contact some small commercial policyholders before renewal. The renewal team asks for a model to “find policies likely to leave.” Its system can deliver a list 60 days before each policy’s expiration. Staff can make outreach calls during the following two weeks. Frame the case using the checklist.
Method. Answer each checklist question in context, and keep the predicted outcome separate from the proposed outreach action.
Decision-maker. The decision-maker is the renewal team that receives the list and arranges the calls. Naming the team identifies who must be able to use the output.
Target. Define the target as whether the policy is not renewed by its scheduled expiration under the insurer’s existing renewal process. This is more precise than “likely to leave”: it identifies an observable outcome and a time by which it is assessed. The target does not claim to identify which policyholders would be persuaded by a call.
Unit. Each prediction is for one upcoming policy renewal opportunity. It is not a prediction about an entire business customer across all policies.
Prediction time and horizon. The prediction is made 60 days before the scheduled expiration. The target is assessed by that expiration, so the prediction concerns the period from the decision point through the scheduled expiration. This gives the outreach team time to act before the renewal deadline.
Intended action. The team uses predictions to decide which policyholders to contact during the next two weeks. The predicted probability of nonrenewal is not an estimate of the causal effect of calling a policyholder.
Practical constraints. The result must be available 60 days before expiration, and customer and policy information must be available by then. The team must also check whether call staffing, contact rules, and the delivery format allow the list to be used before the deadline.
Check. The case names a user, measurable target, unit, time, horizon, action, and constraints. It avoids the common mistake of treating a high predicted nonrenewal risk as proof that outreach will change that outcome.
Worked Example: Early Review of Complex Claims
A hypothetical claims operation wants to direct open property claims to experienced adjusters. Its claims manager requests a score for each claim 14 days after first notice of loss. The proposed target is whether the claim’s total incurred amount will exceed $25,000 by 180 days after first notice. The experienced-adjuster team can only accept cases with a complete basic file. Specify the task and identify the important constraint.
Method. Pin down the unit, threshold event, prediction time, horizon, and action before deciding how a score will be used.
Decision-maker. The claims manager is responsible for the routing decision; experienced adjusters are the operational recipients of selected cases.
Target. For each claim, the target is whether its total incurred amount exceeds $25,000 by 180 days after first notice of loss. The amount threshold and measurement date make “complex” operationally testable. A claim that exceeds the threshold after day 180 is not a positive outcome under this particular definition.
Unit. One open claim is the subject of one prediction. The task is not to score an individual claimant or an entire policy.
Prediction time and horizon. The prediction is made on day 14 after first notice. The target is measured at day 180 after first notice, so 166 days remain between the prediction time and the target measurement date. This arithmetic check catches the mistake of treating the horizon as 180 additional days after day 14.
Intended action. The manager may route claims with relevant predictions to experienced adjusters for review. The target predicts a cost-threshold outcome; it does not, by itself, establish that routing will reduce claim cost.
Practical constraint and check. A claim must have a complete basic file to be eligible for the experienced-adjuster team. Therefore, the operational specification must state whether the model scores every open claim or only eligible claims, and the score must be available after the day-14 information is assembled. A score for an ineligible claim cannot be treated as an actionable referral under the stated process.
The common mistake is to write “predict complexity and send high scores to specialists.” That phrase leaves both “complexity” and the eligible population undefined. This specification gives a measurable target and makes eligibility a real operating constraint rather than an unstated assumption.
Worked Example: Fraud Investigation Referrals
A hypothetical insurer is considering a fraud-investigation queue for newly reported vehicle claims. The investigation manager wants a prediction available within 24 hours of first notice. For this exercise, define the target as a claim being substantiated as involving intentional misrepresentation within 90 days of first notice. The investigation unit can open a case only when required documents have been received. Frame the task and explain why “recommend investigation” is not an adequate target.
Method. Separate the defined future event from the operational decision, then check whether timing and eligibility permit action.
Decision-maker. The investigation manager decides which eligible claims enter the investigation queue. Investigators carry out the resulting reviews.
Target. The target is whether intentional misrepresentation is substantiated within 90 days of first notice, according to the stated definition. “Fraud risk” would be too vague without an event definition and assessment period.
Unit. One newly reported vehicle claim receives one prediction. This aligns the assessment with the claim-level investigation decision.
Prediction time and horizon. The prediction must be available within 24 hours of first notice. The target is assessed within 90 days of first notice, not 90 days after the prediction is delivered. The time origin is stated explicitly so the outcome window is not accidentally extended.
Intended action. The manager may use the prediction as one input to decide whether an eligible claim should enter the queue. “Recommend investigation” is not a suitable target because it describes a decision or model output rather than the future event the model is meant to predict.
Practical constraints and check. The queue can open an investigation only after required documents arrive, and the prediction must be produced within the 24-hour service requirement. The team must establish whether the documents needed for eligibility will usually be available in time. If they are not, a prediction may identify a claim of interest but still fail to support the proposed immediate action.
The common mistake is to label a claim “fraud” because it received a referral. Referral is an action; substantiation is the defined target. Keeping them separate makes it possible to describe both the prediction task and the actual operating process without confusing one for the other.
Common Framing Traps
| Incomplete framing | Why it is a problem | Correction |
|---|---|---|
| “Predict high risk.” | It does not identify an observable event or a time period. | Define the event and state when it is assessed. |
| “Score customers for review.” | The unit and action may not match; the decision might be made at the policy or claim level. | Name what receives a score and what the user will do with it. |
| “Use all available data.” | Some information may not exist at the time the decision is made. | Specify the prediction time and limit usable information to what is available then. |
| “The model recommends intervention.” | A predicted outcome and the value of an intervention are different questions. | State the target separately from the intended action; do not claim the prediction alone measures action benefit. |
| “The output will be used operationally.” | It does not address deadline, eligibility, capacity, or permitted use. | List constraints that could prevent or restrict action on the output. |
A useful final check is to read the case as a single sentence: “For each [unit], at [prediction time], predict [defined target] over [horizon], using information available then, for [decision-maker] to support [action], subject to [constraints].” If the sentence cannot be completed without vague terms, the case needs further clarification before modeling begins.
Check Your Understanding
For each scenario, identify the framing issue and give a concise specification or correction.
- A hypothetical insurer wants to estimate whether a home policy will be cancelled “soon.” The retention team would like to send a message, but the request gives no time period. Name the missing checklist item and give one precise example of how it could be stated. (Answer: The horizon is missing; for example, define cancellation as occurring within 30 days after the prediction date.)
- A claim team scores each newly reported claim at 7 days after notice for the chance that incurred cost exceeds $10,000 by day 120 after notice. How much time remains from prediction to target measurement, and what is the unit? (Answer: 113 days remain; the unit is one claim.)
- A hypothetical travel insurer predicts whether an insured will file a claim within 60 days after policy purchase. The marketing team will offer a safety message to selected customers. Explain why the target and action must be stated separately. (Answer: The target is a claim within 60 days; the action is sending a safety message, and the predicted claim risk alone does not establish the message’s effect.)
- A claims manager requests a score for each open claim at 10 days after first notice, to predict whether the claim will require legal representation by day 100 after first notice. The score must be available that day, but only claims with a complete file can be referred to counsel. State the time remaining to the target date and one practical constraint. (Answer: 90 days remain; referral requires a complete file, so eligibility and file availability must be addressed.)
- A hypothetical insurer predicts whether a commercial account will exceed a loss threshold over the next year, but the output is delivered after the annual account review is complete. Identify the framing mismatch and the relevant questions to revisit. (Answer: The output arrives too late to support the stated review action; revisit prediction timing, delivery deadline, and intended action.)