Start With the Decision, Not the Model
An insurer may ask whether predictive analytics can help its renewal team decide whom to contact. That question is not yet a prediction task. Before selecting a model or reviewing performance metrics, the team needs to specify who will use the predictions, what outcome is being predicted, for which policy opportunities, at what date, and how the prediction will inform an action. Ambiguity in any of these choices can make a technically accurate model irrelevant to the actual decision.
For example, “predict renewal” could mean predicting whether an insurer offers a renewal, whether a customer accepts an offer, or whether coverage is active on a later date. Those outcomes are different. A useful task definition also states the prediction date and horizon: information recorded after the prediction date must not be treated as though it was available to the decision-maker at that time.
A compact notation helps keep the pieces distinct. Let \(i\) index eligible policy-renewal opportunities. Let \(t_i\) be the prediction date for opportunity \(i\), let \(X_i(t_i)\) denote the predictor information available by that date, and let \(Y_i\) denote the specified future outcome. A model may estimate \(P(Y_i=1\mid X_i(t_i))\). The meaning of that probability depends on how \(Y_i\), the eligible population, and the prediction date are defined.
Specify the Outcome and Its Population
Choose an outcome that corresponds to a real question the business can act on. If the outreach team is trying to support customers who have received a renewal offer, a possible target is whether an eligible policyholder accepts and activates that offer by a stated deadline. The task should also specify whether the insurer’s offer has already been made and whether the policy is eligible for outreach. Otherwise, a model could combine cases where the customer chooses not to renew with cases where no renewal offer was available.
The outcome needs a clear observation window. “Does not renew” is incomplete unless it states by when and according to which record. For example, the target could be \(Y_i=1\) if an eligible policyholder has not accepted an offered renewal by the scheduled start date of the next term, and \(Y_i=0\) otherwise. A different horizon, such as acceptance by a date shortly before expiry, defines a different target and may be more useful operationally.
The eligible population matters just as much as the outcome. A prediction for every expiring policy answers a different question from a prediction for only those policies for which the insurer has issued an offer and can legally and operationally conduct outreach. State exclusions in advance. This avoids quietly changing the population during model development or at deployment.
Worked Example: Define the Target for an Offered Renewal
A hypothetical insurer wants its renewal team to contact policyholders who may need help completing a renewal. For each eligible policy, the team plans to generate a prediction 30 days before the scheduled start of the next term. A renewal offer has already been issued. Define the unit, target, and prediction horizon.
Step 1: Name the decision-maker and use. The renewal team is the decision-maker. It will use the prediction to identify offered renewals for possible customer contact.
Step 2: Define the unit. One row represents one eligible policy-renewal opportunity. The unit is not a customer across all products, and it is not an individual claim.
Step 3: Define the prediction date. The prediction date is 30 days before the next term’s scheduled start date. Only information available by that date can be used as a predictor.
Step 4: Define the target. Set \(Y_i=1\) if the policyholder has not accepted and activated the offered renewal by the scheduled start date, and \(Y_i=0\) if the policyholder has done so. This makes the outcome deadline explicit.
Step 5: State eligibility. Include only policies for which an offer has been issued and outreach is permitted and feasible. The resulting task concerns acceptance of an available offer, not whether the insurer will make an offer.
Check. A policyholder who accepts on the scheduled start date is assigned \(Y_i=0\) under this definition; one who has not accepted by that date is assigned \(Y_i=1\). The deadline resolves the otherwise ambiguous phrase “may renew.” The common mistake avoided is combining customer acceptance with the insurer’s separate decision to offer coverage.
Fix the Prediction Date and Predictor Set
For each opportunity, the prediction date is the point at which the prediction is intended to be available for use. A predictor must be known, or legitimately computable from information known, by that date. For example, prior-term payment history may be available, while a response recorded after the prediction date is not. A later record can be useful for defining or checking the target, but it cannot be used as a predictor for a prediction supposedly made earlier.
This is a time-ordering requirement, not merely a data-cleaning preference. If information created after \(t_i\) is included in \(X_i(t_i)\), a historical analysis may appear more predictive than a model could be in actual use. The model would be relying on future information. The same issue arises when a variable is updated later but stored in a database as if it represented an earlier state. Predictor definitions should therefore include both the variable and the time at which its value is taken.
The eligible population and prediction date also determine what “available” means. A renewal status recorded the day before expiry might be legitimate for a prediction generated on that day, but not for one generated 30 days earlier. Write down a consistent snapshot rule so that historical training records match the information that can be supplied when the model is deployed.
Identify the team or role that receives the prediction and the operational question it faces.
State which policy-renewal opportunities can be considered and what one record represents.
Specify the event, its reference date, and the deadline by which it must occur.
Fix when the result must be available and restrict predictors to information available by then.
Describe how the output will inform the team’s work, while distinguishing a prediction from an estimate of what an intervention will cause.
Worked Example: Exclude Information Recorded After the Snapshot
A hypothetical small-business insurer generates a renewal prediction 45 days before the next term begins. For one policy, the file contains a prior-term payment record dated 60 days before the term, a customer-service note entered 12 days before the term, and the final renewal status recorded on the term’s start date. The task predicts whether the offered renewal is active on the start date. Decide which records can be used as predictors.
Step 1: Locate the prediction date. Forty-five days before the start date is the snapshot. The prior-term payment record is dated 60 days before the start date, so it predates the snapshot and can be considered, assuming it was available to the renewal team then.
Step 2: Compare the other dates with the snapshot. The service note is entered 12 days before the start date. Since 12 days before is later than 45 days before, the note is future information relative to the prediction date and must be excluded from this task.
Step 3: Separate predictor from target information. The final renewal status is recorded on the start date. It is used to assign the outcome, not as a predictor for a prediction made 45 days earlier.
Conclusion. The prior-term payment record is a candidate predictor; the later service note and final renewal status are not predictors for this snapshot. The status may label the target once the outcome window has passed. The common mistake avoided is treating all fields in a completed historical file as if they were available at the prediction date.
Prediction Is Not the Same as Outreach Impact
A model that predicts nonrenewal under the process represented in its historical data does not automatically estimate whether outreach will prevent nonrenewal. The prediction \(P(Y_i=1\mid X_i(t_i))\) describes the conditional probability of the defined outcome, given the specified predictors, under the data-generating and operational conditions represented by the data. If some policyholders historically received outreach, that history may be part of those conditions.
A different question is whether a particular policyholder would renew if contacted compared with what would happen if not contacted. That is a causal question about the effect of an intervention. A high predicted probability of nonrenewal does not, by itself, show that the customer is more likely to change their decision because of contact. The prediction can still help organize review or service work, but the intended action and the interpretation of the model output should not overstate what the model establishes.
This distinction is particularly important when the action can affect the target. If outreach changes renewal behavior, then observed historical outcomes may reflect prior outreach practices. The task definition should say whether the model is intended to predict outcomes under the current process or support a separate evaluation of an outreach strategy. A predictive score alone does not answer the second question.
Worked Example: Interpret Scores for a Defined Outreach List
A hypothetical renewal team has already selected three eligible policy opportunities for review under its existing workflow. A model, trained for the target “no accepted renewal by the scheduled start date,” assigns estimated probabilities of \(0.40\), \(0.25\), and \(0.15\) to the three opportunities. Calculate the expected number of nonrenewals among these three under the model’s probability interpretation, then state what that calculation does and does not mean.
Step 1: Represent each outcome. Let \(Y_i=1\) indicate no accepted renewal by the defined deadline and \(Y_i=0\) otherwise. The three predicted probabilities are \(0.40\), \(0.25\), and \(0.15\).
Step 2: Add the expected outcomes. For an indicator \(Y_i\), \(E[Y_i\mid X_i(t_i)]=P(Y_i=1\mid X_i(t_i))\). Therefore, the expected number of nonrenewals among the three is
Step 3: Interpret the result. The model-based expected count is \(0.80\) nonrenewals in this set, not a claim that exactly one policy will fail to renew. By linearity of expectation, adding the individual expected indicators gives the expected total; independence between the outcomes is not required for this calculation.
Step 4: State the limit. This calculation does not estimate how many renewals outreach will cause or prevent. The selected set was given in the problem; this example does not choose an outreach cutoff or establish that contacting these policies will change their outcomes. The common mistake avoided is interpreting predicted nonrenewal risk as predicted responsiveness to outreach.
Make the Intended Action Operational
A prediction task should end with a specific account of how the output will be used. “The model will support renewal outreach” is a start, but the operational description should clarify who receives the predictions, what information they receive, and what action they may take. For example, a renewal team might use predictions to prioritize manual review or to identify cases for a permitted service contact. A score should not be presented as a decision in itself.
The intended action also constrains the useful prediction date. A prediction generated after the renewal team’s contact window has closed cannot support timely outreach, even if its statistical performance is strong. Conversely, moving the prediction date earlier may reduce the information available to the model. These are task-design trade-offs: they should be made explicit before model development rather than hidden inside a later analysis.
Finally, document the assumptions and boundaries of the task. State which renewal offers and policy types are in scope, what happens to records with incomplete outcome observation, and which operational process the model is intended to support. These details make it possible to interpret results in context and to recognize when the deployment population or process no longer matches the defined task. Where an action has fairness or customer-impact implications, the decision-maker and governance process must be clear about how predictions are reviewed and used.
Check Your Understanding
For each situation, specify the task element being tested and check the answer against the final result.
- A team predicts 20 days before an offered renewal begins whether the policyholder will have accepted it by the start date. It includes a payment posted 25 days before the start date and a call outcome recorded 10 days before the start date. Which item is eligible as a predictor for this prediction date? (Answer: The payment record is a candidate predictor; the later call outcome is not available at the prediction date.)
- A model is built only for policies with a renewal offer already issued. Its target is “the insurer issues a renewal offer.” Identify the mismatch between target and eligible population. (Answer: The target is already established by the eligibility condition; the task should instead define a future outcome that can vary within the offered-renewal population, such as acceptance by a specified deadline.)
- For an eligible offered policy, define \(Y_i=1\) if no renewal is active at the scheduled start date. A policyholder activates the renewal on that date. What is \(Y_i\)? (Answer: \(Y_i=0\), because the renewal is active by the specified deadline.)
- A policy-level model estimates nonrenewal probabilities of \(0.30\), \(0.20\), and \(0.10\) for three already-identified opportunities. Under the model’s probability interpretation, what is the expected number of nonrenewals among them? Does this show that outreach prevents any of them? (Answer: \(0.30+0.20+0.10=0.60\) expected nonrenewals; no, it does not estimate the causal effect of outreach.)
- A renewal manager wants a prediction 35 days before term start. A historical file contains a predictor value updated 22 days before term start, a target outcome recorded at term start, and a predictor value captured 50 days before term start. Which values can be predictors for the specified snapshot, assuming they were available when recorded? (Answer: The value captured 50 days before term start can be a predictor; the value updated 22 days before and the term-start outcome are later than the prediction date and cannot be predictors.)