From a Retention Goal to a Defined Task
“Improve policy retention” is a business goal, not yet a predictive task. To make it actionable, specify who will use the result, which policy decision is in scope, what outcome is predicted, and when the prediction is available. Then state what the recipient can do with it. These choices determine what a model’s records and labels must represent.
This tutorial uses a hypothetical insurer planning renewal outreach. The purpose is not to prescribe a particular model. It is to define a task clearly enough that the right policies can be scored at the right time and the result can support a real operational decision. A precise target also makes later evaluation meaningful: the observed outcome must match the outcome the business intended to predict.
The decision-maker should be a role with authority over the next step, not merely the person who builds the model. For renewal outreach, that might be a retention operations manager who can assign staff to contact selected policyholders. The unit should match the decision: if staff can make one outreach decision for each expiring policy term, each policy-term episode is a natural unit. A customer may hold several policies, so “one customer” could combine decisions that should remain separate.
Specify the Policy, Outcome, and Dates
A renewal task needs a policy-level target that can be assigned consistently. For example, a binary target might indicate whether coverage under the same policy continues into a new term. The definition should settle details that can otherwise produce inconsistent labels: the relevant policy, the effective date, whether uninterrupted coverage is required, and the date on which the outcome is assessed.
The prediction time is the date when a score must be available for the decision. A prediction made 45 days before expiration can support an early contact; a prediction made after expiration cannot support the same action. Predictors must represent information available at that prediction time. A later payment, cancellation, or renewal record may help establish the outcome, but cannot be treated as information known when the earlier decision was made.
The outcome horizon states how long after the prediction time, or relative to a specified event such as expiration, the outcome is assessed. For renewal outreach, defining the outcome relative to the scheduled expiration date is often clearer than saying only “within 60 days.” A fixed horizon makes the target reproducible across policies with different dates.
| Task element | Renewal-outreach specification | Question to check |
|---|---|---|
| Decision-maker | Retention operations manager | Who can assign outreach? |
| Unit | One eligible policy-term episode | What receives one score and one decision? |
| Target | Whether that policy has qualifying renewed coverage | Can two reviewers label the outcome the same way? |
| Prediction time | A fixed number of days before scheduled expiration | Will the score arrive in time to act? |
| Horizon | A stated date or interval relative to expiration | When is the outcome considered known? |
| Action | For example, a staff call or renewal reminder | Can the recipient actually carry it out? |
The action should be stated in operational terms. “Use the score” is not an action. “Send the highest-priority eligible policies to the retention team for one call within five business days” identifies what happens next. The team may have limited capacity, so eligibility and any contact limit should be specified. A predicted renewal probability is an input to that decision, not a complete instruction about whom to contact.
Worked Example: Defining a Policy-Term Target
A hypothetical insurer wants to review historical records to prepare for a renewal outreach pilot. It has 120 eligible policy-term episodes in a completed cohort. For this exercise, define a renewal as coverage under the same policy continuing without a gap from the day after scheduled expiration. At 30 days after expiration, the policy must be recorded as active. The records show that 87 episodes meet this definition. Specify the task elements and calculate the cohort’s observed renewal proportion.
1. Decision and unit. The retention operations manager assigns outreach. The unit is one eligible policy-term episode because that is the record for which the team makes a renewal contact decision. It is not a policyholder with all their policies combined.
2. Target. Let \(Y=1\) if the policy has continuous coverage from the day after its scheduled expiration and is active 30 days after expiration; otherwise let \(Y=0\). This definition fixes both continuity and the assessment date.
3. Time and horizon. Set the planned score date at 45 days before expiration. The outcome is assessed 30 days after expiration. Information recorded after the score date may establish the eventual label, but it is not available for making that score.
4. Action. The intended action is to assign selected eligible policy-term episodes to a retention representative for outreach before expiration. The manager, not the model, remains responsible for setting the operating list.
5. Cohort proportion. The number meeting the target is 87 and the total is 120, so the observed proportion is \(87/120=0.725\), or 72.5%. This describes this hypothetical completed cohort under the stated definition. It is not a prediction for a particular policy and does not show that outreach caused renewals.
Check. The target and unit match the proposed decision: there is one binary outcome for each policy-term episode. If “active” were measured at a different date or a coverage gap were allowed, the target and its observed proportion could change.
Why the Outcome Definition and Prediction Date Matter
Labels that sound similar can describe different events. “Renewed,” “paid,” “not cancelled,” and “active after expiration” are not automatically interchangeable. For example, a policy may be paid but then cancelled, or may have a late record update despite uninterrupted coverage. The task definition should identify which recorded event counts as success and how operational exceptions are handled. Where the data cannot distinguish the intended event, the target needs revision before modeling.
Likewise, a useful target does not make an unusable prediction date acceptable. If staff need several days to contact a policyholder, a score produced on the expiration date may be too late. If a feature is recorded only after a renewal conversation, using it for a score dated before that conversation would misrepresent the information available at decision time. State the score date first, then check each proposed input against that date.
The following process provides a concise framing sequence.
Identify the role that can choose or assign the next operational step.
Match one scored record to one decision, such as one eligible policy-term episode.
Define the renewal event, relevant policy, and conditions for a positive outcome.
Specify when the prediction is made and when the outcome is assessed relative to expiration.
Describe what happens to a selected policy, who carries it out, and any practical capacity limit.
Worked Example: Checking a Prediction Date Against the Horizon
A hypothetical commercial insurer considers scoring each eligible policy 40 days before its scheduled expiration. One policy expires on September 30. The proposed target is whether the same policy provides uninterrupted coverage beginning October 1 and is active on October 14, which is 14 days after expiration. The team needs the score before expiration to assign a reminder call. Specify the dates and identify whether a record of a call completed on September 5 can be used as a predictor.
1. Prediction date. Forty days before September 30 is August 21. The score for this policy-term episode must therefore use information available by August 21.
2. Target and outcome date. Set \(Y=1\) if the policy has uninterrupted coverage beginning October 1 and is active on October 14; otherwise set \(Y=0\). October 14 is the 14-day post-expiration assessment date. The call itself is not the target: it is an action the team may take.
3. Predictor availability. A call completed on September 5 occurred after August 21. It cannot be used as a predictor for a score made on August 21, even if the call record is present in the final data extract. It could be relevant to a later decision with a later prediction date, but that would be a different task.
4. Action timing. If the score is available on August 21 and the team can assign calls before September 30, the proposed timing is compatible with the stated purpose. The team should still allow enough operational time for the call to be completed.
Check. The outcome is measured after expiration, while the score and proposed outreach occur before expiration. Keeping those dates distinct prevents a later event from being mistaken for information known at scoring time.
Connect the Prediction to Outreach Carefully
A renewal-risk score can help organize attention, but it does not automatically identify the best outreach recipients. A policy with a low predicted renewal probability may still be difficult to retain, while a policy with a higher probability might respond strongly to a reminder. Predicting whether renewal occurs and estimating whether outreach changes renewal are different questions. A risk prediction alone does not establish that an action will improve the outcome.
The distinction can be made explicit with a simple hypothetical decision calculation. Suppose renewal produces a benefit of \(B\) dollars, no renewal produces no such benefit, and an outreach attempt costs \(c\) dollars. Let \(p_0\) be the probability of renewal without outreach and \(p_1\) the probability with outreach for the same policy under the assumptions of the decision analysis. The expected incremental value of outreach relative to no outreach is:
The justification follows from the binary outcome. If renewal occurs, the benefit is \(B\); if not, it is zero. Thus the expected benefit under a specified option is the renewal probability multiplied by \(B\). Subtract the cost \(c\) for outreach and compare that net amount with the expected benefit without outreach. This calculation requires a credible estimate of how the action changes renewal probability and an appropriate value and cost. It cannot be computed from \(p_0\), a renewal-risk prediction alone.
Worked Example: Framing a Capacity-Limited Outreach Decision
A hypothetical insurer can contact only one of three eligible policy-term episodes this week. For illustration, an analysis supplies the following action-specific renewal probabilities: policy A has probability 0.35 without contact and 0.55 with contact; policy B has 0.60 without contact and 0.75 with contact; policy C has 0.20 without contact and 0.25 with contact. Assume each renewal has a value of $500 and each contact costs $20. Compare expected incremental net values and identify which policy the stated calculation selects.
Method and assumptions. Use \((p_1-p_0)B-c\) for each policy. This comparison assumes the supplied probabilities represent the relevant outcomes with and without the specified contact, and that the same $500 value and $20 cost apply to all three. These are hypothetical inputs, not facts about an insurer.
Policy A. The increase in renewal probability is \(0.55-0.35=0.20\). The expected incremental benefit is \(0.20\times \$500=\$100\). After the $20 contact cost, expected incremental net value is \(\$100-\$20=\$80\).
Policy B. The increase is \(0.75-0.60=0.15\). The expected incremental benefit is \(0.15\times \$500=\$75\). After cost, expected incremental net value is \(\$75-\$20=\$55\).
Policy C. The increase is \(0.25-0.20=0.05\). The expected incremental benefit is \(0.05\times \$500=\$25\). After cost, expected incremental net value is \(\$25-\$20=\$5\).
Decision under the stated rule. The capacity is one contact, and policy A has the largest calculated incremental net value: $80, compared with $55 for B and $5 for C. The calculation selects A. It does not establish that the assumed action-specific probabilities are correct; those estimates and the value assumptions require their own support.
Common mistake avoided. Selecting the lowest no-contact renewal probability would select C, since 0.20 is lower than 0.35 and 0.60. That ranks risk, not expected outreach value. The correction is to compare the expected outcomes with and without the proposed action, when valid action-specific estimates are available.
Common Framing Errors
- Using “customer” when the action is policy-specific. A policyholder with multiple policies may need a separate decision for each eligible policy-term episode.
- Leaving renewal undefined. State the coverage condition and assessment date rather than relying on an informal label.
- Giving a horizon without an index date. “Within 60 days” is unclear unless the starting date and event being measured are specified.
- Scoring too late for the action. The prediction date must leave enough time for the intended outreach to be assigned and completed.
- Treating risk as action value. A probability of renewal describes the target under the prediction setup; by itself, it does not measure the effect of a call.
- Ignoring who can act. A score has no operational role if no team has authority, capacity, or a defined next step.
Check Your Understanding
For each hypothetical situation, identify the relevant task elements and check that the proposed timing and action are coherent.
- A home insurer’s retention team scores each expiring policy 30 days before expiration. Define a binary target as active, with no coverage gap, 10 days after expiration. Name the unit, prediction date, and outcome assessment date. (Answer: One eligible policy-term episode; the score is made 30 days before its scheduled expiration; the target is assessed 10 days after expiration for continuous coverage and active status.)
- A personal-lines team wants to contact “customers likely to leave” but has customers with multiple policies. It can assign one call per expiring policy. What is a better unit, and what must be specified to label the target? (Answer: Use one eligible policy-term episode per decision; define the relevant policy’s renewal event, coverage condition, and assessment date.)
- A score is scheduled 50 days before expiration, but a proposed predictor is the result of a renewal quote generated 20 days before expiration. Is that predictor available for the stated score? State the correction. (Answer: No. The quote is generated after the prediction date; exclude it or define a later prediction date and reconsider whether that date still permits timely outreach.)
- A hypothetical policy has a 0.40 probability of renewal without a call and 0.52 with a call. Renewal value is $600 and the call costs $30. Calculate expected incremental net value under the stated assumptions. (Answer: \((0.52-0.40)\times \$600-\$30=\$72-\$30=\$42\).)
- A manager can contact one policy. Policy X has renewal probabilities 0.25 without contact and 0.35 with contact; policy Y has 0.55 without contact and 0.70 with contact. Each renewal is valued at $400 and each contact costs $15. Which policy has the higher expected incremental net value? (Answer: X: \((0.35-0.25)\times \$400-\$15=\$25\). Y: \((0.70-0.55)\times \$400-\$15=\$45\). Select Y under these assumptions.)