Why Prediction Time Must Be Explicit
A predictive task needs more than a unit of analysis, target, and intended action. It must also say when the prediction is produced. “Predict which policies will lapse” is incomplete if one team means the day a renewal notice is prepared and another means the day coverage expires. Those are different operating points: different actions may still be possible, and the information available to decision-makers may differ.
Prediction time is an operational definition, not just a date field in a data set. It identifies the event or scheduled instant at which a score is generated for a particular case. The definition should be precise enough that two analysts, or two scoring systems, would identify the same prediction event.
This distinction follows the earlier material on identifying the decision-maker and connecting predictions to intended actions. A prediction is useful only in an operating process where it can reach the decision-maker in time to inform an action. Defining \(t_0\) therefore starts with the process: what event triggers scoring, who receives the output, and when must that person act?
Prediction time is not the same as the length of time over which an outcome is considered. These are separate parts of a predictive task. This tutorial focuses on locating the scoring instant in the operational process; the outcome window is specified separately.
Describe a Prediction Time as an Event
A calendar date alone may not be enough. “On the renewal date” can refer to the start of that date, the time a batch job runs, or the moment the policy record is updated. A useful definition names the triggering event and, where relevant, a time of day, time zone, and business-process condition.
For example, a renewal score might be produced “when the renewal review file is finalized, at 08:00 local time, 30 calendar days before the policy expiration date.” The statement specifies a process event, a clock time, a time zone convention, and the relation to a policy date. If the event is delayed or does not occur, the procedure should also define what happens to the scoring event.
A practical specification often records the following:
- Trigger: the scheduled job, transaction, or workflow event that initiates scoring.
- Unit: the case receiving one prediction at that event, such as a policy renewal or a newly reported claim.
- Timestamp convention: date, time, and time zone, including how daylight-saving changes are handled if relevant.
- Output availability: when the score is delivered to the person or system expected to use it.
- Decision deadline: when the intended action must be selected or completed.
The exact contents of a scoring snapshot and the rules for deciding which records are eligible for use require separate specification. For the purpose of defining prediction time, the key is to identify the instant at which the operational system treats the prediction as being made, rather than using a convenient but unrelated date in a later extract.
Worked Examples
Worked Example: A Policy Renewal Review
Scenario. A hypothetical insurer prepares renewal offers for policies expiring on June 30. The renewal team needs a lapse-risk score early enough to decide which cases receive a retention review. The team finalizes its renewal file at 08:00 local time, and the review must be assigned by 12:00 local time that day. Specify the prediction time and distinguish it from the decision deadline.
Method: anchor the score to the workflow trigger. The file-finalization event is the operational trigger. For a policy with an expiration date of June 30, suppose the file is finalized on May 31. The score is defined as produced at 08:00 local time on May 31, when that policy’s renewal record enters the finalized file.
Step 1: state the prediction event. For each eligible renewal record, prediction time is May 31 at 08:00 local time, at file finalization. This is not simply “the renewal date,” because the renewal date is June 30 and the score is needed before then.
Step 2: state when the output can be used. If scoring and delivery take 10 minutes, the output is available at 08:10. The decision deadline is 12:00. The interval between availability and deadline is \(12{:}00-08{:}10=3\) hours and 50 minutes.
Step 3: check operational fit. The score is available before the review assignment deadline, so the described process gives the team time to use it. If the stated trigger were instead the renewal date, the score would arrive after the review deadline and would not support the intended assignment.
Answer and trap avoided. Define \(t_0\) as 08:00 local time on May 31 at renewal-file finalization; define \(t_D\) as 12:00 that day. A common mistake is to call June 30 the prediction time because it is the policy expiration date. That confuses a policy event with the event at which the score is produced.
Worked Example: Claim Intake and a Delayed System Entry
Scenario. In a hypothetical claims operation, a claim is reported by telephone at 14:20 on October 7. A claim-intake score is intended to help route the report to a handling queue. The call record is entered into the system at 14:32, and the scoring service returns an output at 14:34. The routing decision is due by 14:45. Specify the prediction time and assess whether the described score arrives in time.
Method: separate triggering, scoring, and decision events. The intended trigger is the claim report being entered into the intake workflow, not necessarily the caller’s first contact or the later routing decision.
Step 1: identify the event that initiates scoring. The record is entered at 14:32. Under the stated workflow, this is the scoring trigger, so \(t_0=14{:}32\) on October 7, using the operation’s local time zone.
Step 2: identify output availability. The service returns the score at 14:34. Thus delivery takes \(14{:}34-14{:}32=2\) minutes.
Step 3: compare availability with the decision deadline. The decision is due at 14:45, so the score is available \(14{:}45-14{:}34=11\) minutes before the deadline. On the stated facts, it arrives in time to inform the routing decision.
Step 4: record the contact-to-entry delay. The call starts at 14:20 and system entry occurs at 14:32, giving a delay of \(14{:}32-14{:}20=12\) minutes. That delay does not change the stated \(t_0\), but it matters when describing the workflow accurately. If the organization instead defines scoring as triggered at first contact, it must specify how a score can be produced at that event.
Answer and trap avoided. The defined prediction time is 14:32, when the intake record is entered and scoring is triggered; the output is available at 14:34, before the 14:45 deadline. A common mistake is to use 14:20 simply because it is the earliest timestamp. The earliest timestamp is not automatically the prediction time; the process definition determines the trigger.
Worked Example: Two Scores for One Policy
Scenario. A hypothetical policy administration process scores each active policy at 09:00 on the first business day of every month. A service team also requests an extra score when a policyholder changes their mailing address, and a change is recorded at 16:10 on April 12. For one policy, identify the prediction events if the monthly score is produced on April 1 and the address-change score is produced on April 12. Explain why treating both as one prediction time is ambiguous.
Method: define a separate scoring event for each trigger. A unit of analysis may receive more than one prediction when the operating process includes repeated scoring. Each score needs its own timestamp and trigger.
Step 1: specify the scheduled event. The first-business-day scoring event occurs at 09:00 on April 1. For this policy, the monthly prediction has \(t_0=\) April 1 at 09:00.
Step 2: specify the event-triggered score. The address change is recorded at 16:10 on April 12. If that record initiates scoring immediately, the second prediction has \(t_0=\) April 12 at 16:10. If the system processes the request later, the specification should name that processing time instead.
Step 3: distinguish the two outputs. The April 1 score and April 12 score are separate prediction events for the same policy. Saying only “the policy is scored in April” cannot identify which event a row in an analysis represents.
Answer and trap avoided. The two prediction times are April 1 at 09:00 and April 12 at 16:10, each attached to its own trigger. A common mistake is to keep only the latest monthly date or to merge the scores as though there were one event. That loses the operational meaning of each output.
Why the Timestamp Definition Matters
A prediction-time definition helps keep a modeled task aligned with a real workflow. Suppose a model is evaluated using records labeled with a date that differs from the date on which the production system actually scores cases. The resulting analysis may describe performance for a process that the organization does not operate. Even if the target and unit are well chosen, the model may be delivered too early, too late, or at the wrong trigger.
The basic timing requirement can be justified directly from the intended use. Let \(t_0\) be the instant the score is produced, let \(t_A\) be when it becomes available to the decision-maker, and let \(t_D\) be the decision deadline. For the score to inform that decision, it must be available no later than the deadline:
The first inequality holds when delivery does not precede score production; the second expresses the operational requirement that the decision-maker receive the output in time. If \(t_A>t_D\), the score cannot inform the stated decision, regardless of its statistical quality. This is why the timestamp must be defined in relation to the process and not merely chosen because it is easy to extract from a data table.
A score produced exactly at the decision deadline may satisfy the inequality only in a formal sense. If the decision-maker needs time to review the output, then a practical specification should require \(t_A<t_D\) by an adequate margin. That margin depends on the operating process and should be agreed with the people who will use the score.
Common Timing Traps
- Using a convenient date instead of the trigger. A record-creation date, policy effective date, or claim-close date may not be when the prediction is actually produced.
- Confusing prediction time with decision time. The score may be produced hours or days before the final action is due. State both when they differ.
- Ignoring delivery delay. A model may finish scoring before a deadline while its output reaches the decision-maker too late. Record output availability, not only computation start time.
- Leaving time zones implicit. “09:00” is ambiguous when systems, offices, or records use different time zones. Name the convention.
- Collapsing repeated scores. If a case can be scored on a schedule and again after an event, each score is a distinct prediction event.
- Using a future operational event as though it triggered an earlier score. The trigger must be feasible at the stated \(t_0\); do not describe a score as being produced before the event that initiates it.
For an exam-style response, give a concrete answer in the form: “For each [unit], the prediction is produced at [trigger] on [date/time convention]; the output is available by [time]; the associated decision is due by [deadline].” If a date, time, or trigger cannot be determined from the scenario, state what additional operational detail is needed rather than silently assuming it.
Check Your Understanding
For each situation, state the prediction time or identify the missing operational detail. Check that the output timing supports the stated decision.
- A hypothetical renewal team runs a scoring batch at 07:30 local time on September 5. The score is delivered at 07:40, and renewal reviews must be assigned by 10:00 that morning. State the prediction time, the decision deadline, and the available lead time after delivery. (Answer: \(t_0\) is September 5 at 07:30 local time; \(t_D\) is 10:00; the score is available 2 hours 20 minutes before the deadline.)
- A hypothetical claim is first reported by phone at 11:05, entered in the intake system at 11:18, scored at 11:20, and assigned to a queue by 11:35. If scoring is triggered by system entry, identify \(t_0\) and the interval from score availability to the assignment deadline. (Answer: \(t_0\) is 11:18; score availability is 11:20; the interval is 15 minutes.)
- A scheduled score is described only as “run on the policy anniversary.” The operating team must send the result to a reviewer before the end of that business day. Name two details needed to make the prediction-time specification reproducible. (Answer: Specify the exact triggering event or scheduled run time, and specify the time zone; the delivery time should also be defined to verify availability before the deadline.)
- A hypothetical service team scores an account at 13:00, but the output is not delivered until 16:30. The account decision must be completed by 16:00. Can the score inform that decision under the stated process? State the timing reason. (Answer: No; the output is available at 16:30, which is 30 minutes after the 16:00 decision deadline.)
- A hypothetical account receives a routine score at 09:00 on July 1 and a second score when a service request is recorded at 14:25 on July 9. If the service request triggers immediate scoring, how many prediction events are there, and what are their prediction times? (Answer: Two events; July 1 at 09:00 and July 9 at 14:25.)