Why Availability Matters
A prediction is useful only if it can be made at the time it is needed. A model may appear accurate when tested on a historical data set, yet fail in use because some of its inputs describe events that had not happened—or records that had not arrived—when the prediction was supposed to be made. The central question is not simply whether a value appears in the data. It is whether that value could have been known to the intended user at the decision time.
This issue often arises when a data set is assembled after the events being studied. Later updates can be joined to earlier records, making it look as though the information was available all along. A careful problem definition therefore begins with a timeline: identify the moment a prediction is required, then classify each proposed input by whether it is available at that moment.
Availability is a practical condition, not just a matter of dates. A payment may have occurred on one day but not been received or posted in the system until later. Conversely, a past event may be usable if its information was already recorded and accessible by the decision time. The event date and the information-availability date can differ.
Build a Timeline Before Selecting Predictors
Write down the decision time explicitly. Then place proposed predictors and the outcome on the timeline. A predictor must be available by the decision time; the outcome is the later event or result the task is intended to predict. If a proposed predictor is only known after the decision, either remove it or redefine the task at a later decision time—provided that later time matches a real use.
Name the specific date or operational moment when a prediction must be produced, and identify who or what will use it.
For every candidate input, ask when the event occurred and when its value became available in the relevant system.
Keep information available at decision time as candidate predictors. Treat later outcomes as targets, not as inputs to that same prediction.
Specify the population, decision time, permitted information, and future outcome in words that could be implemented at the point of use.
This is a framing check, not a model-fitting procedure. It should be applied before comparing algorithms or interpreting apparent predictive performance. It is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
Worked Examples
Worked Example: Renewal Outreach Before a Payment Posts
Scenario. In a hypothetical insurer, a renewal outreach list is produced at the end of September 30. The intended task is to predict which policies will lapse by November 30. A retrospective file contains the September 30 balance, a payment dated October 4, and a cancellation notice dated October 2. Decide which items can be predictors for the September 30 task and restate the task.
Method: timeline classification. The decision time is the end of September 30. The September 30 balance is known by that time under the scenario, so it is a candidate predictor. The payment dated October 4 occurs after the decision time: September 30 to October 4 is four elapsed days. It is unavailable for the September 30 prediction. The October 2 cancellation notice is also after the decision time, so it is unavailable as a predictor even though it may be strongly associated with lapse by November 30.
Restated task. At the end of September 30, use information then available about each in-scope policy, including its recorded balance, to predict whether it will lapse by November 30. Do not use the October 4 payment or October 2 notice as inputs. The later lapse status is the target.
Check. The notice is not a valid predictor just because it concerns the same policy, and the payment cannot be moved backward in time by attaching it to a September record. No interest rate, mortality assumption, or probability distribution is needed here: this example classifies information by timing rather than calculating a risk estimate.
Common mistake avoided. Treating every field in the completed historical file as if it were known on September 30. The correction is to judge availability at the specified decision time, not at the later date when the file was extracted.
Worked Example: An Invoice With an Earlier Service Date
Scenario. A hypothetical claims team must assign a triage priority at 10:00 a.m. on March 12. The intended target is whether a claim will require a specialist review within the next 14 days. The file shows a medical service dated March 10, an invoice for that service uploaded on March 14, and a claim description submitted on March 11 and visible to the team on March 11. Which information can be used for the March 12 prediction?
Method: distinguish event time from availability time. The prediction time is 10:00 a.m. on March 12. The claim description was submitted and visible on March 11, so it is available. The service occurred on March 10, but the invoice was not uploaded until March 14. The service date alone does not make the invoice details available on March 12; the invoice is unavailable then. The specialist-review outcome is also future information for this task and belongs in the target, not the predictor set.
Restated task. At 10:00 a.m. on March 12, use the submitted claim description and other records already accessible by then to predict whether specialist review will be required between March 12 and March 26. Exclude invoice details first made available on March 14.
Check. March 12 to March 26 is 14 elapsed days. The target period is therefore stated consistently with the task. No interest rate, mortality assumption, or probability distribution is used; the solution depends on the supplied dates and availability facts, not on a statistical estimate.
Common mistake avoided. Using the invoice because its service date precedes the prediction time. The correction is to check when the invoice information became accessible, which is March 14 in this scenario.
Worked Example: A Fraud Referral Score Built After Investigation
Scenario. A hypothetical insurer wants a score to help route new claims for possible fraud investigation. The score is intended to be produced when a claim is first reported, at 9:00 a.m. on June 6. A completed historical extract includes the initial report, an investigation opened June 8, an investigator’s conclusion entered June 20, and the final fraud indicator. Identify the usable inputs and define the prediction task.
Method: locate the operational decision and order the records. The decision time is 9:00 a.m. on June 6, when the claim is first reported. The initial report is available at that time. The June 8 investigation opening and June 20 conclusion occur later, so neither is available. The final fraud indicator is the outcome the model is meant to predict; it must not be fed into the prediction as an input.
Restated task. For each newly reported claim, use information available at 9:00 a.m. on its report date to estimate whether it will later meet the specified fraud-indicator definition. The investigation records and final indicator may be used to establish historical outcomes, but not as predictors for the report-time score.
Check. June 8 is two elapsed days after June 6, and June 20 is 14 elapsed days after June 6. Both records postdate the stated decision time. No interest rate, mortality assumption, or probability distribution is assumed because the example concerns the validity of inputs, not an estimated probability.
Common mistake avoided. Treating later investigative evidence as if it had existed at first report. That would answer a different question—what can be concluded after investigation—rather than the report-time prediction task.
Why the Information-Time Rule Holds
A prediction procedure must be able to produce its output from the information actually available when it is called. Let \(t_0\) denote the decision time, let \(\mathcal{I}(t_0)\) represent the information accessible by then, and let \(Y\) be the future outcome. A usable prediction rule has the form \(f(\mathcal{I}(t_0))\): its output depends only on that information.
The justification is direct. At \(t_0\), the user can evaluate a rule only with values available at \(t_0\). If a proposed input \(Z\) is first recorded after \(t_0\), then \(Z\) is not part of \(\mathcal{I}(t_0)\), so a rule requiring \(Z\) cannot produce the intended output at that time. A retrospective analysis that supplies \(Z\) is evaluating a different information set, \(\mathcal{I}(t_0)\) plus later information. Its apparent performance does not establish that the original task is feasible.
This reasoning applies whether the later field is an outcome, a status update, a transaction, or a data correction. It also explains why a field’s predictive strength does not establish its eligibility: a highly informative value is still unusable if it arrives after the prediction is required.
Common Traps and Practical Checks
| Trap | Why it fails | Correction |
|---|---|---|
| Using a future outcome as a predictor | The answer being predicted is already revealed to the model. | Keep the outcome as the target and remove it from the inputs. |
| Using an event date as the availability date | A past event may not yet have been reported, processed, or visible. | Check when the information reached the user or system. |
| Using a later status update | The status may encode what happened after the decision. | Use only the status value accessible at the stated decision time. |
| Moving the decision time to fit the data | The revised task may no longer support the real operational decision. | Change the timing only if the later prediction point is genuinely useful. |
When reviewing a proposed task, ask: What is the exact decision time? Which person or system needs the prediction? For each candidate field, when was its value recorded and accessible? Which future event is the target? Could the same input be supplied when the task is run prospectively? These questions make hidden timing assumptions visible.
Check Your Understanding
For each case, identify what is available at the specified time and state the corrected prediction task where needed.
- A hypothetical renewal score is produced at the end of January 15 to predict cancellation by February 28. The January 15 account status is visible then; a payment dated January 18 posts January 20. Name a usable input and an unavailable input. (Answer: January 15 status is usable; the January 18 payment, posted January 20, is unavailable.)
- A claim triage decision is made at noon on April 3. A repair inspection occurred April 2, but its report is uploaded April 5. Can the report be used on April 3? (Answer: No; the report is unavailable until April 5, despite the April 2 inspection.)
- A hypothetical service team predicts at 8:00 a.m. on July 9 whether a case will be escalated by July 23. The initial case description was accessible July 8, while an escalation decision is entered July 11. Identify the input and target roles. (Answer: The July 8 description is an available input; escalation by July 23 is the future target; the July 11 decision is not an input at July 9.)
- A report-time model is intended to score a newly filed claim at 3:00 p.m. on May 1. A later extract contains the initial report, a reserve update entered May 4, and the final settlement amount. State the valid task and exclude the unavailable fields. (Answer: Use information accessible by 3:00 p.m. May 1 to predict a later specified outcome; exclude the May 4 reserve update and final settlement amount as predictors.)
- A hypothetical outreach prediction is made at the end of September 30. A transaction is dated October 4 but is shown in a file extracted on October 20. How many elapsed days separate the transaction date from the decision, and is it usable for the September 30 prediction? (Answer: October 4 is four elapsed days after September 30, so it is unavailable for the September 30 prediction.)