Why Audit a Data Specification on a Timeline?
A dataset can contain accurate records and still fail to represent the prediction task of interest. A claim recorded after a prediction date, for example, may describe an event that happened earlier; a feature calculated using that record may not have been available when the prediction would actually have been made. Likewise, an outcome label is ambiguous if its time period or event definition is not specified.
A timeline makes these issues visible. For each proposed row, identify what entity the row describes, when the prediction is made, which historical records may form predictors, when the outcome is measured, and what coverage or exposure supports that outcome. This is a practical audit of the study design, not a model-fitting exercise. It is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
Put Every Time Boundary on the Same Timeline
Use one consistent interval convention. In this tutorial, intervals are half-open: \([a,b)\) includes the start \(a\) and excludes the endpoint \(b\). Thus \([2025\text{-}04\text{-}01, 2025\text{-}05\text{-}01)\) covers the 30 calendar days from April 1 through April 30. Half-open intervals avoid counting a boundary date in both adjacent periods.
For a lookback of length \(L\) and a future horizon of length \(H\), a basic timeline is:
This sketch is only a starting point. The specification must also define whether the entity is eligible at \(t_0\), whether it has usable history, whether it remains under observation over the horizon, and how much exposure is observed. In addition, predictor data must have been available by \(t_0\), not merely describe events that occurred before \(t_0\). The earlier material on data latency and leakage distinguishes event time from data availability time; both belong in this audit.
State exactly what one row represents. If the same person can contribute multiple rows, say what distinguishes their index dates.
Specify the index date and, where relevant, the time of day or operational cutoff. Information arriving after that cutoff is not a predictor available for that prediction.
Write the predictor and outcome intervals with explicit endpoints. State whether a value is based on event date, posting date, report date, or another date.
Specify eligibility, required history, outcome follow-up, and exposure. Apply inclusion rules in a stated order and check which target population remains.
Worked Timelines
Worked Example: Auditing a Policy-Term Renewal Row
A hypothetical insurer proposes one row per commercial policy-term, with a prediction made at 00:00 on July 1, 2025. Predictors use the preceding 180 calendar days. The target is whether at least one covered loss occurs in the next 90 days. The proposed cohort begins with 1,200 policy-terms. In sequence, 90 are not active at the index date, 60 of the remaining terms lack continuous coverage throughout the required historical window, and 100 of the remaining terms lack observation through the outcome horizon. Audit the dates, counts, and exposure.
Method. First fix the index date, then construct the two half-open intervals. The predictor window is \([2025\text{-}01\text{-}02, 2025\text{-}07\text{-}01)\). It contains 30 days from January 2 through January 31, then 28 in February, 31 in March, 30 in April, 31 in May, and 30 in June:
The 90-day outcome horizon is \([2025\text{-}07\text{-}01, 2025\text{-}09\text{-}29)\). It contains 31 days in July, 31 in August, and 28 days from September 1 through September 28:
The endpoint, September 29, is excluded. A loss dated September 28 falls inside this horizon; one dated September 29 does not, under the stated event-date convention.
Apply the exclusions in the order given. After the active-at-index rule, \(1{,}200-90=1{,}110\) terms remain. After the historical-coverage rule, \(1{,}110-60=1{,}050\) remain. After the full-follow-up rule, \(1{,}050-100=950\) rows remain. Because every retained policy-term is observed for all 90 days, total outcome exposure is \(950\times90=85{,}500\) policy-days.
Check. The excluded counts sum to \(90+60+100=250\), and \(1{,}200-250=950\), matching the sequential result. The common mistake is to treat the 180-day lookback as January 2 through June 30 but calculate its days inconsistently. The stated half-open interval includes January 31 and excludes July 1, giving exactly 180 days. A separate design question remains: requiring continuous prior coverage means the retained population excludes terms without that history, so the specification should make that target population explicit.
Worked Example: Separating Event Time From Availability Time
A hypothetical health plan creates one row per enrolled member at 00:00 on April 1, 2025. The predictor window is the preceding 60 days, and the target is whether the member has an emergency-department visit during the next 30 days. A laboratory specimen was collected on March 30, but its result was posted to the plan’s data system on April 2. Decide whether the result is an eligible predictor and audit the proposed follow-up counts. Of 500 candidate rows, 40 are not eligible at the index date, 25 of the remainder lack continuous enrollment through the historical window, and 35 of the remainder lack complete follow-up through the outcome horizon.
Method. The predictor interval is \([2025\text{-}01\text{-}31, 2025\text{-}04\text{-}01)\): January 31 contributes 1 day, February contributes 28, and March contributes 31, for \(1+28+31=60\) days. The outcome interval is \([2025\text{-}04\text{-}01, 2025\text{-}05\text{-}01)\), which is 30 days.
The specimen’s event time is March 30, within the predictor window, but the result was unavailable until April 2. Since the prediction is made on April 1, the result cannot be used as a predictor for that prediction. The fact that the clinical event occurred before \(t_0\) does not make information posted after \(t_0\) historically available.
Apply the stated inclusion rules in sequence: \(500-40=460\) members are eligible at the index date; \(460-25=435\) have the required prior enrollment; and \(435-35=400\) have complete follow-up. Check: the total excluded is \(40+25+35=100\), so \(500-100=400\). Each retained member contributes 30 eligible days, for \(400\times30=12{,}000\) member-days of follow-up.
The common mistake is to include the laboratory result because the specimen date precedes April 1. The correction is to audit both event date and availability date: the result fails the availability cutoff. The specification should also say whether the target visit is dated by the visit date or by the date its claim is received; those definitions can produce different labels.
Worked Example: Finding a Leap-Year and Outcome-Definition Trap
A hypothetical vehicle-warranty analyst proposes one row per vehicle at 00:00 on January 1, 2025. The features are described as “the prior 365 days,” and the target is a warranty claim arising in the next 30 days. A data extract contains 2,400 candidate rows. In sequence, 120 vehicles lack the required historical coverage, 80 of the remaining vehicles have warranty coverage ending before the outcome horizon ends, and all other vehicles are observed through that horizon. Audit the timeline and explain how a claim reported after the horizon should be handled.
Method. A common but incorrect date choice for the 365-day lookback is \([2024\text{-}01\text{-}01, 2025\text{-}01\text{-}01)\). It spans all 366 days of leap year 2024. To obtain the stated 365 days, use \([2024\text{-}01\text{-}02, 2025\text{-}01\text{-}01)\). Since January 2 is day 2 of leap year 2024 and January 1, 2025 is the next day after day 366, the elapsed interval is \(366-1=365\) days.
The 30-day outcome horizon is \([2025\text{-}01\text{-}01, 2025\text{-}01\text{-}31)\), covering January 1 through January 30. A warranty event on January 30 is inside the horizon; one on January 31 is outside it. Apply the inclusion rules: \(2{,}400-120=2{,}280\) vehicles have the required historical coverage, and \(2{,}280-80=2{,}200\) are covered through the end of the outcome horizon. The check is \(120+80+2{,}200=2{,}400\).
Suppose a claim is reported on February 5 for a failure dated January 20. If the target is defined by failure date, it is an outcome within the horizon; the later report date concerns when the label becomes observable. If the target is instead defined as “claim reported during the 30 days,” that claim is not in the target period. The specification must choose one definition and allow sufficient time to ascertain labels. It must not quietly use a report date for some records and a failure date for others.
The common mistakes are to overlook the leap day when defining a 365-day lookback and to treat the outcome horizon and the later data-extraction date as the same boundary. The first changes which historical events qualify; the second can misclassify outcomes or leave labels incomplete.
Why the Timeline Conditions Hold
A predictor is valid for a prediction at \(t_0\) only if the information used to calculate it could have been known at that time. Let \(s\) be an event’s occurrence time and \(a\) the time the relevant value became available to the decision process. A historical event may have \(s<t_0\) but still have \(a>t_0\). Therefore a temporal predictor rule needs both a historical-event restriction and an availability restriction:
This condition follows directly from the meaning of a prediction made at \(t_0\): a value unavailable at that moment could not have informed that prediction. Filtering only on \(s<t_0\) does not establish historical availability. Likewise, an event in the future outcome interval cannot be used to construct a predictor for the same row.
Exposure follows from the outcome opportunity being studied. If the target concerns whether an event occurs while coverage is in force, the relevant observed time is the time during which both the outcome window and coverage apply. For a fully observed binary outcome over a fixed horizon, the specification normally needs complete eligibility throughout that horizon, or it must state how incomplete observation is handled. When rows have different observed periods, simply treating every row as if it had the same opportunity to experience an event can distort comparisons.
What a Complete Audit Should Record
A useful specification makes the timeline auditable without requiring the reader to infer what the author intended. Record the following for each proposed dataset:
- Entity and row frequency: for example, one policy-term at renewal, not merely “one policy.” State whether an entity can contribute multiple rows.
- Index date and cutoff: specify the prediction moment, including an operational cutoff when data arrive during the day.
- Predictor windows: state start and end boundaries, the date field used, and the data-availability cutoff.
- Outcome definition and horizon: specify the event, the date that assigns it to the horizon, and the included and excluded endpoints.
- Exposure and follow-up: state when the entity is at risk and how the specification treats partial observation.
- Inclusion and exclusion rules: list them in order, with counts where available, and describe the resulting target population.
A timeline is especially valuable when the same entity appears repeatedly. For example, monthly rows for one member have overlapping outcome horizons. The design should identify each row’s index date and define whether later rows are permitted to use information accumulated since earlier rows. This is a specification question: without a clear row definition and cutoff, it is impossible to tell whether the assembled data match the intended prediction setting.
Check Your Understanding
For each proposed design, write the key interval or audit result before checking the answer.
- A property policy has index date 00:00 on October 1, 2025, with a 30-day predictor window \([t_0-30,t_0)\). What are the interval endpoints, and how many calendar days are included? (Answer: \([2025\text{-}09\text{-}01, 2025\text{-}10\text{-}01)\), 30 days.)
- A member’s visit occurred on June 29, but the record was first available to the prediction system on July 3. The prediction cutoff is July 1. Can the record be used as a predictor if the event falls in the historical window? (Answer: No; availability was after the July 1 cutoff.)
- A 14-day outcome horizon begins at 00:00 on March 1, 2025, using half-open intervals. What is its endpoint, and which is the last included date? (Answer: \([2025\text{-}03\text{-}01, 2025\text{-}03\text{-}15)\); March 14 is the last included date.)
- A study begins with 800 candidate rows. In order, 50 fail index-date eligibility, 30 of the remaining rows lack the required history, and 20 of the remaining rows lack complete outcome follow-up. How many rows remain, and how many were excluded in total? (Answer: 700 remain; 100 excluded.)
- A warranty target is “failure during the 20 days beginning July 10,” but the specification gives no date field and no endpoint convention. Name two details that must be clarified before assigning labels. (Answer: Specify which event date defines failure timing, such as failure date versus report date, and specify the interval endpoints; under a half-open convention the interval would be \([2025\text{-}07\text{-}10, 2025\text{-}07\text{-}30)\).)