Define the Prediction-Time Information Set

Start With the Moment a Prediction Must Be Made

A variable may be recorded in a data set and still be unusable for a forecast. The deciding question is not merely whether the variable is related to the outcome, or whether it was collected somewhere in the project. It is whether its value would be available when the intended user needs the prediction. A model that relies on information arriving later may look accurate in historical analysis but cannot be used as intended.

To make that boundary precise, specify the prediction origin, the prediction horizon, the target outcome, and the intended user. The origin is the time at which the prediction is made. The horizon states how far ahead the outcome refers to. The intended user and their workflow clarify what information can actually be accessed at that moment.

Definition: The prediction-time information set, written here as \(I_t\), is the collection of case-specific information available to the intended user at prediction origin \(t\). A candidate predictor is eligible only if its value, as used in the model, can be determined from \(I_t\). The target is the outcome to be predicted at the stated horizon, not information that becomes available only after the origin.

The information set is defined for a particular prediction task, not once and for all. For example, a clinic manager making a forecast on Friday afternoon may have a different information set from a manager making the same forecast on Monday morning. The outcome and forecast horizon can be identical while the eligible predictors differ.

Define the Task Before Screening Variables

A useful task statement answers four questions in operational terms:

Who will use the forecast? Name the person or system and the decisions it supports.
When is the prediction made? Specify the prediction origin, including a practical cutoff time when necessary.
What outcome and horizon are forecast? Define what will be measured and the future period to which it refers.
What is known at that time? List the records, plans, and other information the user can access by the cutoff.

For regression, one may represent the candidate model as a prediction of an outcome \(Y\) from predictors \(X_1,\ldots,X_p\). In a forecasting task, write the target as \(Y_{t+h}\), where \(t\) is the prediction origin and \(h\) indicates the horizon. This notation emphasizes that predictor values must be available at \(t\), even though the target concerns a later time.

\widehat{Y}_{t+h}=f(X_1(t),\ldots,X_p(t)), \qquad X_j(t)\text{ is eligible only if it is available in }I_t.

This is an eligibility rule, not a claim that every eligible variable should be used. A candidate may be available but irrelevant, redundant, poorly measured, or inappropriate for other reasons. Conversely, a variable that is strongly associated with the outcome is not eligible if its value is unknown when the prediction must be made. Eligibility comes before questions about model fit or coefficient tests.

The rule does not require every input to be a raw observation. A summary calculated from records already available at \(t\) can qualify, provided its calculation uses no later information. Likewise, a model trained on earlier data can have fitted coefficients available for deployment: those coefficients are part of the prediction system. They are different from case-specific future measurements that the user cannot yet know.

Why the Information Boundary Follows From the Task

A prediction is useful only if it can be computed in the circumstances for which it is intended. Suppose the deployed prediction rule uses a value \(X_j(t)\) that is not available until time \(t+u\), where \(u>0\). At origin \(t\), the user cannot evaluate the rule as written. Replacing that value with its later realization silently changes the information available to the forecast and therefore changes the task.

The reasoning can be stated directly:

Fix the prediction origin.
Record the moment the user must receive or act on the forecast.

Set the information boundary.
Include only case-specific information accessible by that moment, under the intended workflow.

Filter candidate predictors.
Keep a candidate value only if it can be produced from information inside that boundary.

Preserve the boundary in evaluation.
Assess predictions using the information that would have been available at each historical prediction origin.

This establishes a practical justification for the rule: if \(X_j(t)\) is not in \(I_t\), then a prediction rule depending on \(X_j(t)\) cannot be evaluated from the information available at \(t\). A later value may be useful for explaining what happened afterward, but inserting it into a forecast made at \(t\) does not reproduce a usable forecast.

Worked Example: Forecasting Clinic Appointments

A clinic manager wants a forecast each Friday at 5 p.m. of the number of patients who will attend appointments during the following Monday through Sunday. The intended user is the manager, who uses the forecast to arrange staffing. Consider whether three candidates belong in the information set: next week’s appointment schedule as recorded by Friday at 5 p.m., actual next-week attendance, and a weather forecast issued by Friday at 4 p.m.

Define the task. The prediction origin is Friday at 5 p.m.; the horizon is the following seven-day appointment week; the target is the number of appointments attended during that week; and the user is the clinic manager.

Apply the boundary. The schedule currently recorded is available by the origin, so its value can be used. Actual attendance during the following week occurs after the origin and is part of the future outcome period, so it is unavailable and must be excluded. The weather forecast issued at 4 p.m. is available to the manager by 5 p.m.; it may therefore be an eligible candidate, assuming the manager can access it in the stated workflow.

The weather forecast’s eligibility does not mean that it must improve the model. It means only that a prediction can use the issued forecast value at the stated origin. Substituting the actual weather observed next week would change the input: that realization was not yet known on Friday.

Availability Depends on What Is Known, Not Just the Date Label

A column name or calendar date is not enough to decide availability. A value may refer to a future period but already be known at the origin, or it may describe the past but not yet have been entered, processed, or released to the intended user. The operational question is whether that particular value can be accessed at prediction time.

For instance, a planned price for next month may be available if it has been approved and recorded before the forecast is made. The actual price charged next month is not available then. A forecast of future weather issued before the origin may be available, while the subsequently observed weather is not. A report describing last month’s outcome might still be unavailable if it is not published until after the prediction cutoff.

These distinctions matter because historical data sets often store final values without showing when those values first became available. A forecasting analysis should represent the information as it existed at the prediction origin, rather than assuming that everything now present in the data table was known then. When availability is uncertain, state the operational assumption explicitly and check that it matches the intended user’s actual access.

Worked Example: Predicting Late Delivery at Order Acceptance

A hypothetical distributor wants to estimate whether an order will arrive late. The prediction is made when the order is accepted, so the target is late or on-time delivery for that order. At acceptance, the system records the destination, the promised delivery date, and the customer’s late-delivery history calculated from completed earlier orders. The eventual arrival date and the result of the final quality check are also present in the completed-order database.

Set the origin and target. The origin is order acceptance. The target is the delivery status of that order, determined later. The intended user is the staff member deciding whether extra monitoring is needed.

Check each candidate. Destination and promised delivery date are known at acceptance, so both pass the availability rule. The customer’s history also passes if it is calculated using only orders completed before this order is accepted. The eventual arrival date fails: it is determined later and directly reveals information about the target. The final quality-check result fails if that check occurs after acceptance, because it is not yet available to the user.

The historical summary can be expressed as a calculation over eligible records. If \(A_t\) is the set of the customer’s orders completed before origin \(t\), and \(L_i\) equals 1 for a late order and 0 otherwise, then the prior late proportion is

\frac{\sum_{i\in A_t} L_i}{|A_t|}.

For example, if 3 of the 12 previously completed orders were late, the summary is \(3/12=0.25\), or 25%. This value is eligible because the numerator and denominator use only completed prior orders. Including the current order or any orders completed after acceptance would violate the stated boundary.

Training Data Must Recreate Prediction-Time Knowledge

Historical examples are often assembled after all outcomes have occurred. That makes it easy to mistake information available to the analyst now for information that would have been available to a user then. A model can appear to predict well when it uses a later outcome-related field, but this performance does not show that the model could have produced those predictions at the original origins.

For each historical case, the candidate values should therefore correspond to the information available at that case’s prediction origin. The same rule applies to summaries and derived variables: compute them only from records that had entered the information set by then. If a predictor is unavailable in deployment, it should not be rescued by being present in the retrospective file.

This is not a ban on using historical data to estimate the model. Training records and estimated coefficients can be used to build a prediction system. The distinction is between information used to learn the general prediction rule and case-specific information required to make a particular forecast. The latter must be available to the intended user at the origin.

Worked Example: A Monthly Sales Forecast

A retailer makes a forecast on the first day of November for total sales in December. The user is a regional manager allocating staff. By the forecast date, the calendar dates for December are known, a December promotion plan has been approved, and the December sales total is not known. The data system also contains final December inventory levels and the number of December days on which each store was closed.

Calendar dates pass the rule because they are known in advance. The approved promotion plan also passes, provided the manager can access the approved plan on November 1. The December sales total is the target and is not an input.

The final inventory levels and actual closure days fail if they are recorded as December unfolds. They cannot be used as their eventual realized values in a forecast issued on November 1. If the retailer instead has an inventory plan or an already known closure schedule on November 1, those planned values could be considered as separate candidates: they are not the same information as the eventual realizations.

Thus the eligible set from these candidates consists of the calendar information and the accessible, approved promotion plan. This screening does not establish which eligible variable will be useful, nor does it endorse a particular regression form. It establishes that any model used for this task can be computed at the time the manager needs it.

Common Mistake: Treating Retrospective Data as Available Data

A common mistake is to reason, “The variable is in the completed data set and is strongly associated with the outcome, so it belongs in the prediction model.” That reasoning confuses what the analyst can see after data collection with what the user knew when a forecast was required. A later measurement may even encode part of the outcome itself.

The correction is to define the origin and user first, then ask when each candidate value becomes available. Exclude a candidate if its value is not available at the origin, even if it is statistically informative. For eligible candidates, evaluate their predictive usefulness only after this operational screen. A useful final check is to describe, in one sentence, exactly how the deployed system will obtain each input before producing a forecast.

A prediction-time information set makes a predictive specification operational. It prevents a model from depending on future knowledge, clarifies which planned or forecasted values can be used, and helps ensure that a forecast assessed on historical cases could actually have been made at the time. It does not decide whether a predictor is scientifically important or whether it should be included for explanatory purposes; those are different specification questions.

Check Your Understanding

For each question, state the prediction-time boundary before deciding which information is usable.

A transit agency forecasts tomorrow’s passenger count at 6 p.m. today. Is the count of passengers actually carried tomorrow an eligible predictor? Explain.
A hospital makes a forecast at patient admission. A diagnosis entered the next morning is in the completed record. What information determines whether that diagnosis can be a predictor?
A business forecasts next month’s demand using a promotion plan approved before the forecast date. Explain why the planned promotion and the eventual promotion actually delivered are distinct candidate values.
Give the four elements that should be stated when defining a prediction task.
A historical summary uses a customer’s records, including orders completed after the prediction origin. What is wrong with that summary, and how should it be recalculated?