Organizing Repeated Landmark Data for Direct Prediction

From Longitudinal Histories to Prediction Rows

A longitudinal multi-state record follows a person over time, while a direct landmark prediction asks a specific question: given the person’s observed history at a chosen time, what state will the person occupy after a specified horizon? Organizing the data means making that question explicit in every row. A row is not merely a person or a visit; it represents a person at a landmark and the future prediction occasion attached to that landmark.

This construction uses the landmark idea that predictions are conditional on information available at a landmark. The earlier material on landmarking and dynamic prediction establishes the prediction target. Here the focus is the data structure needed to represent it without mixing information from different times or treating an unknown future state as an observed outcome.

For example, if the states are healthy, ill, and dead, a record at landmark \(s\) can use the current state, earlier history, and covariates observed by \(s\) to target the state at \(s+h\). The future target is one state in \(\mathcal S\); it is not the transition path between \(s\) and \(s+h\). If the scientific question is instead about a path or a particular transition, a different outcome structure is needed.

A repeated-landmark data set therefore has a subject identifier and a landmark identifier or time, predictors evaluated at that landmark, and a future outcome for the specified horizon when observed. The pair \((i,s)\) should uniquely identify a row for a fixed horizon. If several horizons are analyzed, include the horizon in the key as well. A single person can contribute several rows, but each row must refer to its own landmark-specific history and target.

Constructing the Rows

Start with longitudinal records that establish state and covariate histories over time. Choose the landmark schedule and horizon before generating outcomes. Then, for each person and landmark, determine whether the person belongs to the target population at that time; retain only information available by the landmark; and determine whether the state at the target time is observed. A state observed after the landmark must never be carried backward into the predictors.

The following hypothetical histories use equally spaced times \(0,3,6,9\), with states healthy (H), ill (I), and dead (D). The fixed horizon is \(h=3\). For illustration, landmarks are at \(0,3,6\), and a person who has died is no longer eligible for a later landmark.

Precisely Defining Predictors and Outcomes

The landmark predictor vector \(Z_i(s)\) may contain baseline covariates, the state at \(s\), and summaries of history observed up to \(s\), such as time already spent in a state. Its content depends on the intended prediction rule, but the timing rule does not: every component must be known at or before \(s\). If a covariate is only recorded at visits, use the value or history genuinely available under the planned prediction procedure; do not silently substitute a later measurement.

A direct state-occupation outcome is categorical. For a known target state, it can be represented with indicators \(I\{Y_i(s,h)=k\}\), one for each state \(k\in\mathcal S\). Exactly one indicator equals one and the rest equal zero. This representation is a property of an observed, known outcome; it does not mean that an outcome is known merely because a row has been created.

The outcome definition also fixes the target population. For example, if predictions are intended for people alive and under observation at landmark \(s\), the eligibility rule must say so. If death is one of the possible states at the horizon, a person who is alive at \(s\) and dies before \(s+h\) has a valid future state of D, provided that death is ascertained. Excluding such a person because death occurred before the target time would change the prediction target.

Why the Reshaping Rules Matter

The basic logic can be stated directly. For a fixed eligible pair \((i,s)\), the desired prediction is conditional on the information \(Z_i(s)\) available at that time, and the target is the state at \(s+h\). If instead a predictor uses information observed after \(s\), it describes a different conditioning set. If instead the outcome is recorded at a different time, it describes a different target. Therefore a row is valid for the intended prediction only when both its information cutoff and target time match its declared landmark and horizon.

When \(Y_i(s,h)\) is known, the one-hot indicators obey

This identity follows because the observed target belongs to exactly one state in the defined state set. It provides a simple structural check: a known categorical outcome should map to precisely one target category. It does not apply to a target that is unobserved; coding an unobserved value as all zeros would incorrectly conflate missingness with a known outcome.

Repeated Rows, Dependence, and Data Splitting

The rows are prediction occasions, not necessarily independent people. A subject can contribute several rows, and adjacent prediction windows can overlap. For example, the outcome at time 6 for a landmark at 3 may also be part of the history used for a landmark at 6. This is not automatically an error: the later row is a new prediction occasion with a later information cutoff. It does mean that any later statistical analysis must account for the actual repeated-subject structure where required.

The row count also encodes a weighting choice. If each eligible subject-landmark pair contributes equally to a model’s fitting criterion, a person contributing five landmarks has five times as many rows as a person contributing one. That can be appropriate when the target is performance over prediction occasions. If the intended target gives equal weight to people, the analysis may need a different weighting or sampling design. The reshaping step should preserve the rows faithfully and make this choice visible rather than implicitly treating the row count as the number of distinct people.

For data splitting or resampling intended to assess predictions on new people, assign all records from a given individual to the same partition. Splitting rows at random can place one person’s early landmark in training and later landmark in testing. The resulting assessment may then benefit from person-specific information shared across the split and no longer represent performance on new individuals.

Diagnostics, Alternatives, and Limitations

Several checks catch common construction failures before any prediction model is fitted:

• Check that the key made from subject, landmark, and (if applicable) horizon has no unintended duplicates.
• Confirm that each target time equals the landmark plus the declared horizon and that target states belong to the defined state set.
• Verify that no predictor uses a measurement recorded after its row’s landmark.
• Check that records after an absorbing terminal state, such as death, are excluded when the target population requires the person to be alive at the landmark.
• Tabulate known and unobserved outcomes by landmark, target state, and reason for unobserved status; inspect whether any landmark has little or no support.
• For known outcomes, verify that exactly one state indicator is one. Keep unknown outcomes distinct from every state.

A common mistake is to treat every eligible landmark row as a complete case and assign the last observed state to the future target. The reasoning is that the last state is the best available information. The correction is that a state at time \(u<s+h\) is not, by itself, the state at \(s+h\): a transition may occur in between. The target must be ascertained at the specified time, or its missingness must be handled using an appropriate method with explicit assumptions.

An alternative is to organize data in a wide format, with one row per person and separate columns for each landmark’s predictors and outcome. That can be convenient for a small, fixed set of landmarks, but it makes the landmark-specific information cutoff and outcome pairing harder to audit and is less flexible when individuals have different numbers of eligible landmarks. A long format makes the prediction occasion explicit. Transition-specific models are another way to obtain state-occupation predictions, but they organize information around transitions and their model components; the present structure is for direct landmark-to-future-state records, not a tutorial on fitting either approach.

Reshaping cannot fix insufficient observation or define the scientific target on its own. The state process, landmark eligibility, horizon, and handling of incomplete follow-up must be defensible for the intended use. A correctly keyed table can still have biased or unsupported outcomes if follow-up patterns, state ascertainment, or the target population are misunderstood.