Tutorials › Competing Risks and Multi-State Models › Choosing Prediction Origins And Horizons

Transition-Specific Prediction Validation · Tutorial 389 of 699

Choosing Prediction Origins And Horizons

You will learn to specify who a prediction applies to, when its follow-up begins, and which transition event its horizon targets.

Advanced 12 min read

What You'll Learn

  • Define a transition-specific target using an origin time and a relative prediction horizon.
  • Distinguish fixed-baseline predictions from predictions made at a later landmark.
  • Translate between a relative horizon and its calendar-time endpoint.
  • Explain how conditioning on state and history at the origin changes the target population.
  • Detect mismatches in eligibility, follow-up, and information available to the prediction.
  • Choose an origin and horizon that match the intended validation question.

Why the Origin and Horizon Define the Question

A predicted probability is not fully specified by naming a transition. It also matters when follow-up begins and how far ahead the prediction looks. A prediction made at diagnosis for a transition within two years answers a different question from a prediction made one year after diagnosis for that transition in the following two years. The latter concerns people who have reached the later time and are eligible for a prediction then; it can also use information collected in the meantime.

This distinction is essential when validating multi-state predictions. The origin determines the information available and the population to which the prediction refers. The horizon determines the end of the prediction window. If either is defined differently for predictions and validation outcomes, the comparison does not assess the intended target.

Definition: Let \(X(t)\) denote the state occupied at time \(t\), and let \(s\) be a prediction origin at which \(X(s)=i\). Let \(h>0\) be the prediction horizon, measured in the same time units as \(s\). Write \(T_{\mathrm{next}}(s)\) for the time of the first transition after \(s\), and \(J_{\mathrm{next}}(s)\) for its destination state. A first-transition target for the transition \(i\to j\) is $$ q_{ij}(s,h\mid H(s)) = P\left(T_{\mathrm{next}}(s)\leq s+h,\ J_{\mathrm{next}}(s)=j\mid X(s)=i,\ H(s)\right), $$ where \(H(s)\) is the history and covariate information available at the origin. The event is that the next transition is \(i\to j\) and occurs by the horizon; a different outgoing transition before it means this target event did not occur.

This definition makes the target explicit: it is conditional on being in state \(i\) at \(s\), uses only information available by \(s\), and concerns the next transition during \((s,s+h]\). A target based on the state occupied at the endpoint, or on any occurrence of \(i\to j\) after leaving and returning to \(i\), is a different event and must be defined separately. Here the focus is choosing origins and horizons for the stated first-transition target.

The horizon \(h\) is elapsed time from the origin. Its calendar-time endpoint is \(s+h\). For example, a 12-month horizon beginning at month 6 ends at month 18. Specifying only “risk by month 18” is insufficient to distinguish a prediction from baseline from one made at month 6.

Fixed-Baseline and Landmark Origins

A fixed-baseline prediction has origin \(s=0\), or another common cohort entry time chosen as baseline. The prediction is made using baseline information \(H(0)\), and its target concerns the transition within the specified time after that origin. The target population is the baseline-eligible cohort. A person who later leaves the initial state, experiences another transition, or is censored is not retrospectively removed from this baseline population merely because they would not be eligible for a later prediction.

A landmark prediction has origin \(s>0\). It applies to people who are observed and eligible at that time, including those in the required state \(i\). The prediction can use covariate values and history known by \(s\), such as a measurement obtained during follow-up. Its target is conditional on reaching the landmark in the relevant state, so it generally describes a selected subset of the baseline cohort.

The following comparison highlights what changes when the origin changes.

FeatureFixed-baseline predictionLandmark prediction
OriginCommon baseline time, usually 0A later time \(s\), specified in advance
EligibilityEligible at baselineObserved and eligible at \(s\), usually in state \(i\)
Information usedInformation available at baselineInformation available by the landmark
Time window for horizon \(h\)\((0,h]\)\((s,s+h]\)
InterpretationRisk from baseline for the baseline-eligible cohortConditional risk from the landmark for those eligible then

Worked Example: A Prediction From Baseline

Suppose a hypothetical study follows patients from diagnosis, which is month 0. A model uses only baseline information to predict the first transition from state 1 to state 2 within 18 months. For a patient in state 1 at diagnosis, the model reports a probability of 0.14.

The origin is \(s=0\), the horizon is \(h=18\) months, and the calendar endpoint is \(s+h=18\) months. The target is $$ q_{12}(0,18\mid H(0)) =P\left(T_{\mathrm{next}}(0)\leq18,\ J_{\mathrm{next}}(0)=2 \mid X(0)=1,\ H(0)\right). $$

The reported 0.14 is a 14% predicted probability of the next transition being \(1\to2\) by month 18, conditional on the patient being in state 1 at diagnosis and on the baseline information used by the model. It is not a probability conditional on still being in state 1 at month 6. That would define a later-origin target and a different eligible group.

Landmarks, Updated Information, and Selection

At a landmark \(s\), conditioning on \(X(s)=i\) is part of the target definition, not a technical adjustment that can be omitted. People who reach that state at \(s\) can differ from the baseline cohort because of transitions, death, loss to follow-up, or other eligibility rules. A landmark prediction therefore does not automatically generalize to everyone who was eligible at baseline.

The history \(H(s)\) must also be restricted to information available at or before \(s\). A biomarker measured at month 6 can be used for a month-6 prediction, but not for a prediction made at diagnosis. Using later information for a baseline prediction leaks information from the future and changes the prediction task.

A useful justification for treating baseline and landmark questions as distinct follows from conditioning. Let \(E\) be an event defined from baseline through a later time \(\tau\), and suppose the observed history at time \(s<\tau\) is \(H(s)\). The law of total probability gives $$ P(E\mid H(0))= \sum_{x} P(E\mid X(s)=x,H(s))P(X(s)=x,H(s)\mid H(0)), $$ where the sum represents a sum or integral over the possible states and histories at \(s\). A conditional probability for people with \(X(s)=i\) is just one part of this mixture. It cannot in general be substituted for the baseline probability: the baseline prediction averages across possible histories and states, with their probabilities under baseline conditions. A landmark prediction instead conditions on the history actually observed for an eligible person at \(s\).

This argument also explains why a baseline prediction for an event by \(\tau\) is not generally the same as a prediction made at \(s\) for the remaining interval \((s,\tau]\). The latter is conditional on what has happened by \(s\), including the person’s state. Even if both predictions end at calendar time \(\tau\), they can differ because their information sets and conditioning populations differ.

Worked Example: A Prediction at a Six-Month Landmark

In a hypothetical follow-up, a patient is in state 1 at month 6 and has a biomarker measured at that visit. A model using the patient’s month-6 history predicts a 0.22 probability that the next transition will be \(1\to3\) within 12 months.

Here \(s=6\) months and \(h=12\) months, so the calendar endpoint is $$ s+h=6+12=18\text{ months}. $$ The target is $$ q_{13}(6,12\mid H(6)) =P\left(T_{\mathrm{next}}(6)\leq18,\ J_{\mathrm{next}}(6)=3 \mid X(6)=1,\ H(6)\right). $$

Thus, 0.22 is a 22% predicted probability for a patient who is observed in state 1 at month 6, using information available by month 6. The prediction does not apply to a patient who was in state 1 at diagnosis but is in another state at month 6. It also does not mean a 22% probability from diagnosis through month 18: the origin and eligibility conditions are different.

Choosing Between a Fixed Horizon and a Fixed Calendar Endpoint

Two common designs answer different questions. With a fixed elapsed-time horizon, each origin uses the same \(h\). A 12-month horizon from month 6 ends at month 18, while the same horizon from month 12 ends at month 24. This design asks about risk over the same amount of time after each prediction.

With a fixed calendar endpoint \(\tau\), the horizon depends on the origin: \(h=\tau-s\). A prediction made at month 6 for outcomes by month 24 has an 18-month horizon; one made at month 12 for outcomes by month 24 has a 12-month horizon. This design asks about reaching an endpoint shared across origins, not about equal lengths of follow-up.

A fixed calendar endpoint is meaningful only when \(\tau>s\), so that \(h>0\). Origins close to \(\tau\) produce short horizons; an origin at or beyond \(\tau\) cannot support a positive-horizon prediction to that endpoint. In reporting either design, state both the origin and the horizon (or the endpoint and the origin), the target state, and the eligibility conditions.

Worked Example: Two Origins and One Calendar Endpoint

A hypothetical validation plan considers patients in state 1 at two landmarks and asks about a first transition \(1\to2\) by month 24. At month 6, 100 eligible patients each have a predicted probability of 0.20 for the event by the endpoint. At month 12, 80 eligible patients each have a predicted probability of 0.30.

For the month-6 predictions, the horizon is \(h=24-6=18\) months. The sum of their predicted event counts is $$ 100(0.20)=20. $$ For the month-12 predictions, the horizon is \(h=24-12=12\) months, and the corresponding sum is $$ 80(0.30)=24. $$

Both sets of predictions refer to month 24, but they use different horizons and eligible groups. The calculations give sums of predicted events within each set, not observed event counts and not evidence that one group has higher risk than the other. Comparing the predictions without noting the different landmark conditions would conceal the change in population and prediction window.

Assumptions, Diagnostics, and Common Failure Modes

Choosing an origin and horizon does not by itself guarantee valid predictions, but the target is interpretable only when the design is coherent. At minimum, specify a meaningful time origin and scale, a target state and transition, eligibility at each origin, the information available to the prediction, and a positive horizon. The prediction model must be intended for that state, time range, and information set. For a landmark analysis, the required state and eligibility must be ascertainable at the landmark; for a baseline analysis, eligibility must be defined at baseline.

Before validation, check that every predicted probability and its corresponding outcome target use the same origin, horizon, transition definition, and eligibility rule. Check timestamps and variable-availability dates, not just variable names. If follow-up ends before the horizon for some individuals, their outcomes may not be fully observed; handling that issue requires an appropriate validation approach, rather than silently treating an unobserved event as absent.

  • Mixing baseline and landmark populations. Comparing baseline predictions with outcomes among only those who remain eligible at a later landmark changes the target population. Keep the eligibility rule aligned with the origin.
  • Confusing elapsed time with calendar time. “By month 24” and “within 12 months” coincide only for an origin at month 12. Write the origin and endpoint explicitly.
  • Using future information. A covariate measured after the origin cannot be used in that origin’s prediction. Verify that each predictor was available by the prediction time.
  • Changing the event definition. A next-transition target is not interchangeable with a target based on endpoint state or any later occurrence of a transition. State which event is being evaluated.

A common mistake is to say, “Both predictions end at month 24, so they target the same probability.” That reasoning overlooks the conditioning and horizon. A prediction from month 6 for the next 18 months is conditional on the person’s state and history at month 6; a prediction from month 12 for the next 12 months is conditional on a later history and on eligibility at month 12. A shared calendar endpoint alone does not make these conditional targets identical.

The practical choice depends on the intended use. A baseline prediction is appropriate when the question is what can be predicted at cohort entry using baseline information. A landmark prediction is appropriate when predictions will be updated during follow-up for people who are still eligible at that time. Neither is universally preferable. The important limitation is that validation results apply to the target population and prediction window actually specified; they should not be interpreted as performance for other origins or horizons without further assessment.

Key takeaway: A transition-specific prediction target is defined by the transition, origin, horizon, information available at the origin, and eligibility conditions. Changing the origin can change both the prediction window and the population, even when the calendar endpoint stays fixed.

Check Your Understanding

For each question, identify the target or explain the relevant distinction. Solutions follow each question.

  1. A model predicts a \(2\to3\) next-transition probability at month 4 for the next 9 months. What is the calendar endpoint, and what is the target conditioned on?
    Solution: The endpoint is \(4+9=13\) months. The target is the probability that the next transition is \(2\to3\) by month 13, conditional on being in state 2 at month 4 and on the history available by month 4.
  2. Two predictions are made at months 3 and 8, each for a fixed 6-month horizon. Do they share a calendar endpoint?
    Solution: No. Their endpoints are month 9 and month 14, respectively. They use equal elapsed-time horizons but different calendar endpoints.
  3. A prediction made at diagnosis uses a biomarker measured at month 5. What is the problem?
    Solution: The prediction uses information unavailable at its origin. This is future-information leakage; either the origin or the predictor information set must change.
  4. Predictions from month 6 and month 12 both concern a transition by month 24. Explain why their conditional targets may differ.
    Solution: The month-6 prediction has an 18-month horizon and conditions on eligibility and history at month 6. The month-12 prediction has a 12-month horizon and conditions on eligibility and history at month 12. The later eligible population and information can differ, so a shared endpoint does not make the targets identical.
Clinical Biostats