Validating Direct Transitions And Indirect Paths

Why the Route Matters

A prediction that a patient will be in state B at one year does not say how the patient will get there. The patient might move directly from state A to B, or move from A to an intermediate state C and then to B. Those are different outcomes, and a model can predict them with different accuracy. Validation must therefore start by identifying the exact event represented by each predicted probability.

A further distinction is essential: first reaching B by a horizon is not always the same as being in B at that horizon. Someone can enter B and then leave it before the horizon. The earlier material on transition probabilities versus state-occupation probabilities distinguishes these endpoint quantities. Here, route-specific validation uses first passage to B; a state-occupation prediction must instead be checked against occupancy at the specified time.

This definition describes the route from the starting state to the first visit to B. For example, A→C→A→B is classed as an indirect route: the first transition was to C, even though the final transition into B was from A. This convention makes the route outcomes mutually exclusive and exhaustive for whether B was first reached by the horizon: direct, indirect, or not reached. Another edge-specific target—such as whether the edge A→B is ever traversed after any number of earlier transitions—is possible, but it is a different estimand and needs a prediction for that event.

Predictions and Their Matching Outcomes

Let \(q_D(\tau)\) denote the predicted probability of the direct-route outcome and \(q_R(\tau)\) the predicted probability of first reaching B by the horizon. The corresponding indirect-route probability is \(q_I(\tau)=q_R(\tau)-q_D(\tau)\), provided both predictions refer to the same starting state, population, and horizon and use the route definitions above. A coherent set of predictions must satisfy \(0\leq q_D(\tau)\leq q_R(\tau)\leq 1\). If the model predicts the three route categories directly, their probabilities must sum to one after including the probability of not reaching B.

A transition matrix that predicts the probability of occupying B at \(t_0+\tau\) cannot automatically be substituted for \(q_R(\tau)\). It may count people who entered B and then left, and it may omit people who reached B but were elsewhere at the horizon. Unless B is absorbing, or the model separately supplies the first-passage probability, validate occupancy predictions using the observed occupancy outcome—not a route-specific first-passage outcome.

The outcomes for direct and indirect routes are competing categories in a route classification, not independent binary events. A person cannot have both \(D_{AB}=1\) and \(I_{AB}=1\) under these definitions. Likewise, a transition from A to a different state is not censoring for the direct first-transition target: it is a competing outcome. Treating such transitions as ordinary loss to follow-up would change the target probability.

Route-Specific Calibration and Scores

For a predicted direct-route probability, assess calibration against the binary outcome \(D_{AB}(\tau)\). For a predicted first-passage probability, use \(R_{AB}(\tau)\). For indirect-route calibration, use \(I_{AB}(\tau)\) and a prediction for that same route. Calibration compares predicted probabilities with observed event frequencies among individuals receiving similar predictions; calibration-in-the-large compares the average prediction with the overall observed event proportion. A good result for one route does not establish good calibration for another.

The Brier score for a binary route outcome \(Y_i\) and its predicted probability \(q_i\) is the mean squared prediction error, \(n^{-1}\sum_{i=1}^{n}(q_i-Y_i)^2\). Smaller values indicate better overall probabilistic accuracy for that outcome and cohort, but they combine aspects of calibration and discrimination. Crucially, the outcome must match the prediction: using first-passage outcomes to score occupancy predictions, or vice versa, gives a number but not a valid assessment of the stated target.

Assumptions, Checks, and a Short Justification

Route-specific validation requires predictions and observed outcomes to refer to the same origin state, target state, horizon, and population. The state definitions and event times must be recorded consistently, and the route classification must follow the stated first-passage rule. If the prediction is conditional on covariates or on a landmark history, validation must use the corresponding eligible individuals and prediction time. The observed process need not be Markov for these outcome definitions; however, a model’s own method for generating probabilities may impose assumptions that should be checked separately.

Complete follow-up to the horizon makes the route outcomes directly observable. With right censoring before the horizon, a person censored before a route is observed cannot simply be classified as “did not reach B.” Use an appropriate censoring-adjusted validation method, such as inverse-probability-of-censoring weighting, under its stated assumptions. In particular, the censoring model must support estimation of the relevant route outcome over the horizon; sparse support or highly variable weights can make the assessment unstable. A competing transition is part of the process, not a censoring event.

The key probability identity follows from the outcome definitions. If \(R_{AB}=1\), then either the first transition was directly to B or the route to the first visit to B was indirect; these possibilities are disjoint. If \(R_{AB}=0\), neither route occurred. Therefore \(R_{AB}=D_{AB}+I_{AB}\) for every individual. Taking expectations gives \(P(R_{AB}=1)=P(D_{AB}=1)+P(I_{AB}=1)\), so coherent predictions for these probabilities must obey \(q_R=q_D+q_I\). This is a route-level coherence check, not a substitute for evaluating how well each prediction matches observed outcomes.

Common Pitfalls and Alternatives

• Scoring the wrong outcome. A first-passage prediction is not validated by recording only who occupies B at the horizon. Use the first-entry outcome for first-passage predictions and the horizon state for occupancy predictions.
• Calling every A-to-B edge traversal a direct route. Under the definition here, direct means the first transition from the specified starting state goes to B. If the desired target is any later traversal of A→B, define and predict that event explicitly.
• Discarding competing transitions. A first transition to C is not ordinary censoring when assessing whether the first transition was A→B; it rules out that direct first-transition outcome.
• Inferring route performance from total performance. A model can predict total first passage well while overpredicting direct routes and underpredicting indirect routes. Report the route-specific assessments needed for the scientific question.

Route-specific first-passage validation complements, rather than replaces, validation of transition probabilities and state-occupation probabilities. It is particularly useful when the route itself has clinical or operational meaning—for example, whether a patient reaches a complication directly or after an intermediate clinical state. Its limitation is that route categories can be uncommon, making observed frequencies and scores imprecise; a good overall score can also conceal poor calibration in a particular route or subgroup. When follow-up is incomplete, route comparisons additionally depend on appropriate handling of censoring.