Estimating Observed Transition Probabilities For Validation

From a Prediction Target to an Observed Estimate

A model may predict the probability that a person in a particular state at a prediction origin will be in a specified state after a fixed horizon. Validation needs an observed estimate of that same quantity in a validation sample. The estimate must use the event history up to the horizon, not merely the fraction whose records happen to be complete. For a multi-state process, the Aalen–Johansen estimator combines observed transition counts over time into a transition-probability matrix.

Suppose the states are numbered \(1,\ldots,K\), and a validation cohort is defined at a common origin time \(t_0\). The target for people starting in state \(r\) is the probability of occupying state \(s\) at horizon \(t_0+\tau\), conditional on being in state \(r\) at \(t_0\). Denote it by \(P_{rs}(t_0,t_0+\tau)\). If people can move through intermediate states, this is an endpoint state probability: it includes every path that ends in \(s\) by the horizon.

Definition: For each state \(j\), let \(Y_j(u)\) be the number at risk in state \(j\) immediately before time \(u\), and let \(dN_{jk}(u)\) be the number of observed transitions from \(j\) to \(k\) at \(u\), for \(j\ne k\). The estimated transition-rate increment matrix at an event time \(u\) has off-diagonal entry \(d\widehat A_{jk}(u)=dN_{jk}(u)/Y_j(u)\) and diagonal entry \(d\widehat A_{jj}(u)=-\sum_{k\ne j}d\widehat A_{jk}(u)\). The Aalen–Johansen estimate from \(t_0\) to \(t\) is the chronological product \(\widehat{\mathbf P}(t_0,t)=\prod_{t_0<u\le t}\{\mathbf I+d\widehat{\mathbf A}(u)\}\), where \(\mathbf I\) is the identity matrix. Its \((r,s)\) entry estimates \(P_{rs}(t_0,t)\).

At an event time, the matrix \(\mathbf I+d\widehat{\mathbf A}(u)\) updates the current distribution across states. Its row for state \(j\) puts the observed transition fractions in the destination states and leaves the remaining fraction in \(j\). Matrices are multiplied in time order: if \(\mathbf Q_1\) and \(\mathbf Q_2\) are the updates at two successive event times, the overall update is \(\mathbf Q_1\mathbf Q_2\). The product therefore adds contributions from all possible intermediate states.

Assumptions and Estimation Steps

The estimate is meaningful only when the validation cohort, state definitions, prediction origin, and horizon match the model target. The observations must also support estimating the relevant transition rates. In particular, there must be people at risk in a state when its transitions are observed; if \(Y_j(u)=0\), a transition fraction from that state cannot be calculated.

For the usual interpretation as transition probabilities, the transition process is assumed to be Markov on the states used: given the current state and time, future transitions do not additionally depend on the earlier path. Independent right censoring is also required, at least conditional on variables used to define or stratify the analysis. In practical terms, people censored at a given time should represent those still observed at risk with respect to their subsequent transition experience. If censoring depends on measured covariates, a suitable stratification or model may be needed; if it depends on unobserved future outcomes, the ordinary estimator may be biased.

Censoring affects the risk sets, not the transition counts. A censored person contributes time at risk until censoring, then contributes no further event information. The estimator does not treat censoring as a transition into a substantive state. To make an observed-versus-predicted comparison, estimate the observed probability in the same target population as the predictions. For a group-level comparison, this commonly means comparing the observed estimate with the average of the individual predictions for members of that group.

Set the target.
Specify the origin state, prediction origin, horizon, and destination state. Confirm that cohort membership is defined at the origin.

Tabulate transitions and risk sets.
At each observed event time, count transitions out of each state and the number at risk in that state just beforehand.

Form and multiply updates.
Construct \(\mathbf I+d\widehat{\mathbf A}(u)\) at each event time and multiply the matrices in chronological order through the horizon.

Compare like with like.
Read the relevant matrix entry and compare it with predictions for the same origin state, horizon, destination, and validation population.

Worked Matrix Calculations

Worked Example: Two Event Times and All Paths to the Horizon

Consider a hypothetical cohort of four people, all in state 1 at the prediction origin. There are three states. At time 1, one person moves from state 1 to state 2 and another moves from state 1 to state 3; the other two remain in state 1. At time 2, one of the two people still in state 1 moves to state 2, and the person in state 2 moves to state 3. There is no censoring before the horizon just after time 2.

At time 1, \(Y_1(1)=4\). Thus the estimated fractions moving from state 1 to states 2 and 3 are each \(1/4\). The update matrix is

\mathbf Q_1= \begin{pmatrix} 1/2&1/4&1/4\\ 0&1&0\\ 0&0&1 \end{pmatrix}.

At time 2, two people are at risk in state 1, and one is at risk in state 2. The state 1 to state 2 fraction is \(1/2\), while the state 2 to state 3 fraction is \(1\). Therefore

\mathbf Q_2= \begin{pmatrix} 1/2&1/2&0\\ 0&0&1\\ 0&0&1 \end{pmatrix}.

The initial distribution is the row vector \((1,0,0)\). Multiplication gives

(1,0,0)\mathbf Q_1\mathbf Q_2 =(1/2,1/4,1/4)\mathbf Q_2 =(1/4,1/4,1/2).

For example, the probability estimate for occupying state 1 is \((1/2)(1/2)=1/4\). For state 2 it is \((1/2)(1/2)=1/4\). For state 3 it is \((1/2)(0)+(1/4)(1)+(1/4)(1)=1/2\). The last calculation includes both people who had already reached state 3 by time 1 and those who reach it from state 2 by time 2. The estimated state probabilities sum to \(1\), as they should.

Worked Example: How Censoring Changes the Risk Set

In a hypothetical cohort of five people starting in state 1, one person moves from state 1 to state 2 at time 1. A second person is censored after time 1 but before time 2, while in state 1. At time 2, one of the three remaining people in state 1 moves to state 3. The person in state 2 has no observed transition by the horizon just after time 2.

At time 1, all five people are at risk in state 1. The update has state 1 row \((4/5,1/5,0)\). At time 2, the censored person is no longer at risk, so \(Y_1(2)=3\), not 4. One of those three moves to state 3, giving state 1 row \((2/3,0,1/3)\). The state 2 row is unchanged. The two update matrices are

\mathbf Q_1= \begin{pmatrix} 4/5&1/5&0\\ 0&1&0\\ 0&0&1 \end{pmatrix}, \qquad \mathbf Q_2= \begin{pmatrix} 2/3&0&1/3\\ 0&1&0\\ 0&0&1 \end{pmatrix}.

Starting in state 1, the estimated horizon distribution is

(1,0,0)\mathbf Q_1\mathbf Q_2 =(4/5,1/5,0)\mathbf Q_2 =(8/15,1/5,4/15).

Numerically, these estimates are \(0.5333\) in state 1, \(0.2000\) in state 2, and \(0.2667\) in state 3. They sum to 1. The censoring changes the second update’s denominator because only three people remain at risk in state 1. This calculation assumes the censoring mechanism is independent of future transitions, conditional on the information defining the analysis.

Worked Example: Comparing a Group Estimate With Predictions

Suppose a hypothetical validation group contains four people, all in state 1 at origin. At time 1, one moves to state 2. At time 2, one of the three remaining in state 1 moves to state 3, and the person in state 2 moves to state 3. Assume no censoring and a horizon just after time 2. The first update has state 1 row \((3/4,1/4,0)\). The second update has state 1 row \((2/3,0,1/3)\) and state 2 row \((0,0,1)\).

The observed Aalen–Johansen estimate is

(1,0,0) \begin{pmatrix} 3/4&1/4&0\\ 0&1&0\\ 0&0&1 \end{pmatrix} \begin{pmatrix} 2/3&0&1/3\\ 0&0&1\\ 0&0&1 \end{pmatrix} =(1/2,0,1/2).

Suppose the model’s predicted probabilities, averaged over these same four people, are \(0.45\) for state 1, \(0.20\) for state 2, and \(0.35\) for state 3. The observed-minus-predicted differences are \(0.50-0.45=0.05\), \(0-0.20=-0.20\), and \(0.50-0.35=0.15\). These are descriptive group-level discrepancies in probability, not evidence by themselves of why the model differs. The observed and predicted probabilities each sum to 1, and so do their differences: \(0.05-0.20+0.15=0\).

Why the Product Works, and What to Check

The product has a direct probability interpretation. Just before an event time, suppose the estimated distribution over states is a row vector \(\boldsymbol p\). After the event-time update, it is \(\boldsymbol p(\mathbf I+d\widehat{\mathbf A})\). The entry for destination state \(s\) is a sum over current states \(j\): the estimated probability of being in \(j\), multiplied by the update fraction from \(j\) to \(s\), then added across \(j\). Repeating this update over event times carries forward probability mass along every observed transition path. This explains why the estimator is a matrix product rather than a single transition count divided by the initial sample size.

A practical check is that every update matrix and the final estimated transition matrix have nonnegative entries and rows summing to 1, apart from possible numerical rounding. Check that event counts do not exceed the corresponding risk sets, that the risk set is measured immediately before each event time, and that the matrices are in chronological order. Also confirm that censoring is not mistakenly entered as a clinical transition.

Very small state-specific risk sets can produce unstable transition fractions: one observed transition among two people at risk contributes \(1/2\). Report the horizon and the amount of data supporting the estimate, and consider uncertainty intervals or resampling methods when making a formal comparison. A smooth-looking point estimate does not guarantee precision. If the Markov assumption is implausible—for example, transition experience depends strongly on time already spent in a state—an ordinary Aalen–Johansen product may not estimate the desired conditional transition probabilities.

The estimator is preferable to a raw fraction among uncensored people because it uses each person’s observed follow-up through their censoring time. It is also distinct from simply applying a single transition fraction to the initial cohort: people can move through intermediate states, and the product accounts for their subsequent opportunities to transition. The estimate is not automatically individualized. If transition patterns differ substantially across covariates, a pooled estimate may not be an appropriate observed comparator for individual predictions; use a defined subgroup or a suitable standardization strategy so observed and predicted quantities refer to the same population.

Common Pitfall

A common error is to calculate the fraction of the original cohort observed in the destination state at the horizon and call it the transition probability. For instance, people censored before the horizon do not have a known state at that horizon; excluding them can bias the fraction if censoring is informative, while counting them as having no transition is incorrect. The correction is to form state-specific risk sets at each event time and use the Aalen–Johansen product under the stated censoring and Markov assumptions. Another error is to multiply the matrices in reverse order. Since matrix multiplication is generally not commutative, the event-time updates must be applied in chronological order.

Key takeaway: For a fixed horizon, the Aalen–Johansen estimate is the relevant entry of the chronological product of empirical transition updates. A defensible validation comparison requires the same origin, horizon, states, and target population for the observed estimate and model predictions.

Check Your Understanding

Answer each question, then check the solutions below.

At an event time, \(Y_1=8\), with one transition from state 1 to state 2 and two from state 1 to state 3. What is the state 1 row of the update matrix?
Why does a person censored before a later event time not remain in the risk set at that later time?
For two successive event-time matrices \(\mathbf Q_1\) and \(\mathbf Q_2\), which product gives the correct update from the initial distribution?
State two assumptions needed for the usual transition-probability interpretation of the Aalen–Johansen estimator.

Solutions. (1) The off-diagonal fractions are \(1/8\) and \(2/8\), so the remaining fraction is \(1-1/8-2/8=5/8\). The row is \((5/8,1/8,2/8)\). (2) Censoring ends that person’s observed follow-up, so their state and transition experience after censoring are unknown and they no longer contribute to the observed risk set. (3) Use \(\mathbf Q_1\mathbf Q_2\), applying the earlier update first. In general, reversing the order changes the result. (4) The usual interpretation requires a Markov model for the specified states and independent right censoring, at least conditional on variables used to define or stratify the analysis.