Markov State-Occupation Prediction From a Landmark

From a Landmark to a Future State Distribution

At a landmark, the prediction starts from information available at that time: the individual’s current state and specified covariate values. The target is the probability of occupying each possible state at a future prediction time, conditional on that landmark information. This is a model-based prediction, not a statement that the covariates cause a particular outcome.

The key calculation is to start with all probability in the observed landmark state and propagate that probability through the fitted transition law. For a prediction over several intervals, the order of the transitions matters: the distribution after one interval becomes the starting distribution for the next.

The Markov assumption here means that, conditional on the current state and the covariates included in the model, the future transition law does not additionally depend on the earlier state path. It is a substantive modeling assumption. If two individuals with the same current state and modeled covariates can have different future transition probabilities because of their earlier histories, a Markov model with this conditioning information may be inadequate.

Propagating Fitted Transition Matrices

Use a row-vector convention. Suppose the states are \(1,\ldots,K\), and let \(P(a,b\mid z)\) be the \(K\)-by-\(K\) transition matrix from time \(a\) to time \(b\), conditional on the specified covariates. Its \(i,j\) entry is the probability of being in state \(j\) at \(b\), given state \(i\) at \(a\). Let \(e_r\) be the row vector with a 1 in position \(r\) and 0 elsewhere. If the fitted transition law is constant over the prediction interval, the predicted distribution is

For successive intervals with different fitted transition matrices, multiply them in chronological order. If \(L=t_0<t_1<\cdots<t_m=t\), then

The resulting \(j\)th component is the predicted probability of occupying state \(j\) at \(t\). It is not generally the probability of ever visiting state \(j\): a person may enter and then leave that state before \(t\).

Constant Transition Intensities

A common fitted Markov model specifies transition intensities. For a time-homogeneous continuous-time Markov chain, let \(q_{ij}(z)\) be the intensity of a transition from state \(i\) to state \(j\), for \(i\ne j\), at the fixed covariate values \(z\). The generator \(Q(z)\) has these off-diagonal entries and diagonal entries \(q_{ii}(z)=-\sum_{j\ne i}q_{ij}(z)\). Over a horizon of length \(u\), the transition matrix is \(P(u\mid z)=\exp\{uQ(z)\}\). It is this matrix—not the generator itself—that gives state-occupation probabilities.

The transition matrix follows from the forward equation. For a small interval of length \(h\), the transition matrix is \(I+hQ+o(h)\), where \(o(h)/h\) tends to zero as \(h\) tends to zero. Consequently, if \(P(u)\) is the transition matrix after elapsed time \(u\), then \(P(u+h)=P(u)\{I+hQ+o(h)\}\). Subtract \(P(u)\), divide by \(h\), and take the limit to obtain \(P'(u)=P(u)Q\), with \(P(0)=I\). The solution is \(P(u)=\exp(uQ)\). Starting from state \(r\), the landmark prediction is therefore \(e_r\exp(uQ)\).

Conditions, Diagnostics, and Alternatives

A calculated prediction is meaningful only to the extent that its model and conditioning information fit the question. Before interpreting a result, check that the landmark state and covariate values are defined at the landmark, the prediction horizon uses the intended time scale, and the fitted transition law applies to that horizon. In particular, a homogeneous generator assumes constant transition intensities over time at the specified covariate values. A time-varying transition law instead requires transition matrices appropriate to successive times and their chronological product.

Basic numerical checks can catch implementation errors:

• Each transition matrix should have nonnegative entries and rows summing to one, subject only to rounding.
• The starting vector should place all mass in the observed landmark state; it should not silently use the cohort’s baseline-state distribution.
• The propagated vector should have nonnegative entries and sum to one, within numerical tolerance.
• Absorbing states should retain their probability once entered if the model specifies no exits from them.
• Check that the prediction horizon and the fitted model’s time origin agree. A matrix for one interval cannot automatically be reused for a different interval if the model is time-inhomogeneous.

These checks establish internal consistency, not model adequacy. Compare observed transition patterns with model implications, and use the counting-process diagnostics introduced earlier, such as martingale residuals and time-rescaling diagnostics, where appropriate. Systematic discrepancies, sparse support for a transition, or evidence that future transitions depend on unmodeled past history can undermine the prediction even when all matrix calculations are correct. A Markov model can also be a poor extrapolation tool beyond the times and covariate patterns represented in the data.

The main alternative is to estimate state-occupation probabilities directly from landmark data rather than obtain them as plug-in predictions from a parametric or regression-based Markov model. The Aalen-Johansen approach is a standard nonparametric estimator for this purpose. It makes different modeling trade-offs; it does not make a misspecified Markov model correct. Conversely, the fitted Markov model can express predictions at specified covariate values and combine fitted transition behavior across intervals, but its results depend on the transition model and its assumptions. Parameter uncertainty is also distinct from the plug-in calculation shown here: the displayed values treat fitted parameters as fixed and do not, by themselves, quantify uncertainty.

Putting the Calculation Into Practice

A reliable workflow separates the conditioning statement from the arithmetic. First specify the landmark time, current state, covariate values, prediction horizon, and states whose probabilities are requested. Next identify whether the fitted model supplies a single constant transition matrix, a homogeneous generator, or matrices that vary across intervals. Then initialize at the landmark state and propagate. Finally, check the output and describe it as a conditional probability under the fitted model, with the state and time horizon named explicitly.