Comparing Markov And Semi-Markov Predictions

Why Time in the Current State Matters

A prediction for someone currently in state A may depend on more than the state label. If transition hazards change with time spent in A, two people who are both currently in A can have different probabilities of moving to another state over the next month because one entered A recently and the other has been there much longer. A Markov model that represents the current state as the full predictive state does not make that distinction; a semi-Markov model can.

The comparison here concerns predictions conditional on the same current state and different elapsed durations in that state. It does not by itself establish which model is preferable: that requires considering the scientific process, model assumptions, and predictive performance. The earlier material on duration dependence, semi-Markov models, and memory structures supplies useful context for this comparison.

For a Markov model, the probability of being in each possible state after horizon \(h\) is obtained from the transition matrix \(P(h)=\exp(Qh)\). In particular, the prediction depends on the current state and horizon, not on the time already spent in that state. A time-inhomogeneous Markov model may allow rates to vary with calendar time, but, conditional on the current state and calendar time, it still does not depend on elapsed state duration.

In a semi-Markov model, elapsed duration changes the residual time-to-exit distribution when hazards vary with duration. The destination can also matter: separate \(\alpha_{ij}(u)\) specify duration-specific hazards for different possible transitions. Unless stated otherwise, the calculations below assume a correctly specified state structure, known hazards, no additional unmodeled history needed for prediction, and duration measured from the most recent entry into the current state.

From Hazards to Future Probabilities

Suppose someone is in state \(i\), has already spent \(u\) time units there, and is predicted over the next \(h\) time units. The total exit hazard at duration \(v\) is \(\lambda_i(v)=\sum_{j\ne i}\alpha_{ij}(v)\). Conditional on remaining in \(i\) at duration \(u\), the probability of no exit during the next \(h\) time units is

This is a residual survival probability: it conditions on having already remained in the state through duration \(u\). The probability density of first exiting to \(j\) at \(v\) time units from now is \(S_i(v\mid u)\alpha_{ij}(u+v)\). Thus the probability that the first exit by horizon \(h\) is to \(j\) is

That first-exit probability is not always the probability of occupying \(j\) at the horizon: the person might enter \(j\) and then leave it before the horizon. For paths involving additional transitions, let \(p_{ij}(h\mid i,u)\) denote the probability of occupying \(j\) after \(h\), starting in \(i\) at duration \(u\). If durations reset on entry and the future depends only on the current state and its duration, the semi-Markov renewal equation is

The first term accounts for no exit from \(i\); each integral accounts for a first transition to \(k\), followed by the prediction from \(k\) with its duration reset to zero. This equation also shows why a simple first-exit calculation is sufficient only when the destination states are absorbing or when the target is specifically the first transition.

Worked Predictions

Justification, Diagnostics, and Model Choice

The residual-survival formula follows from the definition of a hazard. Over a short interval of length \(dv\) at duration \(v\), conditional on still being in A, the probability of exit is approximately \(\lambda_A(v)dv\). Thus survival over that interval is approximately \(1-\lambda_A(v)dv\). Multiplying these conditional survival factors over small intervals from \(u\) to \(u+h\), taking logarithms, and letting the maximum interval width approach zero gives

This derivation requires a well-defined hazard and an appropriate duration clock. In the Markov special case, \(\lambda_i(v)\) and each destination-specific hazard are constant within the state. Then the integral depends on the prediction horizon but not on \(u\), giving the same residual predictions for all elapsed durations.

A useful comparison is to compute predictions at clinically meaningful durations and horizons under each specification, as in the examples. Differences are expected when hazards vary with duration, but differences alone do not show that the semi-Markov model is better. Check whether duration is measured from the right entry time, whether repeated visits reset or retain the clock according to the intended model, and whether calendar-time effects or omitted history could explain apparent duration patterns. Compare predictions with suitable observed outcomes and assess calibration at the horizons and states relevant to the application. Sparse observations at long durations can make those predictions especially uncertain.

A common mistake is to use the cumulative hazard from entry into A when predicting someone who has already survived there for \(u\) months. That computes an unconditional probability from entry, not a probability conditional on still being in A now. The required residual cumulative hazard is \(\int_u^{u+h}\lambda_A(v)\,dv\), not \(\int_0^{u+h}\lambda_A(v)\,dv\). Another error is to treat a first-exit probability as final occupancy when the destination can be left again. Use the renewal equation or an equivalent path calculation when further transitions are possible.

Neither model automatically handles unmeasured history, changing covariates, or misspecified state definitions. A semi-Markov prediction is only as useful as its duration clock and hazard structure; a Markov prediction can still be appropriate if duration adds little predictive information or if the simpler model is better supported. These are predictive comparisons, not causal effects.