Choosing Between Semi-Markov And History-Dependent Models

What Information Should a Transition Model Retain?

When a person enters a state, transition behavior may change as time passes in that state. It may also depend on what happened before the current visit: the route by which the person arrived, the number of earlier visits, or accumulated time in another state. The modeling question is not simply whether the process has “memory.” It is which parts of the observed path are needed to predict the next transition, after accounting for the current state and time already spent there?

A semi-Markov model represents duration dependence: the transition behavior can vary with time in the current state, but earlier path information adds nothing once the current state and its duration are known. A history-dependent model allows earlier path information to add predictive information. These are different assumptions about what information is sufficient, not just different formula choices.

Definition: Let $X(t)$ be the state occupied at time $t$, and let $H_{t-}$ denote the observed path immediately before $t$, including earlier states and transition times. Let $L(t)$ be the time of entry into the current state and $A(t)=t-L(t)$ its duration. For distinct states $i$ and $j$, the transition intensity is $$ \lambda_{ij}(t\mid H_{t-})= \lim_{\Delta t\downarrow 0} \frac{P\{X(t+\Delta t)=j\mid X(t)=i,H_{t-}\}}{\Delta t}, $$ when the limit exists. A duration-based semi-Markov restriction is $\lambda_{ij}(t\mid H_{t-})=\alpha_{ij}\{A(t)\}$: conditional on the current state and duration, earlier path details do not further change the intensity. A history-dependent model instead allows the intensity to depend on relevant earlier path information, such as a summary $Z(H_{t-})$.

The restriction is about the transition law, not a claim that all histories are identical. Two people can have different earlier paths but the same modeled transition behavior if those path differences do not predict the next transition after conditioning on the current state and duration. Conversely, a history-dependent model need not use the entire path: it can use a defined summary, provided that summary captures the relevant predictive information.

This comparison assumes that the state definitions and transition times are recorded adequately, and that the candidate history variables are available at the time prediction is made. A duration-only model also assumes that, within the modeled conditions, the intensity does not depend on earlier history. If calendar time or measured covariates matter, they must be represented separately rather than silently attributed to duration or history. The earlier material on duration dependence and checking whether history adds predictive information provides useful diagnostics; here the goal is to choose between the two memory structures.

Three Patterns and the Model They Suggest

Observed pattern	Model structure suggested	Key comparison
Exit behavior changes with time in the current state, but not with earlier path once duration is controlled.	Semi-Markov	Do transition patterns differ by duration, and do histories within a duration group add prediction?
At the same current state and duration, exit behavior differs by the immediately preceding state or event order.	History-dependent	Does path order predict transitions beyond current duration?
At the same current state and duration, exit behavior differs by cumulative exposure or number of prior visits.	History-dependent, using a prespecified summary	Does accumulated path information add prediction beyond the current spell?

Worked Comparisons

Worked Example: Duration Changes Risk, but Earlier Path Does Not

Suppose a hypothetical process has a current state B and two possible exits, to C or D. In the first week of a B visit, the cause-specific intensities are constant at $\lambda_{BC}=0.020$ and $\lambda_{BD}=0.010$ per week. After at least four weeks in B, they are $0.040$ and $0.010$ per week. Assume the rates are the same across earlier paths within each duration group, and that there are no other exits from B.

The pattern supports duration dependence: the rate to C is higher later in the current visit, while the preceding path does not distinguish people with the same current duration. To translate these competing rates into a one-week probability of a C transition, use the total exit rate, not the C rate alone. With constant competing rates $\lambda_k$, the probability of exit to cause $k$ by time $u$ is

P_k(u)=\frac{\lambda_k}{\sum_r\lambda_r} \left(1-\exp\left\{-u\sum_r\lambda_r\right\}\right).

For a first-week B visit, the total rate is $0.020+0.010=0.030$ per week. Thus the one-week probability of B-to-C is

\frac{0.020}{0.030}\left(1-e^{-0.030(1)}\right) =\frac{2}{3}(0.02955)=0.01970.

For a visit lasting at least four weeks, the total rate is $0.040+0.010=0.050$ per week, giving

\frac{0.040}{0.050}\left(1-e^{-0.050(1)}\right) =0.8(0.04877)=0.03902.

Under these assumptions, the modeled one-week probability of transition to C is about 1.97% in the early-duration group and 3.90% in the later-duration group. This comparison accounts for the competing B-to-D exit. It supports a semi-Markov description if earlier path information does not further distinguish the rates at the same duration; it does not establish that duration itself causes the difference.

Worked Example: Event Order Adds Predictive Information

Consider a hypothetical state C with two possible exits, to D and E. Among people currently in C for one week, suppose the estimated constant rates are $0.030$ per week to D and $0.020$ per week to E when the path was A then B then C. For the same current state and duration, suppose the rates are $0.010$ to D and $0.020$ to E when the path was B then A then C. Assume these are the only exits and rates are constant over the one-week prediction horizon.

The current state and duration are matched, but the rate to D differs by path order. For the A-then-B group, the total exit rate is $0.050$, so the one-week probability of C-to-D is

\frac{0.030}{0.050}(1-e^{-0.050}) =0.6(0.04877)=0.02926.

For the B-then-A group, the total is $0.030$, giving

\frac{0.010}{0.030}(1-e^{-0.030}) =\frac{1}{3}(0.02955)=0.009852.

The resulting one-week probabilities are about 2.93% and 0.985%, respectively. A duration-only model would assign the same prediction to both groups at the same duration, so it cannot represent this pattern. A history-dependent model that includes event order is a candidate, provided the difference is supported by adequate data and is not explained by another relevant factor, such as calendar time or a measured covariate.

Worked Example: Cumulative Exposure Matters Beyond the Current Visit

Suppose people enter state R after different prior paths. Among people in R for one week, consider two groups: those with low prior exposure to state A and those with high prior exposure. In a hypothetical fitted model, the constant rates from R are $0.010$ per week to S and $0.020$ per week to T for the low-exposure group, and $0.040$ to S and $0.020$ to T for the high-exposure group. Assume these are the only exits, and rates are constant over the next week.

For low prior exposure, the probability of R-to-S is

\frac{0.010}{0.010+0.020} \left(1-e^{-(0.010+0.020)}\right) =\frac{1}{3}(0.02955)=0.009852.

For high prior exposure, it is

\frac{0.040}{0.040+0.020} \left(1-e^{-(0.040+0.020)}\right) =\frac{2}{3}(0.05824)=0.03882.

The modeled one-week probabilities of R-to-S are about 0.985% and 3.88%. Since current state and current duration are held fixed, the modeled difference is associated with prior exposure, not with a longer R visit. A history-dependent model could include cumulative exposure as a predictor. The exposure definition and time window should be specified before analysis; trying many summaries and retaining only the strongest apparent association risks overfitting.

Why the Competing-Risk Calculation Works

For a current state with possible exits $r$, write the total exit intensity as $\lambda_{\cdot}(u)=\sum_r\lambda_r(u)$, where $u$ is time since the start of the prediction interval. If the rates are constant, the probability of remaining in the current state through time $u$ is $S(u)=\exp(-u\sum_r\lambda_r)$. The probability of an exit to cause $k$ in a short interval near $u$ is approximately $S(u)\lambda_k\,du$. Integrating over the horizon gives

P_k(t)=\int_0^t S(u)\lambda_k\,du =\int_0^t e^{-u\sum_r\lambda_r}\lambda_k\,du =\frac{\lambda_k}{\sum_r\lambda_r} \left(1-e^{-t\sum_r\lambda_r}\right).

The derivation makes the assumption explicit: every possible exit must be included in the total rate. If rates vary over the horizon, the corresponding calculation uses the time-varying total intensity inside the survival term and integrates $S(u)\lambda_k(u)$. A cause-specific rate by itself is not generally a transition probability when other exits compete.

How to Choose and Check a Model

Define the prediction time and state.
Specify the current state, the moment at which prediction begins, the duration clock, and the possible next transitions. A duration comparison is meaningful only when duration is measured consistently.

Ask whether duration accounts for the pattern.
Compare transition behavior over current-state duration while controlling the comparison for relevant measured factors and calendar time. If behavior varies with duration, a model that allows duration dependence is needed; a memoryless model with constant rates may be inadequate.

Test whether history adds useful prediction.
Within comparable current-state and duration groups, assess whether prespecified features such as preceding state, event order, prior visits, or cumulative exposure improve prediction of the next transition.

Retain only defensible memory.
Use the simplest structure that captures the supported predictive pattern. Check predictions on held-out data or by appropriate resampling when possible, and inspect whether estimates are stable across reasonable specifications.

Diagnostics should be tied to the proposed restriction. If a semi-Markov model is used, compare observed and predicted transition patterns across relevant histories at similar current-state durations. Systematic residual differences by path order or cumulative exposure suggest that the duration-only restriction is insufficient. Conversely, if history terms are unstable, poorly supported, or do not improve out-of-sample prediction, a more complex model may be unnecessary. Sparse histories can make apparent differences noisy, while unrecorded path features can leave residual dependence even after including a chosen summary.

These checks require adequate observation of transitions and their timing. Informative censoring, interval-censored transition times, or histories that are recorded differently across groups can distort comparisons unless handled appropriately. The model should also respect the actual set of possible transitions. In particular, a cause-specific intensity cannot be turned into a transition probability by the expression $1-e^{-\lambda t}$ unless that transition is the sole possible exit over the interval.

Trade-Offs and Common Pitfalls

Approach	Strength	Limitation
Semi-Markov, duration-based	Represents changing transition behavior over a state visit without retaining the entire earlier path.	Can miss predictive differences between people with the same current state and duration but different histories.
History-dependent	Can represent effects of route, event order, repeated visits, or cumulative exposure.	Requires enough data for the histories or summaries included; predictions can be unstable or difficult to interpret.
State augmentation	Can encode selected memory in an expanded state description and use a Markov representation when appropriate.	May create many sparse expanded states, and the expanded state must reflect the relevant memory accurately.

A common mistake is to conclude that a history-dependent model is required whenever the process is not Markov in the original state labels. That reasoning skips the distinction between duration and earlier path. If the apparent memory is fully captured by time in the current state, a semi-Markov model may suffice. The correction is to compare histories at matched current-state durations: only additional predictive information beyond duration motivates a history-dependent structure.

The reverse mistake is to treat a semi-Markov model as automatically adequate once duration is included. It remains a substantive restriction that earlier path details do not matter conditional on duration. A good fit to overall transition counts does not establish that restriction; histories can have different conditional predictions while producing similar marginal counts.

Key takeaway: Choose a semi-Markov model when current state and duration capture the relevant transition information. Choose a history-dependent model when earlier path information predicts transitions beyond those quantities. State the conditioning information clearly, account for all competing exits when computing probabilities, and prefer the least complex model that predicts adequately.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

At the same current-state duration, two groups with different preceding states have different estimated transition intensities. Which model structure can represent this difference?
A state has constant exit rates of 0.02 per week to A and 0.03 per week to B. What is the one-week probability of transition to A?
Transition intensity rises over a state visit, but no earlier-path difference remains after duration is accounted for. Which structure is suggested?
Why is $1-e^{-\lambda t}$ not generally the probability of a particular transition when other exits are possible?
Name one diagnostic for assessing whether history adds prediction beyond current-state duration.

Solutions. (1) A history-dependent model can include preceding state as a predictor, since the difference remains at matched duration. (2) The total rate is $0.02+0.03=0.05$, so the probability is $(0.02/0.05)(1-e^{-0.05})=0.01951$, or about 1.95%. (3) A semi-Markov structure is suggested because behavior varies with current-state duration, while earlier history adds no prediction. (4) The expression treats $\lambda$ as the sole exit rate; competing exits reduce the probability of reaching the specified destination, so all exit rates must enter the cumulative-incidence calculation. (5) Compare observed and predicted transition patterns across prespecified path histories among people at similar current-state durations, or evaluate whether history terms improve out-of-sample prediction.