Time-Homogeneous and Time-Inhomogeneous Markov Models

Why Transition Behavior May Change Over Time

A Markov model describes how a process moves among states, such as being well, ill, or dead. The Markov property says that, conditional on the present state, the future does not additionally depend on the earlier state history. A separate question is whether the transition behavior from that present state stays the same over time. A person who is well at the start of follow-up might have a different probability of entering an illness state in the next month than a person who is well much later in follow-up.

The distinction matters because a single transition matrix summarizes behavior across time only when the model’s homogeneity assumption is appropriate. If transition behavior changes, calculations based on one constant matrix can misstate future state probabilities, even when the process is Markov. Thus, homogeneity is not another name for the Markov property: it is an additional restriction on how transition behavior may vary.

In discrete time with equally spaced observations, homogeneity means the same one-step transition matrix applies at each step. In continuous time, a homogeneous process has the same transition behavior over intervals of the same duration, regardless of their location on the time axis. Homogeneity does not mean that transition probabilities are the same for intervals of different lengths: the chance of a transition over two years will generally differ from the chance over one year.

A homogeneous model can still have a state distribution that changes over time. For example, if more people accumulate in an absorbing state such as death, the proportion in that state increases even though the transition rules remain constant. Homogeneity concerns the conditional transitions given the current state, not whether the overall population composition is stable.

Transition Matrices and Composition Across Intervals

For a discrete-time model with a finite set of states, the transition matrix \(P_k\) for step \(k\) has entry \(p_{ij,k}=P\{X_k=j\mid X_{k-1}=i\}\). Each entry is a probability, so the entries are nonnegative and each row sums to one. Under homogeneity, \(P_k=P\) at every step. Under inhomogeneity, the matrix may change from step to step.

Use the row-vector convention: a row vector gives the probabilities of occupying each state, in the same order as the rows and columns of the transition matrices. If the distribution at the start is \(\boldsymbol{\pi}_0\), then after two intervals with matrices \(P_1\) and \(P_2\), it is \(\boldsymbol{\pi}_0P_1P_2\). The order records the actual sequence of intervals: first apply \(P_1\), then \(P_2\).

The composition rule follows by conditioning on the state at the boundary between intervals. For two intervals, the probability of starting in state \(i\) and ending in state \(j\) is the sum over every possible intermediate state \(k\) of the probability of reaching \(k\) and then moving from \(k\) to \(j\). In matrix notation, this is exactly the \((i,j)\) entry of \(P_1P_2\).

When \(P_1=P_2=P\), the product reduces to \(P^2\). For more intervals, repeated conditioning gives the ordered product \(P_1P_2\cdots P_m\). This argument also explains why reversing the order of two different matrices generally gives the wrong answer: \(P_1P_2\) and \(P_2P_1\) need not be equal.

Time-Inhomogeneous Models in Practice

Transition behavior can change with time for substantive reasons: treatment may become more effective after an initial period, disease risk may rise with duration, or a policy change may alter hospitalization patterns. A time-inhomogeneous Markov model allows the transition probabilities to vary while retaining the Markov condition that the present state summarizes the relevant past for predicting the future.

In continuous time, this distinction is often expressed through transition intensities. For distinct states \(i\) and \(j\), an intensity \(q_{ij}(t)\) describes the instantaneous rate of moving from \(i\) to \(j\) at time \(t\), conditional on being in \(i\) just before \(t\). In a homogeneous continuous-time model, these rates are constant over time; in an inhomogeneous model, they may vary with time. A time-varying rate does not, by itself, violate the Markov property.

Assessing the Homogeneity Assumption

Homogeneity is a modeling assumption, not something established merely by observing that the process is Markov. A practical check is to compare transition patterns across relevant time periods: for example, compare the observed transitions out of a state during early and later follow-up, while accounting for how many people were in that state and at risk of each transition. Large, systematic differences can indicate time variation. Small differences may simply reflect sampling variability, especially when few transitions are observed.

Before attributing differences to time, check that the state definitions and observation intervals are comparable. Changes in who is observed, how states are measured, treatment practice, or censoring can also produce different observed transition patterns. Sparse data can make a time-stratified comparison unstable. These checks are diagnostic rather than a guarantee that a constant-transition model is correct.

A common mistake is to conclude that a model must be homogeneous because the process satisfies the Markov property. These are separate claims: the Markov property concerns dependence on earlier state history after conditioning on the present state; homogeneity concerns whether transition behavior changes with time. Another mistake is to infer inhomogeneity just because the proportion in each state changes. A changing state distribution can arise under a homogeneous transition matrix, as the two-quarter calculation above illustrates.

If transition behavior clearly varies, one alternative is to use interval-specific transition matrices or time-varying transition rates. A homogeneous model is simpler and can summarize behavior compactly when its restriction is reasonable. More flexible time-specific models can represent changing behavior, but may require more data and produce less precise estimates. Neither approach removes the need to define states and to justify the Markov assumption. Also, a time-varying rate is not automatically evidence that time alone explains the change; relevant predictors or changes in the study process may need consideration.