The Markov Assumption for State Models

Why the Markov Assumption Matters

A state model records an individual’s condition as time passes. In an actuarial application, the state might represent whether a person is active, disabled, receiving care, or deceased. Knowing the current state is useful only if the model also specifies how that state relates to future behavior. The Markov assumption makes a particular simplifying claim: once the current state and current time are known, the earlier path does not provide additional information about future states.

This assumption is about conditional behavior, not about saying that the past never matters in reality. A person’s past may affect future outcomes. The modelling question is whether that information is already summarized by the current state and time. If not, the state definition may need to be refined.

Definition: Let $Z(t)$ denote the state occupied at time $t$, and let the history through time $t$ consist of the states occupied at all times up to and including $t$. The process is Markov if, for every future time $t+u$ and state $j$, the conditional probability of being in state $j$ at $t+u$, given the full history through $t$, depends only on $Z(t)$ and the time $t$. In symbols, when $Z(t)=i$, $$ P\bigl(Z(t+u)=j\mid \text{history through }t\bigr) = P\bigl(Z(t+u)=j\mid Z(t)=i\bigr). $$

The equality is required for future behavior, not just for one chosen destination or one particular interval. It can be applied to the probabilities of future states at several specified times. The right-hand side may depend on the calendar time $t$ as well as on the interval $u$. A Markov model therefore need not have the same transition probabilities at every calendar time.

What the Assumption Does—and Does Not—Say

The Markov assumption says that two individuals in the same state at the same time have the same modelled future state distribution, even if they reached that state by different routes. Their histories may differ; those differences are simply not used to determine their future probabilities once the current state and time are specified.

It does not say that future states are independent of the current state. In fact, the current state is the information that determines which conditional distribution to use. Nor does Markov mean that all future times are independent of one another: a future state can affect the distribution of states after that. The assumption concerns the role of the earlier history after conditioning on the present.

Question	Markov answer
Can future behavior depend on the current state?	Yes. The current state is part of the conditioning information.
Can future behavior depend on calendar time?	Yes. The transition law may vary with time.
Can two people in the same state have different histories?	Yes. Under the assumption, those histories do not further change the future state distribution.
Can time already spent in a state matter?	Only if it is included in the state or otherwise represented in the model.

A model with a transition law that does not vary with calendar time is called time-homogeneous. That is an additional assumption, not part of the Markov property itself. A process can be Markov while its transition probabilities change with calendar time. For example, a hypothetical care model might allow the probability of entering a care state to differ by calendar year while still assuming that, at a given time, the current state is enough to determine future behavior.

Deriving the Multi-Step Calculation

The Markov property lets us calculate a future state probability by considering possible intermediate states. This is useful when a direct multi-year probability is not supplied. For clarity, let $p_{ij}(s,t)$ denote the probability of being in state $j$ at time $t$, given state $i$ at time $s$. If the process is Markov, then conditioning on the state at an intermediate time $r$ gives

p_{ij}(s,t) = \sum_k p_{ik}(s,r)\,p_{kj}(r,t), \qquad s<r<t.

To justify the formula, partition the event of being in state $j$ at time $t$ according to the state $k$ occupied at time $r$. These intermediate-state events are mutually exclusive and cover all possibilities. The probability of the route through $k$ is the probability of reaching $k$ at $r$, multiplied by the conditional probability of reaching $j$ at $t$ given that state at $r$. The Markov property makes that second factor depend on state $k$ and time $r$, not on the earlier route to $k$. Summing over $k$ gives the stated result.

This calculation uses transition probabilities over finite intervals. It does not require assuming that a person can make at most one transition during an interval: the probabilities for that interval already describe the model’s state at its endpoint. The separate question of describing transitions over very short intervals is not needed here.

Worked Example: A Two-Year Disability Probability

Consider a hypothetical model with two states: $A$, active, and $D$, disabled. Assume the process is Markov and time-homogeneous over the two years considered. The one-year transition probabilities are:

P(A\text{ after one year}\mid A\text{ now})=0.8,\quad P(D\text{ after one year}\mid A\text{ now})=0.2, $$ $$ P(A\text{ after one year}\mid D\text{ now})=0.4,\quad P(D\text{ after one year}\mid D\text{ now})=0.6.

A person is disabled now. Find the probability that the person is disabled two years from now. The method is to split the two-year outcome according to the state after one year.

Route through $A$. The person must first move from $D$ to $A$, with probability $0.4$, and then from $A$ to $D$, with probability $0.2$. The route probability is $0.4(0.2)=0.08$.

Route through $D$. The person must remain in $D$ after the first year, with probability $0.6$, and then be in $D$ after the second year, with probability $0.6$. The route probability is $0.6(0.6)=0.36$.

These two routes cover all possible states after one year. Therefore the required probability is $0.08+0.36=0.44$. As a check, the two route probabilities are nonnegative and sum to a value no greater than one. The calculation uses the same conditional probabilities for either history leading to the same state at the intermediate time; it does not add a separate adjustment for earlier history.

Common mistake avoided: Using only the probability of remaining disabled for two successive years gives $0.6^2=0.36$, which omits the route $D\to A\to D$. The desired event is being disabled at the end, not remaining disabled throughout.

When Time in a State Is Relevant

A common test of the Markov assumption is whether two histories that end in the same state at the same time imply the same future state distribution. Suppose, for example, that a disabled person’s chance of recovery depends on how long that person has been disabled. If duration is omitted from the state, the future recovery probability may still depend on the path by which the person reached the current state.

One modelling response is to refine the state description so that the relevant duration or history is represented. Another is to choose a model that allows future behavior to depend explicitly on time spent in the current state. The important point here is diagnostic: if two individuals in the same stated condition and at the same calendar time have different future conditional probabilities because of their histories, a model using only that condition as its state does not satisfy the Markov assumption.

Worked Example: Detecting Duration Dependence

A hypothetical disability model records only two states, active and disabled. For a disabled person assessed on the same calendar date, the model’s assumed probability of recovery during the next six months is $0.70$ if the person has been disabled for two months, and $0.25$ if the person has been disabled for 18 months. Decide whether the model is Markov when its only disability state is “disabled.” Also find the overall recovery probability if 80% of the disabled people being considered have been disabled for two months and 20% for 18 months.

Markov assessment. Both histories end in the same recorded state, disabled, at the same assessment time. But their conditional probabilities of recovery over the next six months differ: $0.70\ne 0.25$. Thus the future probability is not determined by the recorded current state and time alone. The model, with only this state definition, is not Markov.

Overall probability for this group. Use the two history groups as a partition: $$ 0.80(0.70)+0.20(0.25)=0.56+0.05=0.61. $$ The overall recovery probability is $0.61$. It is a mixture for the stated group, not a single probability that applies to both histories.

One possible refinement is to distinguish recent disability from long-duration disability, assigning each a separate state. The specified histories would then have different current states, so their different probabilities would no longer contradict the Markov assumption for the refined model. Whether that refinement is suitable depends on the application and the information available.

Common mistake avoided: Calling the model Markov because the overall probability $0.61$ can be calculated. Being able to calculate a group average does not make the conditional probabilities equal for each history.

Starting Distributions and the Markov Property

The starting distribution describes the probabilities of occupying each state at the chosen initial time. It does not, by itself, determine the future state probabilities: the model must also supply transition probabilities. Under the Markov assumption, probabilities at a later time can be calculated by combining the probability of each current state with the appropriate transition probabilities. If two different histories contribute to the same current state, the same conditional transition probabilities apply to both histories.

Worked Example: Combining a Starting Distribution and Two Routes

A hypothetical long-term benefit model has states $W$ (working), $D$ (disabled), and $X$ (deceased). At time 0, the probabilities are $P(W)=0.60$, $P(D)=0.40$, and $P(X)=0$. Assume a time-homogeneous Markov model with the following one-year probabilities: from $W$, the probabilities of $W,D,X$ are $0.80,0.15,0.05$; from $D$, they are $0.25,0.65,0.10$; and $X$ is absorbing. Find the probability of being deceased at the end of two years.

First-year state probabilities. The probability of $W$ at year 1 is $0.60(0.80)+0.40(0.25)=0.48+0.10=0.58$. The probability of $D$ at year 1 is $0.60(0.15)+0.40(0.65)=0.09+0.26=0.35$. The probability of $X$ at year 1 is $0.60(0.05)+0.40(0.10)=0.03+0.04=0.07$. These probabilities sum to $0.58+0.35+0.07=1.00$, as required.

Second-year death probability. From $W$, the probability of death during the next year is $0.05$; from $D$, it is $0.10$; and anyone already in $X$ remains there. Therefore $$ P(X\text{ at year 2}) =0.58(0.05)+0.35(0.10)+0.07(1) =0.029+0.035+0.070 =0.134. $$ The answer is $0.134$, or 13.4%.

The disabled population at year 1 includes two routes: $W\to D$, with probability $0.09$, and $D\to D$, with probability $0.26$. Both use the same next-year death probability of $0.10$, so their combined contribution to death in year 2 is $0.09(0.10)+0.26(0.10)=0.009+0.026=0.035$, matching $0.35(0.10)$. This is a direct check of how the Markov assumption allows histories to be grouped by their current state.

Common mistake avoided: Applying the transition probability from $W$ to the entire year-1 population, or forgetting that the $0.07$ already deceased remain in $X$. The calculation must condition on each year-1 state.

Exam Checks and Common Traps

When assessing a proposed state model, identify the conditioning information precisely. Compare histories at the same calendar time and in the same recorded state. If the future probabilities still differ because of duration, a prior transition, or another omitted feature, the current state description does not support the Markov assumption. If the model is Markov, do not add path-specific factors after conditioning on the current state.

Confusing Markov with time-homogeneous. A Markov model can have probabilities that change with calendar time.
Ignoring the state definition. A model may fail to be Markov with a broad state label and become Markov after that state is refined.
Using one route only. A multi-interval probability may include several intermediate states; all mutually exclusive routes must be included.
Treating a mixture as a conditional probability. An average across people with different histories does not show that each history has the same future probability.

Key takeaway: The Markov assumption says that, given the current state and time, the past path adds no further information about future state behavior. It is a modelling assumption to assess against the chosen state definitions, not a claim that history is irrelevant in every real-world application.

Check Your Understanding

For each problem, use the stated model assumptions and show the intermediate calculations.

In a hypothetical two-state model, a person currently in state $R$ has probability $0.30$ of being in $R$ six months later. Under the Markov assumption, compare this probability for two people assessed at the same time in state $R$, one of whom reached $R$ from state $S$ and the other of whom has remained in $R$. (Answer: $0.30$ for each person, if the stated current state and time are the complete conditioning information.)
A time-homogeneous Markov model has states $A$ and $D$. Over one year, $P(A\text{ next}\mid A)=0.70$, $P(D\text{ next}\mid A)=0.30$, $P(A\text{ next}\mid D)=0.40$, and $P(D\text{ next}\mid D)=0.60$. Starting in $D$, calculate the probability of being in $D$ after two years by separating the intermediate states. (Answer: $0.48$.)
At the initial time, a hypothetical model has probability $0.50$ of state $A$ and $0.50$ of state $D$. In one year, the probability of moving from $A$ to $D$ is $0.20$, and the probability of moving from $D$ to $D$ is $0.50$. Find the probability of being in $D$ after one year. (Answer: $0.35$.)
A model records only “disabled.” At the same assessment time, people disabled for two months have next-six-month recovery probability $0.50$, while people disabled for 18 months have probability $0.20$. If 60% of the group has the shorter duration, calculate the group recovery probability and state whether the one-state disability model is Markov. (Answer: Group probability $0.38$; the model is not Markov with duration omitted.)
A hypothetical time-homogeneous Markov model has states $W,D,X$, with $X$ absorbing. The initial probabilities are $P(W)=0.70$, $P(D)=0.30$, and $P(X)=0$. The one-year probabilities from $W$ to $W,D,X$ are $0.80,0.15,0.05$, and from $D$ they are $0.20,0.70,0.10$. Find the probability of being in $X$ after two years. (Answer: $0.1275$.)

What You'll Learn