Transition Probabilities for Linked State Paths

Introduction

A multi-state model can describe a person moving between conditions such as active, disabled, and dead. A one-period transition probability answers where the person may be after one interval. Many actuarial questions need more: the probability of reaching a state after several intervals, the probability of passing through a particular state along the way, or the expected value of payments made at multiple times. Transition matrices provide a systematic way to calculate these quantities.

The central distinction is between an endpoint probability and a path probability. An endpoint probability includes every possible route to the destination. A path probability specifies one or more intermediate states, so only routes meeting that condition are counted.

Definition: For a discrete-time Markov model with states $1,\ldots,m$, the one-period transition matrix $P$ has entry $p_{ij}=P(X_{t+1}=j\mid X_t=i)$. Rows represent the state at the start of the interval; columns represent the state at its end. Each entry is between 0 and 1, and each row sums to 1. The Markov assumption is that, given the current state, the next-state distribution does not depend on the earlier path.

Throughout, use row probability vectors: if the initial distribution is $\boldsymbol{\pi}_0$, then after one interval it is $\boldsymbol{\pi}_1=\boldsymbol{\pi}_0P$. If the person starts in a known state $i$, the initial vector has a 1 in position $i$ and 0 elsewhere. A state such as death is often absorbing, meaning its row has probability 1 in the death column.

Multiplying Matrices Across Intervals

If the same transition matrix applies in every interval, the $n$-interval transition matrix is $P^n$. If assumptions differ by interval, let $P^{(r)}$ be the matrix for interval $r$; then the transition matrix from the start of interval 1 through interval $n$ is the ordered product $P^{(1)}P^{(2)}\cdots P^{(n)}$. The order matters: the first interval's matrix appears on the left when row vectors are used.

For a two-interval model, the probability of starting in state $i$ and ending in state $k$ sums over all possible intermediate states $j$. This is the law of total probability applied to the state at the end of the first interval:

p^{(1,2)}_{ik}=\sum_{j=1}^{m}p^{(1)}_{ij}p^{(2)}_{jk}

The product $p^{(1)}_{ij}p^{(2)}_{jk}$ is the probability of the particular path $i\to j\to k$. Summing over $j$ counts every allowed intermediate state once. In matrix notation, this calculation is exactly the $(i,k)$ entry of $P^{(1)}P^{(2)}$.

Worked Example: Two-Interval Probabilities With an Absorbing State

A hypothetical income protection model has states $A$ (active), $D$ (disabled), and $X$ (dead). For each year, the transition matrix is

P= \begin{pmatrix} 0.7&0.2&0.1\\ 0.3&0.6&0.1\\ 0&0&1 \end{pmatrix}.

Find the probability that a person starting active is disabled after two years, and find the full distribution after two years.

Method. Use the $A$ row of $P^2$. The probability of being disabled at year 2 is the sum of paths through each possible state at year 1:

P(A\to D\text{ by year 2}) =(0.7)(0.2)+(0.2)(0.6)+(0.1)(0) =0.14+0.12+0 =0.26.

The three terms represent the paths $A\to A\to D$, $A\to D\to D$, and $A\to X\to D$. The last has probability zero because death is absorbing. The other entries in the starting-active row are

\begin{aligned} P(A\to A\text{ by year 2})&=(0.7)(0.7)+(0.2)(0.3)+(0.1)(0)=0.55,\\ P(A\to X\text{ by year 2})&=(0.7)(0.1)+(0.2)(0.1)+(0.1)(1)=0.19. \end{aligned}

Thus the two-year distribution is $(0.55,0.26,0.19)$. It checks because $0.55+0.26+0.19=1.00$. The common mistake this method avoids is counting only $A\to D\to D$, which gives $0.12$ and omits the route $A\to A\to D$.

Path Probabilities and Time-Varying Matrices

When a problem specifies an intermediate state, the calculation must retain that condition. For a fully specified path $i_0\to i_1\to\cdots\to i_n$, its probability is the product of the transition probabilities on its successive edges. If some intermediate states are not specified, sum the products over the permitted paths. This is different from calculating an endpoint probability, which sums over all paths to the endpoint, whether or not they meet an additional condition.

Path probability: Under the Markov assumption, for a specified sequence of states and interval-specific matrices, $P(X_0=i_0,\ldots,X_n=i_n\mid X_0=i_0)=\prod_{r=1}^{n}p^{(r)}_{i_{r-1}i_r}$. For an event allowing multiple sequences, add the probabilities of the mutually exclusive sequences that satisfy the event.

Worked Example: Valuing Payments Across Linked States

A hypothetical two-year contract has states $E$ (eligible), $I$ (receiving care), and $D$ (dead). Its transition matrices are

Q^{(1)}= \begin{pmatrix} 0.8&0.15&0.05\\ 0.4&0.5&0.1\\ 0&0&1 \end{pmatrix}, \qquad Q^{(2)}= \begin{pmatrix} 0.7&0.2&0.1\\ 0.3&0.6&0.1\\ 0&0&1 \end{pmatrix}.

A person starts in $E$. A payment of $2,000 is made at the end of year 1 if the person is in $I$. At the end of year 2, $4,000 is paid in state $I$, or $500 in state $E$; there is no payment in state $D$. Assume an effective annual interest rate of 5%, with payments at year-end. Find the expected present value.

Method. First find the year-end state probabilities. At year 1, the probability of $I$ is $0.15$, so the expected year-1 payment is $(0.15)(\$2{,}000)=\$300$.

For year 2, multiply the first row of $Q^{(1)}$ by $Q^{(2)}$. The probability of $I$ is

$$ (0.8)(0.2)+(0.15)(0.6)+(0.05)(0)=0.16+0.09=0.25. $$

The probability of $E$ is $(0.8)(0.7)+(0.15)(0.3)+(0.05)(0)=0.56+0.045=0.605$. The remaining probability is for death: $(0.8)(0.1)+(0.15)(0.1)+(0.05)(1)=0.08+0.015+0.05=0.145$. The probabilities sum to $0.25+0.605+0.145=1$. The expected year-2 payment is $(0.25)(\$4{,}000)+(0.605)(\$500)=\$1{,}000+\$302.50=\$1{,}302.50$.

Discounting each expected payment to time 0 gives

\operatorname{EPV} =\frac{\$300}{1.05}+\frac{\$1{,}302.50}{1.05^2} =\$285.7143+\$1{,}181.4059 =\$1{,}467.1202.

Rounded to the nearest cent, the expected present value is $1,467.12. The common mistake avoided here is using the second-year matrix alone to calculate year-2 probabilities. That would ignore the year-1 distribution; the correct calculation is the first row of $Q^{(1)}Q^{(2)}$, not the first row of $Q^{(2)}$ or $Q^{(2)}Q^{(1)}$.

Separating an Intermediate-State Condition From an Endpoint

A matrix product provides endpoint probabilities, but an exam question may ask for an endpoint subject to a condition on the route. One reliable approach is to split the calculation at the required intermediate time. First calculate the probability of being in each permitted intermediate state, then apply the relevant later transition probabilities. This makes the conditioning explicit and reduces the risk of including paths that do not qualify.

Worked Example: Requiring a Particular State at the Intermediate Time

Use the active, disabled, and dead model from the first example, with the same annual matrix $P$. A person starts active. Find (a) the probability of being active after three years, and (b) the probability of being disabled at the end of year 2 and active at the end of year 3.

Method. The two-year distribution, already obtained by multiplying $P$ twice, is $(0.55,0.26,0.19)$. Apply $P$ once more to find the three-year distribution. Its active component is

$$ (0.55)(0.7)+(0.26)(0.3)+(0.19)(0) =0.385+0.078 =0.463. $$

For part (b), the required intermediate state is disabled at year 2, with probability $0.26$. From disabled, the probability of moving to active during year 3 is $0.3$. Therefore,

P(X_2=D,\ X_3=A\mid X_0=A)=(0.26)(0.3)=0.078.

The answers are $0.463$ for the unrestricted active-at-year-3 endpoint and $0.078$ for the specified year-2-disabled and year-3-active event. The latter is smaller because it includes only paths passing through $D$ at year 2. A common error is to report $0.463$ for part (b): that is the endpoint probability, not the probability subject to the intermediate-state requirement.

Using State Probabilities to Value Cash Flows

If a payment at time $t$ depends only on the state occupied at that time, its expected amount is the sum, over states, of the probability of each state multiplied by the payment in that state. If the payment depends on a transition or on a longer path, use the probability of that transition or path instead. For deterministic payments at times $t=1,\ldots,n$, discount each time-specific expected payment using the applicable interest assumption and then add the present values.

\operatorname{EPV} =\sum_{t=1}^{n}v^t\sum_{j=1}^{m}P(X_t=j)\,b_{t,j}, \qquad v=\frac{1}{1+i},

where $b_{t,j}$ is the payment at time $t$ in state $j$. This expression assumes the payment is made at the stated time and is determined by the state at that time. If payment depends on the path, replace the state probability with the probability of the specified path event. A transition payment should not be confused with a payment merely for occupying the destination state.

A useful checking sequence is to verify that every one-period matrix row sums to 1, that each computed state distribution sums to 1, and that the requested path condition is represented in the probability calculation. For time-varying matrices, confirm that the chronological order matches the row-vector convention. These checks are particularly valuable when an absorbing state is present: probability entering that state remains there, but it must not be discarded from subsequent distributions.

Key takeaway: Matrix multiplication sums over intermediate states to produce endpoint probabilities. Retain a specified intermediate condition by multiplying only the qualifying path probabilities, and value cash flows using the probability measure that matches the payment trigger.

Check Your Understanding

For each problem, state the transition calculation used and check that any relevant probabilities are consistent.

A two-state model has states $U,V$ and transition matrix $\begin{pmatrix}0.6&0.4\\0.25&0.75\end{pmatrix}$. Starting in $U$, find the probability of being in $V$ after two intervals. (Answer: 0.5400)
A model has states $A,B,D$, with $D$ absorbing. In the first interval, the row for $A$ is $(0.7,0.2,0.1)$. In the second interval, the rows for $A$ and $B$ are $(0.8,0.1,0.1)$ and $(0.2,0.7,0.1)$, respectively. Starting in $A$, find the probability of being in $B$ after two intervals. (Answer: 0.2100)
A model has states $A,I,D$, with $D$ absorbing, and annual transition rows $A=(0.75,0.20,0.05)$ and $I=(0.40,0.50,0.10)$. Starting in $A$, find the probability of being in $A$ at time 2 and in $I$ at time 1. (Answer: 0.0800)
A hypothetical contract has states $A,B,D$, with $D$ absorbing. Its first-interval rows for $A,B$ are $(0.7,0.3,0)$ and $(0.2,0.8,0)$. Its second-interval rows are $A=(0.6,0.3,0.1)$, $B=(0.2,0.7,0.1)$. Starting in $A$, it pays $1,000 at the end of year 1 in $B$, and $2,000 at the end of year 2 in $B$. At an effective annual interest rate of 4%, calculate the expected present value, rounded to the nearest cent. (Answer: $1,065.09)

What You'll Learn