Why an Intermediate State Matters
A transition probability over a long interval can reflect many different routes through a state model. For example, a person could be active at the beginning of the interval, disabled at an intermediate time, and active again at the end. The probability of the final state must account for every possible intermediate state, not just the route that seems most direct.
The Chapman-Kolmogorov relationship formalizes this calculation. It is useful whenever a transition interval is split into consecutive parts, including when the probability model is specified separately for each part. The relationship requires the Markov property, but it does not require transition probabilities to be constant over time.
The sum is over the possible states \(k\) at the intermediate time \(t\). Each term is the probability of being in state \(k\) at time \(t\), starting from \(i\) at time \(s\), multiplied by the probability of reaching \(j\) at time \(u\) from \(k\) at time \(t\). When \(E\) is countably infinite, the sum is an infinite series of nonnegative terms.
The finite-or-countable qualification matters. If the state space is uncountable, the general relationship uses a transition kernel and an integral, not a sum over individual states. For a measurable set \(B\) of possible states at time \(u\), it is $$ P(s,u)(i,B)=\int_E P(t,u)(k,B)\,P(s,t)(i,dk). $$ The calculations in this tutorial use finite state spaces, so the ordinary sum and matrix product apply.
Why the Formula Holds
The intermediate state at time \(t\) must be exactly one state in \(E\). The events “the process is in state \(k\) at time \(t\)” are mutually exclusive and, over all \(k\in E\), exhaustive. Thus a transition from \(i\) to \(j\) over the whole interval can be counted by adding the contributions associated with each intermediate state. The Markov property means that, once the state at time \(t\) is specified, the transition probability from \(t\) to \(u\) is \(P(t,u)_{kj}\). Multiplying the two interval probabilities gives the contribution through \(k\); adding those contributions gives the displayed formula.
This reasoning also shows why the time arguments should be retained. In general, the transition matrix for the first interval need not equal the matrix for the second: the rates or conditions governing transitions may differ across calendar time. The relationship composes the matrices for the actual intervals; it does not assume that they are identical.
Write the start, intermediate, and end times in order: \(s\leq t\leq u\).
For an entry \(P(s,u)_{ij}\), keep \(i\) and \(j\) fixed.
For each \(k\), multiply \(P(s,t)_{ik}\) by \(P(t,u)_{kj}\).
Sum over \(k\). If calculating a full matrix, check that all entries are nonnegative and each row sums to 1.
Worked Applications
Worked Example: Combining Two Three-State Transition Matrices
A hypothetical model has states \(A,B,C\). Its transition probabilities from time 0 to time 1 and from time 1 to time 2 are respectively $$ P(0,1)= \begin{pmatrix} 0.70&0.20&0.10\\ 0.10&0.60&0.30\\ 0.20&0.20&0.60 \end{pmatrix}, \qquad P(1,2)= \begin{pmatrix} 0.50&0.30&0.20\\ 0.20&0.50&0.30\\ 0.10&0.40&0.50 \end{pmatrix}. $$ Find the probability of starting in \(A\) at time 0 and being in \(C\) at time 2. The hypothetical model is specified directly by transition probabilities; no interest rate or separate mortality assumption is needed.
Method. Apply Chapman-Kolmogorov and sum over the three possible states at time 1: $$ P(0,2)_{AC} =P(0,1)_{AA}P(1,2)_{AC} +P(0,1)_{AB}P(1,2)_{BC} +P(0,1)_{AC}P(1,2)_{CC}. $$
The contribution through intermediate state \(A\) is \(0.70(0.20)=0.14\). Through \(B\), it is \(0.20(0.30)=0.06\). Through \(C\), it is \(0.10(0.50)=0.05\). Therefore, $$ P(0,2)_{AC}=0.14+0.06+0.05=0.25. $$
As a check, the entire row for a process starting in \(A\) is $$ \begin{aligned} P(0,2)_{AA}&=0.70(0.50)+0.20(0.20)+0.10(0.10)=0.40,\\ P(0,2)_{AB}&=0.70(0.30)+0.20(0.50)+0.10(0.40)=0.35,\\ P(0,2)_{AC}&=0.25. \end{aligned} $$ The row sums to \(0.40+0.35+0.25=1.00\), as required. Common mistake avoided: using only the direct \(A\)-to-\(C\) contribution through \(C\), \(0.10(0.50)=0.05\), would omit the routes through \(A\) and \(B\).
Worked Example: A Hypothetical Disability Model With an Absorbing State
A hypothetical income-protection model has states \(A\) (active), \(D\) (disabled), and \(X\) (dead). Death is absorbing over the intervals shown. The transition matrices for years 0 to 1 and 1 to 2 are $$ P(0,1)= \begin{pmatrix} 0.80&0.15&0.05\\ 0.10&0.80&0.10\\ 0&0&1 \end{pmatrix}, \qquad P(1,2)= \begin{pmatrix} 0.70&0.20&0.10\\ 0.05&0.75&0.20\\ 0&0&1 \end{pmatrix}. $$ Find the probability that a person active at time 0 is disabled at time 2. The matrices specify the hypothetical transition assumptions completely; the absorbing row means a person in state \(X\) remains there.
Method. The intermediate state can be \(A\), \(D\), or \(X\). The contributions to ending in \(D\) are $$ \begin{aligned} \text{through }A &: 0.80(0.20)=0.16,\\ \text{through }D &: 0.15(0.75)=0.1125,\\ \text{through }X &: 0.05(0)=0. \end{aligned} $$ The third contribution is zero because the absorbing state cannot transition to disability. Thus $$ P(0,2)_{AD}=0.16+0.1125+0=0.2725. $$
Check by calculating the other entries in the starting-\(A\) row: $$ \begin{aligned} P(0,2)_{AA}&=0.80(0.70)+0.15(0.05)+0.05(0)=0.5675,\\ P(0,2)_{AX}&=0.80(0.10)+0.15(0.20)+0.05(1)=0.16. \end{aligned} $$ The row total is \(0.5675+0.2725+0.16=1.0000\). Common mistake avoided: omitting the intermediate death state without checking its contribution. Here that contribution is zero because of the absorbing-state assumption, not because the intermediate state can be ignored in general.
Worked Example: Composing Three Intervals
A hypothetical two-state process has states 0 and 1. The transition matrices over three successive intervals, from time 0 to 1, 1 to 2, and 2 to 3, are $$ P(0,1)= \begin{pmatrix} 0.75&0.25\\ 0.40&0.60 \end{pmatrix}, \quad P(1,2)= \begin{pmatrix} 0.80&0.20\\ 0.30&0.70 \end{pmatrix}, \quad P(2,3)= \begin{pmatrix} 0.60&0.40\\ 0.10&0.90 \end{pmatrix}. $$ Find the probability of being in state 1 at time 3, given state 0 at time 0. These are supplied transition probabilities; neither interest nor mortality assumptions are part of this example.
Method. First obtain the probabilities of the intermediate state at time 2. Starting in state 0 at time 0, the probability of state 0 at time 2 is $$ P(0,2)_{00}=0.75(0.80)+0.25(0.30)=0.60+0.075=0.675. $$ The probability of state 1 at time 2 is $$ P(0,2)_{01}=0.75(0.20)+0.25(0.70)=0.15+0.175=0.325. $$
Now sum over the two possible states at time 2 to find the probability of state 1 at time 3: $$ P(0,3)_{01}=P(0,2)_{00}P(2,3)_{01}+P(0,2)_{01}P(2,3)_{11}. $$ Substitution gives $$ P(0,3)_{01}=0.675(0.40)+0.325(0.90)=0.270+0.2925=0.5625. $$
The complementary probability of ending in state 0 is \(0.675(0.60)+0.325(0.10)=0.405+0.0325=0.4375\). The two final probabilities sum to 1.0000. Common mistake avoided: treating the three intervals as if they had the same transition matrix. The supplied matrices differ, so the calculation must use the matrix for each actual interval.
Interpretation and Exam Traps
For a finite state space, composing two interval matrices calculates every endpoint probability at once. In a particular entry, the row index of the first matrix is the original state, its column index is the intermediate state, and the row index of the second matrix is that same intermediate state. The intermediate indices must match. This is the reason the product is \(P(s,t)P(t,u)\) under the stated row-and-column convention.
A frequent error is to multiply probabilities entry by entry, or to multiply just one pair of entries and treat that as the full endpoint probability. Either approach discards possible intermediate states. Another error is to reverse the order of the matrices, which changes which states are treated as intermediate and generally gives a different result. A useful check is to calculate a full row: for valid stochastic transition matrices, the composed row must be nonnegative and sum to 1.
The relationship is not restricted to health or insurance states. It applies to any finite- or countable-state Markov model, provided the transition probabilities are defined for the intervals and the Markov property holds. For actuarial state models, it is a basic way to connect shorter-period transition information to a longer-period probability without listing every possible sequence of transitions separately.
Check Your Understanding
For each problem, use the stated transition matrices and sum over the intermediate state or states.
- A two-state model has \(P(0,1)_{0,\cdot}=(0.65,0.35)\), and the probabilities of moving to state 1 over the next interval are \(P(1,2)_{0,1}=0.20\) and \(P(1,2)_{1,1}=0.75\). Find \(P(0,2)_{0,1}\). (Answer: 0.3925)
- For a three-state model, the row from starting state \(A\) in \(P(0,1)\) is \((0.60,0.30,0.10)\). The probabilities of moving to target state \(C\) in \(P(1,2)\), from \(A,B,C\) respectively, are \(0.20,0.50,0.40\). Find \(P(0,2)_{AC}\). (Answer: 0.3100)
- A two-state model has \(P(0,1)=\begin{pmatrix}0.8&0.2\\0.3&0.7\end{pmatrix}\) and \(P(1,2)=\begin{pmatrix}0.6&0.4\\0.1&0.9\end{pmatrix}\). Find the probability of starting and ending in state 0. (Answer: 0.5000)
- A three-state model has \(P(0,1)_{A,\cdot}=(0.50,0.30,0.20)\). The probabilities of ending in \(C\) over the second interval from intermediate states \(A,B,C\) are \(0.10,0.40,0.70\). Calculate the probability of starting in \(A\) and ending in \(C\) over both intervals. (Answer: 0.3100)
- In a finite-state model, a proposed calculation of one entry in \(P(s,u)\) uses only the route through one intermediate state, even though the process can occupy three states at time \(t\). State the correct operation and explain what must be included. (Answer: Sum \(P(s,t)_{ik}P(t,u)_{kj}\) over all three possible intermediate states \(k\).)