Introduction
A transition model may change its intensities over time. If the intensity matrix is constant within each interval but differs between intervals, an endpoint probability must account for every interval in the order in which it occurs. The key operation is matrix multiplication: it combines all possible intermediate states while preserving the chronology of the model.
This tutorial works through calculations in which the intensity matrix describes each regime and the transition matrix over that regime’s duration is either supplied or obtained from the stated model. The target is a probability of being in a specified state at a specified endpoint. It is not, by itself, a probability of having visited that state at some earlier time.
From Intensities to Interval Matrices
An intensity matrix \(Q\) specifies the instantaneous transition rates for a continuous-time Markov model. Its off-diagonal entry \(q_{ij}\) is the instantaneous rate of transition from state \(i\) to state \(j\), and its diagonal entry is \(q_{ii}=-\sum_{j\ne i}q_{ij}\). Rates are expressed per unit of time, so the interval duration must use the same unit. For example, a rate per year must be paired with a duration measured in years.
When \(Q\) is constant over an interval, the transition matrix for that interval depends on both \(Q\) and its duration. In a multi-interval problem, write each interval matrix in chronological order. If the first interval runs from time \(t_0\) to \(t_1\), and the next runs from \(t_1\) to \(t_2\), then
This is the Chapman–Kolmogorov composition rule. If there are more intervals, continue multiplying the next interval matrix on the right. The result remains a transition matrix from the initial time to the final time.
Why the Matrix Product Works
The product combines the possible states at the boundary between intervals. For example, to go from state \(i\) at \(t_0\) to state \(k\) at \(t_2\), the process can be in any intermediate state \(j\) at \(t_1\). Conditional on that intermediate state, the probability of the two successive transitions is the product of their probabilities. Summing over all intermediate states gives
The right-hand side is exactly the \((i,k)\) entry of \(P(t_0,t_1)P(t_1,t_2)\). This explains both the multiplication and its order: the first matrix takes the process to the intermediate time, and the second carries it from that time to the endpoint.
Use the same order for the rows and columns of every interval matrix.
Check that each interval’s intensity matrix and duration refer to the intended period.
Multiply the first interval matrix by the second, then by the third, and so on.
The row is the starting state and the column is the endpoint state.
Worked Calculations
Worked Example: Two Regimes for a Two-State Model
A hypothetical income-protection model has states 1 (working) and 2 (unable to work). It starts in state 1. Each interval lasts one year, and the intensities are constant within each interval. In the first year the intensity matrix is \(Q_1=\begin{pmatrix}-\frac14\ln 2&\frac14\ln 2\\ \frac34\ln 2&-\frac34\ln 2\end{pmatrix}\). In the second year it is \(Q_2=\begin{pmatrix}-\frac12\ln 2&\frac12\ln 2\\ \frac12\ln 2&-\frac12\ln 2\end{pmatrix}\). Find the probability that the person is unable to work at the end of year 2.
Method. Calculate the one-year transition matrix for each regime, compose in time order, and select the entry from starting state 1 to ending state 2. For a two-state model with rates \(a\) from state 1 to state 2 and \(b\) from state 2 to state 1, the sum of rates is \(k=a+b\). The probability of state 2 at time \(t\), starting from state 1, is \(\frac{a}{k}(1-e^{-kt})\); the probability of state 1 is its complement. Starting in state 2, the probability of state 1 is \(\frac{b}{k}(1-e^{-kt})\).
First year. Here \(a=\frac14\ln 2\), \(b=\frac34\ln 2\), so \(k=\ln 2\), and \(e^{-k}=1/2\). Thus the probability of moving from state 1 to state 2 is \(\frac14(1-\frac12)=\frac18\), and the probability of remaining in state 1 is \(\frac78\). From state 2, the probability of moving to state 1 is \(\frac34(1-\frac12)=\frac38\), leaving probability \(\frac58\) in state 2. Therefore
Second year. Both transition rates are \(\frac12\ln 2\), so \(k=\ln 2\), with equal long-run probabilities for the two states. Over one year, each state’s probability of remaining in its starting state is \(\frac12+\frac12e^{-k}=\frac34\), and the probability of moving to the other state is \(\frac14\). Hence
Compose and select. The required entry is row 1, column 2 of \(P_1P_2\):
The endpoint probability is \(0.3125\). As a check, the full first row of the product is \((\frac{11}{16},\frac5{16})\), which sums to 1. Common mistake avoided: using only the first-year transition probability \(\frac18\) ignores the second year, during which movement between states remains possible.
Worked Example: Three States Across Two Intervals
A hypothetical care model has three states, numbered 1, 2, and 3. Each state can transition directly to either of the other two states. During the first year, the intensity from each state to each other state is \(-\ln(0.4)/3\) per year. During the second year, each such intensity is \(-\ln(0.1)/3\) per year. Find the probability of ending in state 3, given that the model starts in state 1.
Method. For three states with equal intensity \(\lambda\) from each state to each other state, the diagonal entries of \(Q\) are \(-2\lambda\). Over an interval of length \(t\), the transition matrix has diagonal entries \((1+2x)/3\) and off-diagonal entries \((1-x)/3\), where \(x=e^{-3\lambda t}\). This follows because the uniform distribution over the three states is unchanged by the transitions, while deviations from uniformity decay by the factor \(e^{-3\lambda t}\).
For the first year, \(x=e^{-3\lambda}=0.4\). The diagonal probability is \((1+2(0.4))/3=1.8/3=0.6\), and each off-diagonal probability is \((1-0.4)/3=0.2\). Thus
For the second year, \(x=e^{-3\lambda}=0.1\). The diagonal probability is \((1+2(0.1))/3=0.4\), and each off-diagonal probability is \((1-0.1)/3=0.3\). Therefore
To finish in state 3, the intermediate state after year 1 could be 1, 2, or 3. The corresponding contributions are \(0.6(0.3)=0.18\), \(0.2(0.3)=0.06\), and \(0.2(0.4)=0.08\). Summing them gives
The answer is \(0.32\). Each row of both interval matrices sums to 1, and the calculated probability is between 0 and 1. Common mistake avoided: the answer is not just the entry \(0.2\) in the first-year matrix. That entry gives the probability of being in state 3 after year 1, not after both intervals.
Worked Example: Three Intervals and a Specified Endpoint
A hypothetical two-state rehabilitation model starts in state 1. Its three successive one-year transition matrices are \(P_1=\begin{pmatrix}0.875&0.125\\0.375&0.625\end{pmatrix}\), \(P_2=\begin{pmatrix}0.75&0.25\\0.25&0.75\end{pmatrix}\), and \(P_3=\begin{pmatrix}0.70&0.30\\0.45&0.55\end{pmatrix}\). These correspond, respectively, to the one-year intensity matrices from the previous example’s first regime, the equal-rate second regime, and a third regime with rates \(a=0.4\ln 4\) and \(b=0.6\ln 4\) per year. Find the probability of ending in state 2 after year 3.
Method. Multiply in chronological order and keep the full distribution at each intermediate time. After year 1, starting in state 1 gives the row vector \((0.875,0.125)\). After year 2, the probability of state 1 is \(0.875(0.75)+0.125(0.25)=0.65625+0.03125=0.6875\). The probability of state 2 is \(0.875(0.25)+0.125(0.75)=0.21875+0.09375=0.3125\). Thus the distribution after two years is \((0.6875,0.3125)\).
In the third interval, the probability of ending in state 2 is \(0.6875(0.30)+0.3125(0.55)\). The two contributions are \(0.20625\) and \(0.171875\), so
The endpoint probability is \(0.378125\). The probability of ending in state 1 is \(0.6875(0.70)+0.3125(0.45)=0.48125+0.140625=0.621875\), and \(0.378125+0.621875=1\), as required. Common mistake avoided: selecting an entry from \(P_3\) alone treats the state at the start of year 3 as known, even though the model’s state at that time is uncertain.
Order, Interpretation, and Checks
With the row-vector convention, interval matrices are multiplied from left to right in chronological order. Reversing the order generally changes the answer because interval matrices need not commute. In the first worked example, the probability from state 1 to state 2 for the correct order was \(\frac5{16}=0.3125\). Reversing the matrices gives \(\frac34\cdot\frac18+\frac14\cdot\frac58=\frac3{32}+\frac5{32}=\frac14=0.25\), a different result.
A multi-interval endpoint probability also does not identify whether the model visited a particular state along the way. It describes the state at the specified endpoint only. For an endpoint calculation, the intermediate states are summed over in the matrix product; their order-specific paths are not reported separately.
- Check the dimensions. The number and order of states must agree across adjacent matrices.
- Check each row sum. A transition matrix should have nonnegative entries and each row should sum to 1, apart from small rounding differences.
- Check the time units. A rate per year combined with a duration in months requires a duration of months divided by 12.
- Check the selected entry. Use the starting state for the row and the requested endpoint state for the column.
- Keep the interval order. Under the row-vector convention, multiply the earliest interval matrix first.
These checks are particularly useful when a problem supplies several matrices and asks for one probability. Writing the state order and the chronological product before calculating reduces the risk of using a plausible but incorrect matrix entry.
Check Your Understanding
For each problem, use the row-vector convention and compose interval matrices in chronological order.
- A process starts in state 1. Its two successive transition matrices are \(P_1=\begin{pmatrix}0.8&0.2\\0.3&0.7\end{pmatrix}\) and \(P_2=\begin{pmatrix}0.6&0.4\\0.1&0.9\end{pmatrix}\). Find the probability of ending in state 2. (Answer: \(0.5000\))
- A three-state process starts in state 2. Its successive matrices are \(P_1=\begin{pmatrix}0.7&0.2&0.1\\0.1&0.6&0.3\\0.2&0.2&0.6\end{pmatrix}\) and \(P_2=\begin{pmatrix}0.5&0.3&0.2\\0.2&0.5&0.3\\0.1&0.4&0.5\end{pmatrix}\). Find the probability of ending in state 1. (Answer: \(0.2000\))
- A two-state process starts in state 1 and has three successive matrices \(P_1=\begin{pmatrix}0.9&0.1\\0.4&0.6\end{pmatrix}\), \(P_2=\begin{pmatrix}0.7&0.3\\0.2&0.8\end{pmatrix}\), and \(P_3=\begin{pmatrix}0.8&0.2\\0.1&0.9\end{pmatrix}\). Find the probability of ending in state 2. (Answer: \(0.4450\))
- A process starts with state distribution \((0.6,0.4)\). It then has transition matrices \(P_1=\begin{pmatrix}0.8&0.2\\0.3&0.7\end{pmatrix}\) and \(P_2=\begin{pmatrix}0.9&0.1\\0.4&0.6\end{pmatrix}\). Find the probability of ending in state 2. (Answer: \(0.3000\))