Why Make the Target State Absorbing?
A transition matrix describes the state occupied at a particular time. A first-passage question asks something different: has the process reached a specified state at any time up to the horizon? For instance, a hypothetical insurer might track a policy through several administrative states and want the probability that it enters a specified claim state within two years. If a policy enters that state and later leaves it, the event has still occurred.
For a finite-state, discrete-time Markov model, a simple matrix modification turns this “reached by time” event into an ordinary endpoint probability. The key is to make the target state absorbing in the calculation: once the modified process reaches it, the process remains there. All probabilities below are conditional on the stated initial state or initial distribution, and each transition represents one specified time interval.
Making the Target Absorbing
Use a row-vector convention: the entry \(p_{ij}\) of a transition matrix \(P\) is the probability of moving from state \(i\) to state \(j\) in one step. To make target state \(T\) absorbing, replace its row with a row that has probability 1 in column \(T\) and 0 in every other column. Leave all other rows unchanged. Call the resulting matrix \(P^*\).
For a time-homogeneous Markov model with transition matrix \(P\), this matrix modification gives the first-passage probability over \(n\) steps:
The formula applies when the process is Markov at the chosen time steps and the stated transition matrix describes each step. It does not require the original target state to be absorbing. If the target is already occupied at step 0, then \(\tau_T=0\), so the probability of reaching it by any nonnegative horizon is 1.
Why the Modified Matrix Works
The modification does not change transitions before the first visit to \(T\): every non-target row is unchanged. After the first visit, it keeps the modified process in \(T\). Therefore, at step \(n\), the modified process is in \(T\) exactly when the original process has visited \(T\) at least once by that step.
Every transition from a non-target state to \(T\) retains its original probability.
Set the target row to probability 1 of remaining in \(T\).
Under the modified model, being in \(T\) at step \(n\) means that \(T\) has been reached by step \(n\).
This is a path-by-path justification: until the original process first reaches \(T\), its path has exactly the same transition probabilities under both models. Every path that reaches \(T\) by step \(n\) is then held there in the modified model, while a path that has not reached \(T\) cannot be in \(T\) at step \(n\). Adding the probabilities of those paths gives the stated matrix entry.
Worked Examples
Worked Example: Reaching a Target Within Two Steps
A hypothetical process has states \(A\), \(B\), and target \(T\), with transition matrix
where rows and columns are ordered \(A,B,T\). Starting in \(A\), find the probability of reaching \(T\) within two steps. The entries are probabilities per step; no interest or other time discounting is involved.
Method. Replace the row for \(T\) by \((0,0,1)\). The modified matrix is
The two-step probability in question is entry \((A,T)\) of \(P^{*\,2}\). Expanding by the state occupied after the first step gives
Thus the probability is 0.39. The three terms account, respectively, for paths that are in \(A\), in \(B\), or already in \(T\) after one step. As a check, the probability of still being outside \(T\) after two steps is \(0.5(0.5+0.3)+0.3(0.2+0.5)=0.40+0.21=0.61\); therefore the probability of having reached \(T\) is \(1-0.61=0.39\). The method avoids counting only paths that end in \(T\) under the original matrix.
Worked Example: Different Transition Matrices Across Two Intervals
A hypothetical population begins in states \(X,Y,Z\) with probabilities \(0.70,0.20,0.10\), respectively. The target is \(T\), and the horizon is two successive intervals. In the first interval, the relevant transition probabilities are: from \(X\), to \(U,V,D,T\) with probabilities \(0.50,0.30,0.10,0.10\); from \(Y\), to \(U,V,D,T\) with probabilities \(0.20,0.50,0.10,0.20\); and from \(Z\), to \(T\) with probability 1. In the second interval, \(U\) moves to \(T\) with probability \(0.20\) and otherwise to \(D\); \(V\) moves to \(T\) with probability \(0.30\) and otherwise to \(D\); \(D\) stays in \(D\); and \(T\) stays in \(T\). Find the probability of reaching \(T\) by the end of the second interval.
Use a common state space containing all the named states, and make \(T\) absorbing in both interval matrices. Any rows not used by the initial distribution or its resulting distribution can be completed with valid transition probabilities; they do not affect this calculation. First propagate the initial distribution through interval 1. The resulting probabilities in \(U,V,D,T\) are
These probabilities sum to \(0.39+0.31+0.09+0.21=1.00\), as required. In interval 2, the probabilities of being in \(T\) at the end, conditional on being in \(U,V,D,T\) at its start, are \(0.20,0.30,0,1\). Hence
The first-passage probability is 0.381. The calculation correctly preserves the \(0.21\) that reached \(T\) during the first interval: in the modified model, that probability remains in \(T\) during interval 2. A common mistake is to omit this already-reached mass or to use the first-interval matrix again for the second interval.
Worked Example: Reaching the Target Within Three Steps
A hypothetical process starts in \(A\), with non-target states \(A,B\) and target \(T\). At each step, \(A\) moves to \(A,B,T\) with probabilities \(0.50,0.30,0.20\), and \(B\) moves to \(A,B,T\) with probabilities \(0.20,0.50,0.30\). Once in \(T\), the modified process stays there. Find the probability of reaching \(T\) by step 3.
Method. Track the probabilities that remain in the non-target states. At step 0, these are \(P(A)=1\) and \(P(B)=0\). After step 1, they are \(P(A)=0.50\) and \(P(B)=0.30\), so the probability already in \(T\) is \(0.20\).
After step 2, the non-target probabilities are
The probability newly reaching \(T\) at step 2 is \(0.50(0.20)+0.30(0.30)=0.19\). Thus the cumulative probability by step 2 is \(0.20+0.19=0.39\).
After step 3, the non-target probabilities are
The remaining probability outside \(T\) is \(0.215+0.243=0.458\). Therefore, the probability of reaching \(T\) by step 3 is \(1-0.458=\mathbf{0.542}\). As a second check, the probability newly reaching \(T\) at step 3 is \(0.31(0.20)+0.30(0.30)=0.152\), and \(0.39+0.152=0.542\). The method avoids adding only the step-3 arrivals while forgetting arrivals at earlier steps.
Matrix Order and Time Units
When transition probabilities vary by interval, modify the target row in each interval matrix. With row vectors and interval matrices \(P_1,P_2,\ldots,P_n\), the distribution at the horizon is obtained by multiplying in chronological order:
The component for \(T\) is the probability of reaching \(T\) by the end of interval \(n\). The initial distribution \(\boldsymbol{\pi}_0\) must be stated in the same state ordering as the matrices. If the question gives a horizon in years but the matrices describe months, convert the horizon to the corresponding number of monthly transitions before multiplying. A transition step has the duration assigned by the model; it is not automatically one year.
Common Exam Trap
In Worked Example 1, using the original matrix \(P\) and reading its two-step endpoint probability would give
That is not the probability of ever reaching \(T\) within two steps: it counts only paths that are in \(T\) at step 2. Some paths reach \(T\) at step 1 and then leave under the original matrix. The corrected calculation uses the absorbing row, giving \(0.39\). Always identify whether the requested event is “in \(T\) at the horizon” or “reached \(T\) by the horizon” before selecting a matrix calculation.
A second useful check is that a first-passage probability cannot decrease as the horizon increases. If a computed probability for reaching \(T\) by step \(n+1\) is smaller than the probability by step \(n\), the calculation or the event definition should be reviewed. Also check that each transition-matrix row sums to 1 and that the final probability lies between 0 and 1.
Check Your Understanding
For each problem, use the stated time steps and interpret “by” as including a visit at any earlier step.
- A process starts in \(A\). In each step, \(A\) moves to \(A,B,T\) with probabilities \(0.60,0.25,0.15\), and \(B\) moves to \(A,B,T\) with probabilities \(0.40,0.40,0.20\). Find the probability of reaching \(T\) within two steps. (Answer: 0.2900)
- A process starts in \(B\). At each step, \(A\) moves to \(A,B,T\) with probabilities \(0.40,0.40,0.20\), and \(B\) moves to \(A,B,T\) with probabilities \(0.10,0.60,0.30\). Find the probability of reaching \(T\) within two steps. (Answer: 0.5000)
- An initial population has probabilities \(0.60\) in \(A\) and \(0.40\) in \(B\). Each step has transitions \(A\to(A,B,T)\) with probabilities \((0.40,0.40,0.20)\) and \(B\to(A,B,T)\) with probabilities \((0.10,0.60,0.30)\). Find the probability of reaching \(T\) within two steps. (Answer: 0.4400)
- A process starts in \(A\). In the first interval, \(A\) moves to \(A,B,T\) with probabilities \(0.50,0.30,0.20\). In the second interval, the probabilities of moving to \(T\) from \(A\) and \(B\) are \(0.10\) and \(0.40\), respectively; \(T\) is made absorbing in both intervals. Find the probability of reaching \(T\) by the end of interval 2. (Answer: 0.3700)
- A process starts in \(A\), and at every step \(A\) moves to \(A,B,T\) with probabilities \(0.50,0.20,0.30\), while \(B\) moves to \(A,B,T\) with probabilities \(0.25,0.50,0.25\). Find the probability of reaching \(T\) within three steps. (Answer: 0.6400)
- An initial distribution assigns probabilities \(0.10,0.60,0.30\) to \(T,A,B\), respectively. In one step, the probabilities of moving to \(T\) from \(A\) and \(B\) are \(0.20\) and \(0.30\), and \(T\) is absorbing for the first-passage calculation. Find the probability of reaching \(T\) by the end of that step. (Answer: 0.3100)