Why the Generator’s Block Structure Matters
In a finite continuous-time Markov chain, an absorbing state cannot be left once entered. If several states are absorbing, the generator separates into a block for the states that have not yet been absorbed and a block for the absorbing states. Exponentiating this block matrix gives more than a compact representation of the transition semigroup: its upper-right block directly records the probabilities of having reached each absorbing state by a specified time.
Let \((X_t)_{t\ge0}\) be a finite-state continuous-time Markov chain on a probability space \((\Omega,\mathcal F,\mathbb P)\), with generator \(G\) and transition semigroup \(P(t)=e^{tG}\). Suppose its state space is ordered with the \(m\) nonabsorbing states first and the \(r\) absorbing states last. Here, “absorbing” means that from an absorbing state the chain stays there with probability \(1\) at every future time. Thus every absorbing row of \(G\) is zero. Write the generator in block form as
where \(T\) is the \(m\times m\) block of rates between nonabsorbing states and \(R\) is the \(m\times r\) block of rates from those states to absorbing states. Since generator rows sum to zero, each row of \(T\), added to the corresponding row of \(R\), sums to zero. The block \(T\) is therefore generally not itself a generator: its row sums can be negative because rates of leaving for absorption are included in its diagonal entries.
Proof. Let \(E(t)=e^{tG}\), and write its blocks as \(E(t)=\begin{bmatrix}A(t)&B(t)\\C(t)&D(t)\end{bmatrix}\). The matrix exponential satisfies \(E'(t)=GE(t)\) and \(E(0)=I\). Block multiplication gives \(A'=TA+RC\), \(B'=TB+RD\), \(C'=0\), and \(D'=0\), with \(A(0)=I_m\), \(B(0)=0\), \(C(0)=0\), and \(D(0)=I_r\). The last two differential equations give \(C(t)=0\) and \(D(t)=I_r\). Therefore \(A'=TA\), \(A(0)=I_m\), so \(A(t)=e^{tT}\). Also \(B'=TB+R\), \(B(0)=0\). Define \(H(t)=\int_0^t e^{uT}R\,du\). Differentiating the equivalent expression \(H(t)=\int_0^t e^{(t-s)T}R\,ds\) gives \(H'=TH+R\) and \(H(0)=0\). The linear matrix differential equation for \(B\) has a unique solution, so \(B=H\). This proves the block formula. The probability interpretations follow because \(P(t)_{ij}\) is the transition probability from \(i\) to \(j\), and an absorbing state occupied at time \(t\) must have been reached by time \(t\). \(\square\)
If the initial distribution is the row vector \(\mu=(\alpha,\beta)\), where \(\alpha\) assigns mass to the nonabsorbing states and \(\beta\) to the absorbing states, multiplication by the block formula gives the distribution at time \(t\): its nonabsorbing part is \(\alpha e^{tT}\), and its absorbing part is \(\beta+\alpha\int_0^t e^{uT}R\,du\). In particular, the initial mass \(\beta\) stays where it is; only mass in \(\alpha\) contributes new absorption probability.
Worked Example: Competing Absorption Destinations
A chain starts in state \(1\). It leaves that state at rate \(3\), going to absorbing state \(2\) at rate \(2\) or absorbing state \(3\) at rate \(1\). Find the state probabilities at time \(t\), and the probabilities of absorption at each destination by time \(t\).
In the state order \(1,2,3\), the generator is \(G=\begin{bmatrix}-3&2&1\\0&0&0\\0&0&0\end{bmatrix}\). Its first row sums to \(-3+2+1=0\), and each absorbing row sums to \(0\). Thus \(T=[-3]\) and \(R=[2\ \ 1]\). The blocks in the theorem are \(e^{tT}=[e^{-3t}]\) and
Starting in state \(1\), the probability of still being in state \(1\) at time \(t\) is \(e^{-3t}\). The probabilities of being in states \(2\) and \(3\) are respectively \(\tfrac23(1-e^{-3t})\) and \(\tfrac13(1-e^{-3t})\). Their sum is \(e^{-3t}+\tfrac23(1-e^{-3t})+\tfrac13(1-e^{-3t})=1\), as required.
Check at \(t=1\). The probabilities are \(e^{-3}\), \(\tfrac23(1-e^{-3})\), and \(\tfrac13(1-e^{-3})\). The second probability is a cumulative probability, obtained by integrating the rate \(2\) times the probability \(e^{-3u}\) of still being in state \(1\): \(\int_0^1 2e^{-3u}\,du=\tfrac23(1-e^{-3})\). This also checks the total, since the two absorption probabilities sum to \(1-e^{-3}\), leaving \(e^{-3}\) in state \(1\).
Worked Example: Two Sequential Transitions Before Absorption
Consider states \(1,2,3\), with state \(3\) absorbing. From state \(1\) the chain moves to state \(2\) at rate \(2\); from state \(2\) it moves to state \(3\) at rate \(3\). Find the transition probabilities at time \(t\) when the chain starts in state \(1\).
In the order \(1,2,3\), the generator is \(G=\begin{bmatrix}-2&2&0\\0&-3&3\\0&0&0\end{bmatrix}\). Its rows sum to \(-2+2=0\), \(-3+3=0\), and \(0\). Hence \(T=\begin{bmatrix}-2&2\\0&-3\end{bmatrix}\) and \(R=\begin{bmatrix}0\\3\end{bmatrix}\). Directly solving the two-dimensional matrix exponential, or differentiating the displayed matrix and checking its value at \(t=0\), gives
For example, the upper-right entry \(v(t)\) satisfies \(v'(t)=-2v(t)+2e^{-3t}\) and \(v(0)=0\); substituting \(v(t)=2(e^{-2t}-e^{-3t})\) gives \(v'(t)=-4e^{-2t}+6e^{-3t}=-2v(t)+2e^{-3t}\). Thus the transient block is verified. The absorbing block is
Therefore, from state \(1\), the probabilities of being in states \(1,2,3\) at time \(t\) are \(e^{-2t}\), \(2(e^{-2t}-e^{-3t})\), and \(1-3e^{-2t}+2e^{-3t}\). The first two sum to \(3e^{-2t}-2e^{-3t}\); adding the third gives \(1\). At \(t=0\), the probabilities are \(1,0,0\), as the initial condition requires.
Eventual Absorption and Its Conditions
The finite-time block formula does not require absorption to occur eventually. To obtain eventual absorption probabilities by taking \(t\) to infinity, an additional condition is needed: from every nonabsorbing state, the chain must reach the absorbing set with probability \(1\). In a finite chain, under this condition the transient block decays to zero, and its integral converges.
Proof. For each nonabsorbing state \(i\), almost-sure absorption implies that there is a finite time \(t_i\) for which the probability of absorption by \(t_i\) is positive. Since there are finitely many such states, choose \(t_*=\max_i t_i\) and \(\delta=\min_i\mathbb P_i(T_A\le t_*)\), where \(T_A=\inf\{t\ge0:X_t\text{ is absorbing}\}\). Monotonicity of the absorption probability gives \(\delta>0\). By the Markov property, conditional on still being nonabsorbing at time \(kt_*\), the probability of remaining nonabsorbing for the next interval of length \(t_*\) is at most \(1-\delta\). Consequently every row sum of \(e^{kt_*T}\) is at most \((1-\delta)^k\), so \(e^{tT}\to0\). The same bound over successive intervals implies that \(\int_0^\infty e^{uT}\,du\) converges entry by entry. If \(Tx=0\), then \(e^{tT}x=x\) for every \(t\), while \(e^{tT}\to0\); hence \(x=0\), and the finite square matrix \(T\) is invertible. Finally, integrating \(\frac{d}{du}e^{uT}=Te^{uT}\) from \(0\) to \(b\) gives \(T\int_0^b e^{uT}\,du=e^{bT}-I\). Letting \(b\to\infty\) gives \(T\int_0^\infty e^{uT}\,du=-I\), so the integral equals \(-T^{-1}\). Multiplication by \(R\) proves the formula for \(B_\infty\). Almost-sure absorption means the eventual destination is one of the absorbing states, so its probabilities in each row sum to \(1\). \(\square\)
Worked Example: Eventual Destination in a Symmetric Chain
There are two nonabsorbing states, \(1\) and \(2\), and one absorbing state, \(3\). From either nonabsorbing state, the chain jumps to the other at rate \(1\) and to state \(3\) at rate \(2\). Find the probability of eventual absorption in state \(3\) from each starting state, and verify the finite-time formula.
The generator is \(G=\begin{bmatrix}-3&1&2\\1&-3&2\\0&0&0\end{bmatrix}\). The first two rows sum to \(-3+1+2=0\), and the absorbing row sums to \(0\). Thus \(T=\begin{bmatrix}-3&1\\1&-3\end{bmatrix}\) and \(R=\begin{bmatrix}2\\2\end{bmatrix}\). Since \(T\begin{bmatrix}1\\1\end{bmatrix}=-2\begin{bmatrix}1\\1\end{bmatrix}\), solving \(Tx=-R\) gives \(x=\begin{bmatrix}1\\1\end{bmatrix}\): indeed, \(T\begin{bmatrix}1\\1\end{bmatrix}=\begin{bmatrix}-2\\-2\end{bmatrix}=-R\). The eventual absorption probability is therefore \(1\) from either starting state.
For a finite-time check, the vectors \(\begin{bmatrix}1\\1\end{bmatrix}\) and \(\begin{bmatrix}1\\-1\end{bmatrix}\) are eigenvectors of \(T\) with eigenvalues \(-2\) and \(-4\), respectively. It follows that
Each row sums to \(e^{-2t}\). Also \(e^{uT}R=2e^{-2u}\begin{bmatrix}1\\1\end{bmatrix}\), so the finite-time absorption block is \((1-e^{-2t})\begin{bmatrix}1\\1\end{bmatrix}\). For either initial state, the transient probability \(e^{-2t}\) and absorption probability \(1-e^{-2t}\) sum to \(1\); as \(t\to\infty\), the absorption probability tends to \(1\), matching the linear-system calculation.
Rates, Probabilities, and a Common Pitfall
The entry \(R_{ia}\) is a rate, not a probability of absorption by a specified time. The finite-time probability is obtained by integrating the rate weighted by the probability of occupying each transient state. More generally, its \((i,a)\) entry is \(\int_0^t\sum_j(e^{uT})_{ij}R_{ja}\,du\). This accounts for paths that pass through several transient states before absorption. The corresponding integrand is an instantaneous probability flow; the integral accumulates that flow over time.
Common mistake. In the competing-destinations example, the instantaneous probability flow from state \(1\) to state \(2\) at time \(t\) is \(2e^{-3t}\). It is incorrect to report \(2e^{-3}\) as the probability of being absorbed in state \(2\) by time \(1\). The correct cumulative probability is \(\int_0^1 2e^{-3u}\,du=\tfrac23(1-e^{-3})\). These expressions are not equal: equality would require \(2e^{-3}=\tfrac23(1-e^{-3})\), hence \(4e^{-3}=1\), or \(e^3=4\). But \(e^3>1+3+\tfrac92>4\). The rate at the endpoint and the probability accumulated over the full interval are different quantities.
The condition for the eventual-absorption formula also matters. For example, take states \(1,2,3,4\), with state \(4\) absorbing, transitions \(1\to2\) at rate \(1\), \(2\to3\) at rate \(1\), and \(3\to2\) at rate \(1\), with no other transitions. The generator rows sum to zero: its nonzero rows are \((-1,1,0,0)\), \((0,-1,1,0)\), and \((0,1,-1,0)\). Starting from \(1\), the chain enters the closed class \(\{2,3\}\) and never reaches \(4\). The finite-time block formula still applies with \(T\) on states \(1,2,3\), but \(e^{tT}\) does not tend to zero and \(T\) is singular. Thus one cannot use \(-T^{-1}R\) without the almost-sure absorption condition.
For finite-time calculations, the practical sequence is to order transient states before absorbing states, verify that the generator rows sum to zero, identify \(T\) and \(R\), and then use the block exponential. Keep the distinction between transient-state probabilities \(e^{tT}\), cumulative absorption probabilities \(\int_0^t e^{uT}R\,du\), and eventual absorption probabilities \(-T^{-1}R\) when its hypotheses hold.
Check Your Understanding
For each question, identify whether it concerns a finite-time probability or an eventual probability, and check any proposed probability rows by summing their entries.
- A single nonabsorbing state has rates \(1\) and \(2\) to two absorbing states. Write \(T\), \(R\), and the two absorption probabilities by time \(t\).
- For the sequential chain in the second worked example, verify directly that the probabilities from state \(2\) sum to \(1\) at time \(t\).
- Explain why the upper-right block is an integral of \(e^{uT}R\), rather than just \(R\).
- For the symmetric chain in the third worked example, verify that \(-T^{-1}R\) has row sums equal to \(1\).
- What fails in the eventual-absorption argument if a nonabsorbing closed class can be reached?