The Transient and Absorption Blocks Q and R

Reading the Two Transient-State Blocks

In a finite absorbing chain, a transition from a transient state can go to a transient state—possibly the same state—or to an absorbing state. Separating these destinations in the transition matrix gives two useful blocks: $Q$, which records transitions that remain among transient states, and $R$, which records transitions from transient states into absorbing states. Their row sums divide the one-step probability mass between those two possibilities.

Let $(X_n)_{n\ge0}$ be a discrete-time Markov chain on a finite state space $S$, defined on a probability space $(\Omega,\mathcal{F},\mathbb{P})$. Its transition matrix is $P=(p_{ij})$, where $p_{ij}=\mathbb{P}(X_{n+1}=j\mid X_n=i)$ is the transition probability from $i$ to $j$; each row of $P$ sums to $1$. Assume the states are partitioned into a nonempty set $T$ of transient states and a nonempty set $A$ of absorbing states. A state $a$ is absorbing when $p_{aa}=1$, so $p_{aj}=0$ for every $j\ne a$. Order the states with all states in $T$ first and all states in $A$ last.

Definition: In this ordering, the canonical form of the transition matrix is $$ P=\begin{bmatrix}Q&R\\0&I\end{bmatrix}. $$ Here $Q=(p_{ij})_{i,j\in T}$ contains transient-to-transient transition probabilities, including any self-transition probabilities; $R=(p_{ia})_{i\in T,\ a\in A}$ contains transient-to-absorbing transition probabilities; $0$ is the block of transitions from absorbing states to transient states; and $I$ is the identity matrix for the absorbing states.

The names of the blocks describe both their rows and their columns. Each row of $Q$ corresponds to a starting transient state, and each column corresponds to a possible transient destination. Each row of $R$ has the same starting-state interpretation, but its columns correspond to absorbing destinations. In particular, an entry on the diagonal of $Q$ is not part of $R$: it is the probability of staying in the same transient state for one step.

Why the Row Sums Split in Two

Theorem: For every transient state $i\in T$, the entries in row $i$ of $Q$ and row $i$ of $R$ are nonnegative, and $$ \sum_{j\in T}q_{ij}+\sum_{a\in A}r_{ia}=1. $$ Consequently, each row sum of $Q$ is at most $1$, each row sum of $R$ is at most $1$, and the two row sums add to $1$. The row sum of $Q$ is the one-step probability of moving to a transient state, while the row sum of $R$ is the one-step probability of moving to an absorbing state.

Proof. Fix $i\in T$. The entries $q_{ij}$ and $r_{ia}$ are entries of the transition matrix $P$, so they are nonnegative. By the partition of the state space, every possible destination of a step from $i$ belongs to exactly one of $T$ or $A$. Thus the full row sum for $i$ splits into the sum over these two disjoint sets: $\sum_{j\in T}q_{ij}+\sum_{a\in A}r_{ia}=\sum_{k\in S}p_{ik}=1$. For the probability interpretation, under $\mathbb{P}_i$ the chain starts at $i$, and the events $X_1=j$, for $j\in T$, are disjoint and have total probability $\sum_{j\in T}q_{ij}$. The events $X_1=a$, for $a\in A$, are also disjoint and have total probability $\sum_{a\in A}r_{ia}$. Together these events exhaust all possible destinations at time $1$, proving the interpretation. Since the two sums are nonnegative and add to $1$, each is at most $1$. $\square$

This theorem does not say that every row sum of $Q$ is strictly less than $1$. A transient state may have no chance of entering $A$ on the next step, in which case its row of $R$ sums to $0$ and its row of $Q$ sums to $1$. Conversely, if a transient state must enter $A$ on the next step, its row of $Q$ sums to $0$ and its row of $R$ sums to $1$. The identity $Q\mathbf{1}+R\mathbf{1}=\mathbf{1}$, with ones-vectors of the appropriate dimensions, is the compact form of the row-sum relationship.

The block form itself follows from the absorbing-state definition, not from a calculation about long-run behavior. In the ordering above, rows for absorbing states have zeros in every transient-state column, and have a $1$ in the column for that same absorbing state. Those rows therefore make up the lower blocks $0$ and $I$. The upper blocks are obtained by splitting each transient state's row according to whether its destination is in $T$ or $A$.

Extracting the Blocks in Practice

Worked Example: Reading Q and R in Canonical Order

Consider states $1,2,3,4$, with $1,2$ transient and $3,4$ absorbing. In the order $1,2,3,4$, let $$ P=\begin{bmatrix} \tfrac12&\tfrac14&\tfrac14&0\\ 0&\tfrac13&\tfrac13&\tfrac13\\ 0&0&1&0\\ 0&0&0&1 \end{bmatrix}. $$ Before extracting blocks, check the full transition rows: row $1$ sums to $\tfrac12+\tfrac14+\tfrac14+0=1$; row $2$ sums to $0+\tfrac13+\tfrac13+\tfrac13=1$; row $3$ sums to $0+0+1+0=1$; and row $4$ sums to $0+0+0+1=1$. The last two states are absorbing because each of their rows places probability $1$ on that same state.

The first two rows and first two columns give $Q$, while the first two rows and last two columns give $R$: $$ Q=\begin{bmatrix}\tfrac12&\tfrac14\\0&\tfrac13\end{bmatrix}, \qquad R=\begin{bmatrix}\tfrac14&0\\\tfrac13&\tfrac13\end{bmatrix}. $$ The row sums of $Q$ are $\tfrac12+\tfrac14=\tfrac34$ and $0+\tfrac13=\tfrac13$. The row sums of $R$ are $\tfrac14+0=\tfrac14$ and $\tfrac13+\tfrac13=\tfrac23$. Row by row, the totals are $\tfrac34+\tfrac14=1$ and $\tfrac13+\tfrac23=1$.

Check: From state $1$, the probability of a transient destination is $p_{11}+p_{12}=\tfrac12+\tfrac14=\tfrac34$, and the probability of an absorbing destination is $p_{13}+p_{14}=\tfrac14+0=\tfrac14$. From state $2$, those probabilities are $p_{21}+p_{22}=0+\tfrac13=\tfrac13$ and $p_{23}+p_{24}=\tfrac13+\tfrac13=\tfrac23$. The two sums match the row sums of $Q$ and $R$.

Sometimes the original state order is not canonical. Then first identify the two sets and reorder both rows and columns in the same way. Reordering only the rows would change which states the columns represent, so the resulting submatrices would not have the intended transition interpretation.

Worked Example: Reordering Before Extracting Blocks

Use the state order $a,b,c,d$, where $a,d$ are absorbing and $b,c$ are transient. Suppose the transition matrix in that order is $$ P=\begin{bmatrix} 1&0&0&0\\ \tfrac14&\tfrac12&\tfrac14&0\\ 0&\tfrac13&0&\tfrac23\\ 0&0&0&1 \end{bmatrix}. $$ Check each row before reordering: the row sums are $1$, $\tfrac14+\tfrac12+\tfrac14+0=1$, $0+\tfrac13+0+\tfrac23=1$, and $1$. The rows for $a$ and $d$ confirm that these states are absorbing.

Choose the canonical order $b,c,a,d$, putting the transient states first and the absorbing states last. Reordering the rows and columns together gives $$ P_{\mathrm{can}}=\begin{bmatrix} \tfrac12&\tfrac14&\tfrac14&0\\ \tfrac13&0&0&\tfrac23\\ 0&0&1&0\\ 0&0&0&1 \end{bmatrix}. $$ The reordered row sums are $\tfrac12+\tfrac14+\tfrac14+0=1$, $\tfrac13+0+0+\tfrac23=1$, $1$, and $1$. Therefore, $$ Q=\begin{bmatrix}\tfrac12&\tfrac14\\\tfrac13&0\end{bmatrix}, \qquad R=\begin{bmatrix}\tfrac14&0\\0&\tfrac23\end{bmatrix}. $$ The row sums are $\tfrac34,\tfrac13$ for $Q$, and $\tfrac14,\tfrac23$ for $R$. Adding them in corresponding rows gives $\tfrac34+\tfrac14=1$ and $\tfrac13+\tfrac23=1$.

Check: From $b$, the transient destinations $b,c$ have probabilities $\tfrac12,\tfrac14$, and the absorbing destinations $a,d$ have probabilities $\tfrac14,0$. From $c$, the transient destinations $b,c$ have probabilities $\tfrac13,0$, and the absorbing destinations $a,d$ have probabilities $0,\tfrac23$. These entries reproduce both extracted blocks, in the stated canonical order.

What the Extremes of a Row Mean

The same row-sum check helps catch errors in cases where one block row contains only zeros or where all probability is assigned to one destination type. These are valid possibilities. A zero row of $Q$ means that the associated transient state moves into $A$ with probability $1$ on its next step; it does not mean the full transition row is invalid. Similarly, a zero row of $R$ means no absorbing state can be reached in one step from that state; the transition row is still valid if the $Q$ row sums to $1$.

Worked Example: A Transient State That Absorbs in One Step

Let the transient states be $x,y,z$, the absorbing states be $u,v$, and use the canonical order $x,y,z,u,v$. Take $$ P=\begin{bmatrix} \tfrac12&\tfrac14&0&\tfrac14&0\\ 0&0&\tfrac12&0&\tfrac12\\ 0&0&0&\tfrac13&\tfrac23\\ 0&0&0&1&0\\ 0&0&0&0&1 \end{bmatrix}. $$ The five row sums are, in order, $\tfrac12+\tfrac14+0+\tfrac14+0=1$, $0+0+\tfrac12+0+\tfrac12=1$, $0+0+0+\tfrac13+\tfrac23=1$, $1$, and $1$. The last two rows are absorbing-state rows, as required.

Taking the first three rows and columns for $Q$, and the first three rows with the last two columns for $R$, gives $$ Q=\begin{bmatrix} \tfrac12&\tfrac14&0\\ 0&0&\tfrac12\\ 0&0&0 \end{bmatrix}, \qquad R=\begin{bmatrix} \tfrac14&0\\ 0&\tfrac12\\ \tfrac13&\tfrac23 \end{bmatrix}. $$ The row sums of $Q$ are $\tfrac12+\tfrac14+0=\tfrac34$, $0+0+\tfrac12=\tfrac12$, and $0+0+0=0$. The row sums of $R$ are $\tfrac14+0=\tfrac14$, $0+\tfrac12=\tfrac12$, and $\tfrac13+\tfrac23=1$. Thus the combined totals are $\tfrac34+\tfrac14=1$, $\tfrac12+\tfrac12=1$, and $0+1=1$.

Check: From $z$, there are no transient destinations in the displayed row, and the absorbing destinations have probabilities $\tfrac13$ and $\tfrac23$. Hence the zero row sum of $Q$ and unit row sum of $R$ correctly describe a one-step move into the absorbing set. From $x$, the self-transition of probability $\tfrac12$ is included in $Q$, and the remaining transient destination has probability $\tfrac14$.

A Common Extraction Error

A common mistake is to treat a transition back to the current transient state as though it were not a transient-to-transient transition. That incorrectly removes a diagonal entry of $Q$. In the first example, state $1$ has a self-transition of probability $\tfrac12$, a transition to transient state $2$ of probability $\tfrac14$, and a transition to absorbing state $3$ of probability $\tfrac14$. The correct first row of $Q$ is $(\tfrac12,\tfrac14)$, with row sum $\tfrac34$, and the first row of $R$ is $(\tfrac14,0)$, with row sum $\tfrac14$.

The incorrect calculation that drops the self-transition would use the row $(0,\tfrac14)$ for $Q$, giving row sum $\tfrac14$. Adding the correct $R$ row sum $\tfrac14$ would then give $\tfrac12$, not $1$. The missing $\tfrac12$ identifies the omitted transition. The correction is to include every destination in $T$, including the starting state itself, in $Q$. A reliable procedure is to check the original transition rows, list the transient and absorbing states, reorder rows and columns together, extract the blocks, and finally verify the two row sums for each transient state.

Check Your Understanding

For each question, keep the state order explicit and check the relevant row sums before interpreting the blocks.

A chain has transient states $1,2$ and absorbing state $3$. Its rows in canonical order are $(\tfrac12,\tfrac14,\tfrac14)$, $(0,0,1)$, and $(0,0,1)$. Extract $Q$ and $R$, then give their row sums.
In a canonical matrix, what does a zero row sum of $Q$ mean for the corresponding transient state? What must the corresponding row sum of $R$ be?
Suppose the original order is $a,b,c$, with $a,c$ absorbing and $b$ transient. Which order puts the matrix in canonical form, and why must both rows and columns be reordered?
A transient-state row has $Q$-block entries $(\tfrac13,\tfrac12)$. What must the row sum of the corresponding $R$ block be?
Explain why a self-transition of a transient state belongs in $Q$, not $R$.