Recognizing Absorbing States and Absorbing Chains

Absorbing States and the Transition Graph

Some Markov chains have states that, once entered, cannot be left. These states model outcomes such as completion, failure, or termination. But finding one such state does not by itself show that the chain will eventually enter it: other states may form a separate class that the chain cannot leave. The transition graph makes this distinction visible.

Let \((X_n)_{n\ge0}\) be a discrete-time Markov chain on a finite state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with transition matrix \(P=(p_{ij})\). Thus \(p_{ij}=\mathbb{P}(X_{n+1}=j\mid X_n=i)\), and each row sums to \(1\). We use the natural filtration \(\mathcal{F}_n=\sigma(X_0,\ldots,X_n)\). The transition graph has one vertex for each state and a directed edge \(i\to j\) exactly when \(p_{ij}>0\). A directed path may have length zero, so every state is reachable from itself.

The definition of an absorbing state is a condition on its whole row, not just its diagonal entry being positive. A self-loop with probability less than \(1\) means the chain can stay for a step, but it can also leave. Also, a chain with an absorbing state need not be an absorbing chain: the latter requires reachability from every state.

For a set \(A\subseteq S\), write \(T_A=\inf\{n\ge0:X_n\in A\}\), with \(\inf\varnothing=\infty\). If \(A\) is the set of all absorbing states, then reaching \(A\) means that the chain has been absorbed. The following result explains why graph reachability is the right finite-state test.

Proof. First suppose that every state can reach \(A\). For each state \(i\notin A\), choose a path from \(i\) to \(A\), and let its length be \(n_i\ge1\). The probability \(c_i\) of following that path is the product of its transition probabilities, so \(c_i>0\). If \(i\in A\), set \(n_i=1\) and \(c_i=1\), since the chain stays in \(A\). Because \(S\) is finite, \(N=\max_{i\in S} n_i\) is finite and \(\varepsilon=\min_{i\in S}c_i>0\). Starting from any state, absorption therefore has probability at least \(\varepsilon\) within the next \(N\) steps. By the Markov property, conditional on \(\mathcal{F}_{kN}\), if absorption has not occurred by time \(kN\), the probability of avoiding \(A\) for the next \(N\) steps is at most \(1-\varepsilon\). Hence, for every integer \(k\ge0\),

The last bound tends to \(0\) as \(k\to\infty\), so the probability of never reaching \(A\) is \(0\). For the converse, the event \(T_A<\infty\) is the union over finite \(n\ge0\) of the events \(T_A=n\). If this union has positive probability under \(\mathbb{P}_i\), at least one of its events has positive probability. That event requires a finite sequence of transitions from \(i\) to a state in \(A\); positive probability means each transition in that sequence has positive probability, giving a directed path. This proves both claims and the stated characterization. \(\square\)

Transient States in an Absorbing Chain

A state is transient if, starting there, the probability of returning to it at a strictly later time is less than \(1\). In the notation of the course, this means \(\mathbb{P}_i(T_i^+<\infty)<1\), where \(T_i^+=\inf\{n\ge1:X_n=i\}\). An absorbing state is not transient: starting there, the chain returns at the very next step with probability \(1\).

In a finite absorbing chain, every nonabsorbing state is transient. To see why, take a nonabsorbing state \(i\) and a shortest path from \(i\) to an absorbing state. The path cannot visit \(i\) again after it leaves: removing the segment between two visits to \(i\) would produce a shorter path. Following this path has positive probability and reaches absorption without returning to \(i\). Thus the probability of returning to \(i\) is strictly less than \(1\). This classification applies when all states can reach absorption; outside that setting, a nonabsorbing state need not be transient.

A Reliable Classification Procedure

For a finite transition matrix or diagram, use the following sequence. Check that the rows sum to \(1\) (or, in a diagram, that the outgoing probabilities from each vertex sum to \(1\)). Identify absorbing states by checking for a self-transition of probability \(1\), with no other positive entries in that row. Then trace directed paths from every state to the set of absorbing states. If at least one absorbing state exists and every state has such a path, the chain is absorbing and every nonabsorbing state is transient. If some state cannot reach an absorbing state, the chain is not absorbing, even if other states are absorbed. In that case, inspect the states outside the absorbing outcomes separately: as the second example shows, they may form a recurrent class.

Common mistake. It is incorrect to see an absorbing state and conclude that the whole chain is absorbing. In the second example, state \(c\) is absorbing since \(p_{cc}=1\), but the path from \(a\) alternates \(a\to b\to a\to b\to\cdots\); there is no directed path from \(a\) to \(c\). The correction is to test reachability from every state, not merely to locate one absorbing row. A related error is to label any state with a self-loop absorbing: in the first example \(p_{11}=\tfrac13\), but state \(1\) is not absorbing because \(p_{10}=\tfrac13>0\) and \(p_{12}=\tfrac13>0\).

These distinctions matter when a model represents a process that is expected to terminate. A transition matrix may include a genuine termination state while also allowing a class of states that never reaches it. The graph test detects that structural possibility before one attempts to quantify absorption probabilities or times. It also prevents a common classification error: the transient-state conclusion for nonabsorbing states is justified for a finite absorbing chain, but cannot be carried over to a chain with a closed nonabsorbing class.