Recognizing Transience Through Escape
A state is transient when a chain started there has a positive probability of never returning. In examples, that event may be visible: the chain can reach an absorbing state, or it can keep moving outward forever. The main task is to show that the escape event really has positive probability. A finite route requires only multiplying transition probabilities; an infinite route requires an additional argument that its probability does not shrink to zero.
Throughout, let \((X_n)_{n\ge0}\) be a discrete-time Markov chain on a finite or countable state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with transition probabilities \(p_{ij}\). Each row of the transition kernel sums to \(1\). Under \(\mathbb{P}_i\), the chain starts at \(i\). The first return time to \(i\) is \(T_i^+=\inf\{n\ge1:X_n=i\}\), with \(\inf\varnothing=\infty\). Write \(f_{ii}=\mathbb{P}_i(T_i^+<\infty)\). A state is transient if \(f_{ii}<1\), and recurrent if \(f_{ii}=1\). Thus, to prove transience, it is enough to exhibit an event of positive probability on which the chain never returns to its starting state.
Proof. By the Markov property, the probability under \(\mathbb{P}_i\) of following this particular path for the first \(m\) steps is \(\prod_{r=1}^{m}p_{x_{r-1}x_r}>0\). On that event the chain does not visit \(i\) at times \(1,\ldots,m\). It is at \(a\) at time \(m\), and because \(a\) is absorbing and differs from \(i\), it never visits \(i\) afterwards either. Consequently \(\mathbb{P}_i(T_i^+=\infty)>0\), so \(f_{ii}<1\) and \(i\) is transient. \(\square\)
The path condition is sufficient, not necessary: a state can be transient even when escape does not lead to an absorbing state. For an infinite outward path, it is not enough to say that every step has positive probability. The probability of following the first \(m\) steps might tend to zero as \(m\) grows. The next result gives a useful condition that rules this out.
Proof. Let \(E_m\) be the event that the chain follows \(x_0,x_1,\ldots,x_m\) during its first \(m\) transitions. The Markov property gives \(\mathbb{P}_i(E_m)=\prod_{r=1}^{m}(1-a_r)\). For any finite list \(b_1,\ldots,b_m\) with \(0\le b_r\le1\), \(\prod_{r=1}^{m}(1-b_r)\ge1-\sum_{r=1}^{m}b_r\). This follows by induction: if the inequality holds for \(m\), multiplying by \(1-b_{m+1}\ge0\) gives a lower bound \((1-\sum_{r=1}^{m}b_r)(1-b_{m+1})\); alternatively, when the partial sum is at most \(1\), this equals \(1-\sum_{r=1}^{m+1}b_r+b_{m+1}\sum_{r=1}^{m}b_r\), which is at least \(1-\sum_{r=1}^{m+1}b_r\). If a partial sum exceeds \(1\), the desired lower bound is nonpositive and holds because the product is nonnegative. Applying the inequality with \(b_r=a_r\) yields \(\mathbb{P}_i(E_m)\ge1-\sum_{r=1}^{m}a_r\ge1-\sum_{r=1}^{\infty}a_r>0\). The events \(E_m\) decrease as \(m\) increases. By continuity of probability for decreasing events, \(\mathbb{P}_i(\bigcap_{m\ge1}E_m)=\lim_{m\to\infty}\mathbb{P}_i(E_m)\ge1-\sum_{r=1}^{\infty}a_r>0\). On their intersection, the chain visits only distinct states after time zero, so it never returns to \(i\). Therefore \(\mathbb{P}_i(T_i^+=\infty)>0\), and \(i\) is transient. \(\square\)
Worked Examples
Worked Example: Escape to an Absorbing State
Consider a chain on \(S=\{0,1,a\}\), where \(a\) is absorbing. From \(0\), the chain stays at \(0\) with probability \(\tfrac12\) and moves to \(1\) with probability \(\tfrac12\); from \(1\), it moves to \(a\) with probability \(1\). Classify state \(0\).
In the state order \(0,1,a\), the transition matrix is
The row sums are \(\tfrac12+\tfrac12+0=1\), \(0+0+1=1\), and \(0+0+1=1\), so each row is a probability distribution. A first return to \(0\) occurs at time \(1\) exactly when the first transition is \(0\to0\), which has probability \(\tfrac12\). If the first transition is \(0\to1\), the next transition is \(1\to a\), after which the chain stays at \(a\); there is no return to \(0\). Thus \(f_{00}=\tfrac12<1\), and state \(0\) is transient.
Check by an escape path: the path \(0\to1\to a\) has probability \(\tfrac12\cdot1=\tfrac12>0\). It avoids \(0\) after time zero and reaches an absorbing state, as required by the finite escape-path criterion.
Worked Example: Adding All Possible First Returns
Consider \(S=\{0,1,2,a\}\), with transitions \(0\to0,1,a\) each having probability \(\tfrac13\); transitions \(1\to0,2\) each having probability \(\tfrac12\); transition \(2\to a\) having probability \(1\); and \(a\) absorbing. Find the probability that a chain started at \(0\) ever returns to \(0\).
The rows of the transition matrix, in the state order \(0,1,2,a\), are \((\tfrac13,\tfrac13,0,\tfrac13)\), \((\tfrac12,0,\tfrac12,0)\), \((0,0,0,1)\), and \((0,0,0,1)\). Their sums are respectively \(\tfrac13+\tfrac13+0+\tfrac13=1\), \(\tfrac12+0+\tfrac12+0=1\), \(0+0+0+1=1\), and \(0+0+0+1=1\).
A return at time \(1\) occurs on the transition \(0\to0\), with probability \(\tfrac13\). A return at time \(2\) occurs along \(0\to1\to0\), with probability \(\tfrac13\cdot\tfrac12=\tfrac16\). These are the only possible return routes: from \(0\), a move to \(a\) prevents a return, and the route \(0\to1\to2\to a\) also prevents one. Therefore
Since \(f_{00}=\tfrac12<1\), state \(0\) is transient. Check by first-step conditioning: starting from \(1\), the chain returns to \(0\) with probability \(\tfrac12\), by moving directly there; otherwise it moves to \(2\), then to \(a\), and cannot return. Hence \(f_{00}=\tfrac13+\tfrac13\cdot\tfrac12=\tfrac12\), agreeing with the route enumeration.
Worked Example: A Countable Chain That Can Move Outward Forever
Let \(S=\{0,1,2,\ldots\}\). From state \(0\), the chain moves to \(1\) with probability \(1\). For each \(n\ge1\), it moves from \(n\) to \(n+1\) with probability \(1-2^{-(n+2)}\), and from \(n\) to \(0\) with probability \(2^{-(n+2)}\). Show that \(0\) is transient.
At state \(0\), the only transition probability is \(p_{01}=1\), so the row sums to \(1\). At each \(n\ge1\), the two probabilities sum to \((1-2^{-(n+2)})+2^{-(n+2)}=1\). Starting at \(0\), the chain first moves to \(1\). If it then moves from \(1\) to \(2\), from \(2\) to \(3\), and continues outward at every step, it never returns to \(0\).
For \(m\ge1\), the probability of making all the outward moves from states \(1,\ldots,m\) is \(\prod_{n=1}^{m}(1-2^{-(n+2)})\). The sum of the corresponding failure probabilities is
The finite-product inequality proved above now gives \(\prod_{n=1}^{m}(1-2^{-(n+2)})\ge1-\sum_{n=1}^{m}2^{-(n+2)}\ge1-\tfrac14=\tfrac34\) for every \(m\). The events of making the first \(m\) outward moves decrease with \(m\), so the probability of making every outward move is their limiting probability, at least \(\tfrac34\). This event is contained in \(\{T_0^+=\infty\}\), and therefore \(\mathbb{P}_0(T_0^+=\infty)\ge\tfrac34>0\). It follows that \(f_{00}\le\tfrac14<1\), so state \(0\) is transient.
Check: the bound uses the full sum of the failure probabilities, \(\tfrac14\), not just the first one, \(\tfrac18\). It establishes a lower bound of \(\tfrac34\) for the probability of outward motion forever; it does not claim that this probability is exactly \(\tfrac34\).
What Escape Arguments Do—and Do Not—Show
These examples use two distinct ways to certify a positive probability of no return. In a finite escape-path argument, one particular finite sequence of transitions reaches a state from which return is impossible. Its probability is a finite product of positive transition probabilities. In an outward-path argument, the chain must keep making a sequence of transitions forever; the finite products must have a positive lower bound before continuity of probability can be used.
A common mistake is to count only returns on the first step. In the second example, that calculation would give \(f_{00}=\tfrac13\), because \(p_{00}=\tfrac13\). It is wrong: the route \(0\to1\to0\) also returns, at time \(2\), with probability \(\tfrac13\cdot\tfrac12=\tfrac16\). The correct calculation adds the probabilities of the disjoint first-return events, giving \(f_{00}=\tfrac13+\tfrac16=\tfrac12\). To establish transience, it is enough to show this total is strictly below \(1\); omitting possible return routes can make that conclusion invalid.
Finally, a positive chance of escaping from one particular state proves that this state is transient; it does not, by itself, classify every state in the chain. Nor does the presence of an absorbing state imply that every other state is transient: a chain might have other routes or classes with different return behavior. The argument must be tied to the state being classified and to the event of avoiding its first return.
Check Your Understanding
For each question, identify the return or escape event before calculating.
- In the first example, what is the probability of no return to \(0\), and which finite path certifies that it is positive?
- In the second example, list the possible first-return times and calculate the probability for each.
- In the outward-path example, why does a positive probability for each finite number of outward moves not alone prove a positive probability of moving outward forever?
- Suppose a chain started at \(i\) has a finite path of positive probability to an absorbing state \(a\ne i\), with no intermediate visit to \(i\). State precisely what this implies about \(f_{ii}\).
- Why is it incorrect to identify \(f_{ii}\) with \(p_{ii}\) whenever a return can occur after more than one transition?