Stationarity and Transience
A transient state can be visited, and it can even receive transitions from other states. What it cannot do is carry positive probability in a stationary distribution. The reason is not that a transient state has no incoming transitions. Rather, under stationarity, positive mass at that state would produce a positive probability of a visit with no later return. Shifting that event forward in time would give infinitely many disjoint events of the same positive probability, which is impossible.
Let \((X_n)_{n\ge0}\) be a discrete-time Markov chain on a finite or countable state space \(S\), with transition matrix \(P=(p_{ij})\) and probability space \((\Omega,\mathcal{F},\mathbb{P})\). A probability distribution \(\pi=(\pi_i)_{i\in S}\), written as a row vector, is stationary if \(\pi P=\pi\). Thus, if \(X_0\) has distribution \(\pi\), every \(X_n\) has distribution \(\pi\). For a state \(i\), define the first return time \(T_i^+=\inf\{n\ge1:X_n=i\}\). The state \(i\) is transient when \(\mathbb{P}_i(T_i^+<\infty)<1\); equivalently, its probability of never returning after time zero is positive. These are the standard recurrence and transience definitions from the earlier material on recurrence classification.
Why the Theorem Holds
Start the chain with distribution \(\pi\). Its path is stationary in time: shifting all the time indices by the same nonnegative integer leaves every finite-dimensional distribution unchanged. To see this, the distribution at time \(n\) is \(\pi P^n=\pi\), and the Markov property gives the same transition probabilities after any time shift. We use this pathwise stationarity to compare events that a visit to \(i\) is the final visit.
Proof. Fix a state \(i\), and let \(\mathbb{P}_\pi\) denote the law of the chain started with distribution \(\pi\). For each \(n\ge0\), define \(B_n=\{X_n=i,\ X_m\ne i\text{ for every }m>n\}\). The events \(B_n\) are pairwise disjoint: if \(n<r\), then \(B_n\) requires \(X_r\ne i\), whereas \(B_r\) requires \(X_r=i\). By stationarity, \(\mathbb{P}_\pi(B_n)=\mathbb{P}_\pi(B_0)\). This equality also holds for the events involving infinitely many future times: write each event as the decreasing intersection of the events that exclude \(i\) only through time \(n+\ell\), use stationarity for each finite \(\ell\), and take the limit as \(\ell\) increases. Since the \(B_n\) are disjoint, for every positive integer \(N\), \(N\mathbb{P}_\pi(B_0)=\sum_{n=0}^{N-1}\mathbb{P}_\pi(B_n)\le1\). Letting \(N\) increase gives \(\mathbb{P}_\pi(B_0)=0\). On the other hand, conditioning on \(X_0\) and using the Markov property, \(\mathbb{P}_\pi(B_0)=\mathbb{P}_\pi(X_0=i)\mathbb{P}_i(T_i^+=\infty)=\pi_i\mathbb{P}_i(T_i^+=\infty)\). If \(i\) is transient, the second factor is strictly positive. Therefore \(\pi_i=0\). \(\square\)
The proof uses a probability distribution on paths with stationary time marginals. It does not require a finite state space: countability ensures the usual transition-kernel description, while the argument itself concerns one state at a time. The key step is the disjointness of the final-visit events, not an assumption that transient states have no incoming transitions.
Proof. Let \(A\subseteq S\) be the set of transient states. The theorem gives \(\pi_i=0\) for each \(i\in A\). Since \(A\) is finite, \(\sum_{i\in A}\pi_i=0\). \(\square\)
Worked Examples
Worked Example: A Transient State with an Incoming Transition
Consider the chain on \(S=\{a,b,c\}\) with transition matrix \(P=\begin{bmatrix}0&0&1\\1&0&0\\0&0&1\end{bmatrix}\). Every row sums to \(1\): the row sums are \(0+0+1=1\), \(1+0+0=1\), and \(0+0+1=1\). State \(b\) moves to \(a\), and \(a\) moves to \(c\), which is absorbing. Find all stationary probability distributions.
From \(a\) or \(b\), the chain cannot return to its starting state, so both states are transient. State \(a\) does have an incoming transition, from \(b\). Write \(\pi=(x_a,x_b,x_c)\). The equation \(\pi P=\pi\), coordinate by coordinate, is
The second equation gives \(x_b=0\), and then the first gives \(x_a=0\). Normalization \(x_a+x_b+x_c=1\) yields \(x_c=1\), so \(\pi=(0,0,1)\). Substitution checks the answer: \((0,0,1)P=(0,0,1)\), and its entries sum to \(1\).
This calculation also shows why incoming transitions do not contradict the theorem. Although the stationary equation for \(a\) includes incoming mass from \(b\), stationarity forces \(x_b=0\), which in turn forces \(x_a=0\).
Worked Example: A Finite Chain with Two Closed Classes
Let \(S=\{1,2,3,4\}\), with transition matrix \(P=\begin{bmatrix}0&\tfrac12&\tfrac12&0\\0&\tfrac12&\tfrac12&0\\0&\tfrac14&\tfrac34&0\\0&0&0&1\end{bmatrix}\). The row sums are \(0+\tfrac12+\tfrac12+0=1\), \(0+\tfrac12+\tfrac12+0=1\), \(0+\tfrac14+\tfrac34+0=1\), and \(0+0+0+1=1\). State \(1\) is transient because after leaving it the chain cannot return. Find every stationary probability distribution.
Write \(\pi=(x_1,x_2,x_3,x_4)\). Equating the four coordinates of \(\pi P\) and \(\pi\) gives
The first equation sets \(x_1=0\). The second then gives \(\tfrac12x_2=\tfrac14x_3\), so \(x_3=2x_2\). The third equation is then satisfied: its right-hand side is \(\tfrac12x_2+\tfrac34(2x_2)=2x_2=x_3\). Normalization gives \(x_2+x_3+x_4=1\), or \(3x_2+x_4=1\). Set \(\alpha=x_2+x_3=3x_2\). Nonnegativity is equivalent to \(0\le\alpha\le1\), and the stationary distributions are exactly \(\pi=(0,\tfrac{\alpha}{3},\tfrac{2\alpha}{3},1-\alpha)\) for \(0\le\alpha\le1\).
Check: The entries sum to \(0+\tfrac{\alpha}{3}+\tfrac{2\alpha}{3}+1-\alpha=1\). In \(\pi P\), the second coordinate is \(\tfrac12(\tfrac{\alpha}{3})+\tfrac14(\tfrac{2\alpha}{3})=\tfrac{\alpha}{3}\), the third is \(\tfrac12(\tfrac{\alpha}{3})+\tfrac34(\tfrac{2\alpha}{3})=\tfrac{2\alpha}{3}\), and the fourth is \(1-\alpha\); the first is zero. Thus \(\pi P=\pi\). The transient coordinate is zero for every choice of \(\alpha\), even though the chain may start at state \(1\).
Worked Example: Deterministic Descent on a Countable State Space
Take \(S=\{0,1,2,\ldots\}\). From state \(0\), the chain stays at \(0\); from each \(k\ge1\), it moves to \(k-1\) with probability \(1\). Each transition row has exactly one entry equal to \(1\) and all other entries equal to \(0\), so every row sums to \(1\). Every state \(k\ge1\) is transient: after exactly \(k\) steps the chain reaches \(0\), and it never returns to \(k\). Find the stationary probability distribution.
For any probability distribution \(\pi\), the stationary equation for state \(0\) is \(\pi_0=\pi_0+\pi_1\), since both \(0\) and \(1\) transition to \(0\). Hence \(\pi_1=0\). For each \(k\ge1\), the only state that transitions to \(k\) is \(k+1\), so stationarity requires \(\pi_k=\pi_{k+1}\). Starting with \(\pi_1=0\), this gives \(\pi_2=0\), then \(\pi_3=0\), and inductively \(\pi_k=0\) for every \(k\ge1\). Normalization forces \(\pi_0=1\).
Check: The distribution concentrated at \(0\) is stationary because state \(0\) stays at \(0\), so \(\pi P=\pi\). Its total mass is \(1\), and it assigns zero mass to every transient state, as the theorem requires.
What the Result Does—and Does Not—Say
The theorem constrains stationary probability, not the chain’s behavior from an arbitrary initial distribution. For example, in the finite example with states \(1,2,3,4\), starting at state \(1\) gives positive probability of being at \(1\) at time zero. That initial law is not stationary. The theorem says that if the initial law is stationary, then its mass at every transient state must already be zero. It does not say that a chain started elsewhere has zero probability of occupying a transient state at each finite time.
Nor does the theorem assert that every nontransient state must receive positive mass under every stationary distribution. The four-state example has two closed parts, and the stationary law can place all its mass on either one, or divide mass between them. The theorem gives a necessary restriction: stationary probability is zero on transient states. It is not, by itself, a complete method for constructing or classifying stationary distributions.
Common mistake: treating a stationary equation as if it involved only self-transitions. In the three-state example, an incorrect calculation might write \(x_a=x_a p_{aa}=0\) and claim this is the stationary equation for \(a\). That omits the incoming transition from \(b\). The correct balance equation is \(x_a=x_b p_{ba}+x_a p_{aa}=x_b\). The equation for \(b\) then gives \(x_b=0\), and only then does one conclude \(x_a=0\). In general, the stationary equation at \(i\) is \(\pi_i=\sum_{j\in S}\pi_jp_{ji}\), including all incoming transitions.
The practical use is immediate in finite chains: identify transient states first, set their stationary probabilities to zero, and solve the remaining balance and normalization equations. This can reduce the number of unknowns. On a countable chain, the same zero-mass conclusion holds state by state, even when the stationary equations involve infinitely many states.
Check Your Understanding
Use the return-time definition of transience and the full incoming balance equations when answering.
- Why are the events that time \(n\) is the final visit to a fixed state pairwise disjoint?
- If \(\pi\) is stationary and \(\mathbb{P}_i(T_i^+=\infty)=\tfrac13\), what does the theorem imply about \(\pi_i\)?
- In the three-state example, which state sends a transition into the transient state \(a\), and which stationary equation forces its mass to vanish?
- For the deterministic-descent chain, write the stationary equation for a state \(k\ge1\). What does it imply once \(\pi_1=0\)?
- Does the theorem say that a chain started from a nonstationary distribution has zero probability of being in a transient state at time zero? Explain.