Stationary Laws When a Chain Has Several Closed Classes
A finite Markov chain need not have a unique stationary distribution. If it can remain forever in one of several separate closed parts of its state space, each part supports its own stationary law. Any weighted average of these laws is stationary as well. The central task is to identify the closed classes, find the stationary law on each, and determine which weights are allowed.
Let \((X_n)_{n\ge0}\) be a discrete-time Markov chain on a finite state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with row-stochastic transition matrix \(P=(p_{ij})\). Thus \(p_{ij}=\mathbb{P}(X_{n+1}=j\mid X_n=i)\), and each row sums to \(1\). We use the natural filtration \(\mathcal{F}_n=\sigma(X_0,\ldots,X_n)\). A probability row vector \(\mu\) is stationary if \(\mu P=\mu\); if \(X_0\) has distribution \(\mu\), then every \(X_n\) has distribution \(\mu\).
For a finite chain, at least one closed communicating class exists. Write the distinct closed classes as \(C_1,\ldots,C_m\). They are disjoint. Each restricted chain on \(C_r\) is finite and irreducible, so it has a stationary distribution, denoted \(\pi^{(r)}\). Existence of a stationary distribution for a finite chain is a standard result from the earlier material on stationary distributions. The next result describes all stationary laws of the full chain, not just examples of them.
We first justify the uniqueness fact about each class used in the theorem.
Proof. Let \(\pi\) and \(\nu\) be stationary probability row vectors for a finite irreducible chain. Both have strictly positive coordinates: for either vector, choose a state \(j\) with positive mass. For any state \(i\), irreducibility gives an integer \(n\ge0\) with \(p_{ji}^{(n)}>0\), so stationarity gives \(\pi_i=(\pi P^n)_i\ge\pi_jp_{ji}^{(n)}>0\), and the same argument applies to \(\nu\). Now set \(c=\min_i\nu_i/\pi_i\). The vector \(w=\nu-c\pi\) is nonnegative and stationary, and at least one of its coordinates is zero. If \(w\) were nonzero, it would have a positive coordinate \(w_j\). Irreducibility would give, for every \(i\), some \(n\) with \(p_{ji}^{(n)}>0\), and stationarity would imply \(w_i=(wP^n)_i\ge w_jp_{ji}^{(n)}>0\). This contradicts the zero coordinate. Thus \(w=0\), so \(\nu=c\pi\); because both vectors sum to \(1\), \(c=1\), and \(\nu=\pi\). \(\square\)
Proof of the theorem. First suppose \(\mu\) is stationary. Let \(T=S\setminus\bigcup_{r=1}^m C_r\). We show that \(\mu\) assigns no mass to \(T\). From every state in \(T\), some closed class is reachable by a finite path: in the finite directed graph of positive-probability transitions, following reachable classes eventually leads to a class with no outgoing edge to another class. For each \(i\in T\), choose such a path, of length \(n_i\), whose probability is positive. Since \(T\) is finite, there are an integer \(N\ge1\) and a number \(\delta>0\) such that from every \(i\in T\), the probability of entering a closed class within \(N\) steps is at least \(\delta\). (If \(T\) is empty, the desired zero-mass statement is immediate.) Once the chain enters a closed class, it cannot return to \(T\). Therefore, conditional on starting in \(T\), the probability of still being in \(T\) after \(kN\) steps is at most \((1-\delta)^k\). Starting outside \(T\) cannot lead into \(T\), because the other states lie in closed classes. Stationarity now gives, for every \(k\ge1\), $$ \mu(T)=\sum_{i\in T}\mu_i =\mathbb{P}_{\mu}(X_{kN}\in T) \le \mu(T)(1-\delta)^k. $$ If \(\mu(T)>0\), division by \(\mu(T)\) would give \(1\le(1-\delta)^k<1\), a contradiction. Hence \(\mu(T)=0\).
Set \(\alpha_r=\sum_{i\in C_r}\mu_i\). These numbers are nonnegative and sum to \(1\). If \(\alpha_r>0\), normalize the restriction of \(\mu\) to \(C_r\) by dividing by \(\alpha_r\). The stationarity equations on \(C_r\) show that this normalized vector is stationary for the restricted chain: no other closed class can enter \(C_r\), and \(\mu\) has zero mass on \(T\). By uniqueness on the irreducible class, it is \(\pi^{(r)}\). If \(\alpha_r=0\), the restriction is zero. Thus \(\mu=\sum_r\alpha_r\pi^{(r)}\). Conversely, each \(\pi^{(r)}\), extended by zero, is stationary for the full matrix because \(C_r\) is closed. Linearity gives \((\sum_r\alpha_r\pi^{(r)})P=\sum_r\alpha_r\pi^{(r)}\), so every stated mixture is stationary. Finally, the classes are disjoint and \(\pi^{(r)}\) has total mass \(1\) on \(C_r\) and \(0\) on the other closed classes. Summing \(\mu\) over \(C_r\) therefore recovers \(\alpha_r\), proving uniqueness of the weights. \(\square\)
Finding the Component Laws and Weights
The theorem separates the calculation into two parts. First solve the stationary equations inside each closed class, using the restricted transition matrix. Then choose nonnegative weights summing to \(1\). There is no additional stationarity equation that fixes those weights: a chain already in one closed class cannot move to another. If the chain has just one closed class, the mixture has only one possible weight and the stationary distribution is unique.
Worked Example: Two Closed Classes and a Free Weight
Consider \(S=\{1,2,3\}\) and $$ P=\begin{bmatrix} 0&1&0\\ 1&0&0\\ 0&0&1 \end{bmatrix}. $$ Each row sums to \(1\): the row sums are \(0+1+0=1\), \(1+0+0=1\), and \(0+0+1=1\). The closed classes are \(C_1=\{1,2\}\) and \(C_2=\{3\}\). On \(C_1\), the chain alternates between its two states, so its stationary law is \((\tfrac12,\tfrac12)\); on \(C_2\), the stationary law is \((1)\).
The theorem gives the full family $$ \mu=\left(\frac{\alpha}{2},\frac{\alpha}{2},1-\alpha\right),\qquad 0\le\alpha\le1. $$ For example, if the mass assigned to \(C_1\) is \(\alpha=\tfrac25\), then \(\mu=(\tfrac15,\tfrac15,\tfrac35)\). Check the balance equations: \((\mu P)_1=\mu_2=\tfrac15\), \((\mu P)_2=\mu_1=\tfrac15\), and \((\mu P)_3=\mu_3=\tfrac35\). The coordinates sum to \(\tfrac15+\tfrac15+\tfrac35=1\), as required.
Worked Example: A Transient State Does Not Set the Mixture Weights
Let \(S=\{1,2,3,4\}\), with transition matrix $$ P=\begin{bmatrix} 0&\tfrac13&0&\tfrac23\\ 0&0&1&0\\ 0&1&0&0\\ 0&0&0&1 \end{bmatrix}. $$ The row sums are \(0+\tfrac13+0+\tfrac23=1\), \(0+0+1+0=1\), \(0+1+0+0=1\), and \(0+0+0+1=1\). The closed classes are \(C_1=\{2,3\}\) and \(C_2=\{4\}\); state \(1\) is outside them. The law on \(C_1\) is \((\tfrac12,\tfrac12)\), and the law on \(C_2\) puts mass \(1\) at state \(4\).
Writing \(\mu=(\mu_1,\mu_2,\mu_3,\mu_4)\), the first coordinate of \(\mu P=\mu\) gives \(\mu_1=0\). The second and third give \(\mu_2=\mu_3\) and \(\mu_3=\mu_2\); the fourth gives \(\mu_4=\tfrac23\mu_1+\mu_4\), which is satisfied when \(\mu_1=0\). Normalization leaves \(\mu_2+\mu_3+\mu_4=1\). Thus the stationary vectors are $$ \mu=\left(0,\frac{\alpha}{2},\frac{\alpha}{2},1-\alpha\right),\qquad 0\le\alpha\le1. $$ For \(\alpha=\tfrac35\), the vector is \((0,\tfrac3{10},\tfrac3{10},\tfrac25)\). Directly, \(\mu P=(0,\tfrac3{10},\tfrac3{10},\tfrac25)\), and its entries sum to \(\tfrac3{10}+\tfrac3{10}+\tfrac25=1\).
From state \(1\), the first step enters \(C_1\) with probability \(\tfrac13\) and \(C_2\) with probability \(\tfrac23\). Those are probabilities of entering the classes from that particular starting state; they do not force the weights of every stationary law. For instance, \((0,\tfrac12,\tfrac12,0)\) is also stationary. The weights are chosen by the stationary distribution being considered, not by the one-step split from state \(1\).
Worked Example: Computing Unequal Weights on Two Two-State Classes
Consider two closed classes, \(C_1=\{1,2\}\) and \(C_2=\{3,4\}\), with transition matrix $$ P=\begin{bmatrix} \tfrac34&\tfrac14&0&0\\ \tfrac12&\tfrac12&0&0\\ 0&0&\tfrac13&\tfrac23\\ 0&0&\tfrac14&\tfrac34 \end{bmatrix}. $$ The row sums are \(\tfrac34+\tfrac14=1\), \(\tfrac12+\tfrac12=1\), \(\tfrac13+\tfrac23=1\), and \(\tfrac14+\tfrac34=1\). For the first class, stationarity requires \(\pi_1^{(1)}\tfrac14=\pi_2^{(1)}\tfrac12\), so \(\pi_1^{(1)}=2\pi_2^{(1)}\); normalization gives \(\pi^{(1)}=(\tfrac23,\tfrac13,0,0)\). For the second class, \(\pi_3^{(2)}\tfrac23=\pi_4^{(2)}\tfrac14\), so \(\pi_3^{(2)}:\pi_4^{(2)}=3:8\), and \(\pi^{(2)}=(0,0,\tfrac3{11},\tfrac8{11})\).
Choose weights \(\alpha_1=\tfrac25\) and \(\alpha_2=\tfrac35\). The resulting law is $$ \mu=\frac25\pi^{(1)}+\frac35\pi^{(2)} =\left(\frac4{15},\frac2{15},\frac9{55},\frac{24}{55}\right). $$ Its entries sum to \(\tfrac4{15}+\tfrac2{15}+\tfrac9{55}+\tfrac{24}{55}=\tfrac25+\tfrac35=1\). To check stationarity, the first two output coordinates are \(\tfrac34\cdot\tfrac4{15}+\tfrac12\cdot\tfrac2{15}=\tfrac4{15}\) and \(\tfrac14\cdot\tfrac4{15}+\tfrac12\cdot\tfrac2{15}=\tfrac2{15}\). The last two are \(\tfrac13\cdot\tfrac9{55}+\tfrac14\cdot\tfrac{24}{55}=\tfrac9{55}\) and \(\tfrac23\cdot\tfrac9{55}+\tfrac34\cdot\tfrac{24}{55}=\tfrac{24}{55}\). Therefore \(\mu P=\mu\). Summing its coordinates within the classes recovers the weights: \(\tfrac4{15}+\tfrac2{15}=\tfrac25\) and \(\tfrac9{55}+\tfrac{24}{55}=\tfrac35\).
What the Mixture Description Does—and Does Not—Say
The mixture theorem describes stationary distributions, not necessarily the long-run distribution from every starting state. A chain can have several stationary distributions, and a particular initial distribution may determine which closed class it eventually enters. Periodicity can also affect whether the distribution at time \(n\) converges, even when a stationary distribution exists. The mixture characterization itself requires neither aperiodicity nor convergence in time: it is exactly a statement about solutions of \(\mu P=\mu\).
Common mistake. A stationary vector cannot put positive mass on a state that has no incoming transitions if that mass is not preserved by the transition step. In the second example, the candidate \(\mu=(\tfrac14,\tfrac38,\tfrac38,0)\) sums to \(1\) and gives equal mass to states \(2\) and \(3\), but it is not stationary. Its first output coordinate is \((\mu P)_1=0\), whereas \(\mu_1=\tfrac14\). The first balance equation requires \(\mu_1=0\). After setting it to zero, the remaining equations allow a full range of mixture weights; one must not infer a unique stationary law from the transient state's transition probabilities.
A reliable calculation therefore has three checks: each proposed class is closed; each component law is normalized and satisfies its class balance equations; and the mixture weights are nonnegative and sum to \(1\). The resulting full vector can then be checked directly by multiplying by \(P\).
Check Your Understanding
Use the closed classes and stationary equations to answer each question.
- A chain has two closed classes with stationary laws \((\tfrac12,\tfrac12,0)\) and \((0,0,1)\). Write the general stationary distribution and state how to recover its mixture weight from the coordinates.
- Why does a finite irreducible class have at most one stationary distribution?
- In the four-state example with a transient state, what are the stationary mixture weights for \((0,\tfrac14,\tfrac14,\tfrac12)\)? Verify that they are nonnegative and sum to \(1\).
- Explain why the probabilities of entering two classes from a particular transient state do not specify every stationary distribution.
- For the two-class matrix in the third example, check the first class balance equation for \((\tfrac23,\tfrac13)\) by comparing the mass moving from state \(1\) to state \(2\) with the mass moving from state \(2\) to state \(1\).