Uniqueness from Regeneration at a Fixed State

Why a Fixed-State Cycle Determines the Stationary Law

A stationary distribution describes the long-run balance of probability across states, but its balance equations can be difficult to solve directly, especially on a countable state space. A useful alternative is to choose one state and divide the chain’s path into cycles from that state back to itself. The expected number of visits to each state in one such cycle determines the stationary probabilities up to a common normalizing constant. We will show that every stationary distribution must have exactly this form, so there can be no two different stationary distributions.

Let \((X_n)_{n\ge0}\) be a time-homogeneous Markov chain on a finite or countable state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with transition matrix \(P=(p_{ij})\). Its rows sum to \(1\). When the chain starts at \(i\), write \(\mathbb{P}_i\) and \(\mathbb{E}_i\). Use the natural filtration \(\mathcal{F}_n=\sigma(X_0,\ldots,X_n)\). Fix a state \(a\in S\), and define its first return time by \(T_a^+=\inf\{n\ge1:X_n=a\}\). The return cycle includes times \(0,\ldots,T_a^+-1\), but not the endpoint \(T_a^+\).

Assume throughout that the chain is irreducible and positive recurrent. Irreducibility means every state can reach every other state with positive probability in some finite number of steps. Positive recurrence means the mean return time is finite; in an irreducible chain, this holds at every state if it holds at one. Thus \(\mathbb{E}_a[T_a^+]<\infty\). A standard consequence of irreducibility and recurrence is that \(\mathbb{P}_i(T_a<\infty)=1\) for every state \(i\). This return-time fact is part of the earlier material on positive recurrence. A stationary distribution is a probability row vector \(\pi\) satisfying \(\pi P=\pi\).

The Cycle Occupation Measure Is Invariant

Proof. First, all terms in the occupation sum are nonnegative, so summing over states and taking expectation can be interchanged by monotone convergence. At each time \(n<T_a^+\), exactly one state is occupied. Therefore \(\sum_{j\in S}\nu_j=\mathbb{E}_a[\sum_{n=0}^{T_a^+-1}1]=\mathbb{E}_a[T_a^+]<\infty\). For a fixed state \(j\), the Markov property gives \(\sum_i\nu_i p_{ij}=\mathbb{E}_a[\sum_{n=0}^{T_a^+-1}p_{X_nj}]\). The event \(\{n<T_a^+\}\) is determined by \(X_0,\ldots,X_n\), so conditioning on \(\mathcal{F}_n\) replaces \(p_{X_nj}\) by \(\mathbf{1}_{\{X_{n+1}=j\}}\) inside this expectation. Hence the last expression equals \(\mathbb{E}_a[\sum_{n=0}^{T_a^+-1}\mathbf{1}_{\{X_{n+1}=j\}}]\), the expected number of visits to \(j\) at times \(1,\ldots,T_a^+\). The counts over times \(0,\ldots,T_a^+-1\) and \(1,\ldots,T_a^+\) differ only by removing \(X_0=a\) and adding \(X_{T_a^+}=a\). For every \(j\), these changes cancel, so the expected counts are equal. Thus \(\sum_i\nu_i p_{ij}=\nu_j\) for every \(j\), proving invariance. \(\square\)

The proposition constructs an invariant measure from one cycle. Since its total mass is finite, it can be normalized to a stationary distribution. The stronger point needed for uniqueness is that any stationary distribution is forced to be proportional to these same cycle counts.

Every Stationary Distribution Has the Same Cycle Ratios

Proof. Let \(B=S\setminus\{a\}\), let \(Q=(p_{ij})_{i,j\in B}\) be the transition matrix restricted to moves within \(B\), and let \(r=(p_{aj})_{j\in B}\) be the row of transition probabilities from \(a\) into \(B\). The rows of \(Q\) sum to at most \(1\), since some transitions may go to \(a\).

First, \(\pi_a>0\). Since \(\pi\) is a probability distribution, some state \(i\) has \(\pi_i>0\). By irreducibility there is an integer \(m\ge0\) with \(p_{ia}^{(m)}>0\). Stationarity iterated \(m\) times gives \(\pi=\pi P^m\), and thus \(\pi_a=\sum_k\pi_kp_{ka}^{(m)}\ge\pi_i p_{ia}^{(m)}>0\). This also covers \(i=a\) by taking \(m=0\).

For \(j\in B\), the stationary equation separates transitions from \(a\) and from \(B\): \(\pi_j=\pi_a r_j+\sum_{i\in B}\pi_iQ_{ij}\). Iterating this identity \(m\) times gives, in row-vector notation on \(B\), \(\pi_B=\pi_a r\sum_{k=0}^{m-1}Q^k+\pi_BQ^m\). All terms are nonnegative, so the iteration and the following sums are valid on a countable state space by monotone convergence.

The total mass of the residual row is \(\sum_{j\in B}(\pi_BQ^m)_j=\sum_{i\in B}\pi_i\sum_{j\in B}(Q^m)_{ij}=\sum_{i\in B}\pi_i\mathbb{P}_i(T_a>m)\). For each \(i\in B\), the row sum here is the probability that the chain avoids \(a\) for the next \(m\) steps. Since \(\mathbb{P}_i(T_a<\infty)=1\), it tends to \(0\) as \(m\to\infty\). Each summand is bounded by \(\pi_i\), and \(\sum_i\pi_i=1\), so dominated convergence shows that the residual total mass tends to \(0\). In particular, each coordinate of \(\pi_BQ^m\) tends to \(0\).

It follows that \(\pi_j=\pi_a\sum_{k\ge0}(rQ^k)_j\) for each \(j\in B\). The term \((rQ^k)_j\) is the probability that the chain, starting from \(a\), is at \(j\) at time \(k+1\) without having returned to \(a\) by that time. Summing over \(k\ge0\) therefore gives the expected number of visits to \(j\) during the return cycle: \(\nu_j=\sum_{k\ge0}(rQ^k)_j\). Thus \(\pi_j=\pi_a\nu_j\) for \(j\ne a\). At \(a\), the same identity holds because \(\nu_a=1\).

Finally, \(\pi\) has total mass \(1\), so \(1=\sum_j\pi_j=\pi_a\sum_j\nu_j\). The sum \(\sum_j\nu_j\) is finite and fixed by the chain and the chosen state \(a\), as shown in the proposition. Since \(\pi_a\) must be the unique constant that makes these cycle counts sum to \(1\), every stationary distribution has the same coordinates. \(\square\)

Worked Examples

A Common Counting Error

A cycle must exclude its return endpoint. In the two-state example, state \(0\) occurs once in the cycle, at time \(0\), so \(\nu_0=1\). If one mistakenly includes time \(T_0^+\), state \(0\) is counted twice and one gets the incorrect counts \((2,2)\), leading to the incorrect candidate \((\tfrac12,\tfrac12)\). Substitution exposes the error: \((\tfrac12,\tfrac12)P=(\tfrac38,\tfrac58)\ne(\tfrac12,\tfrac12)\). The correct cycle interval is \(0\le n<T_a^+\); the initial state is counted and the return state is not.

The fixed-state method is useful because it separates two tasks. The chain’s dynamics determine the relative expected visit counts \(\nu_j\), while the requirement that probabilities sum to \(1\) supplies the common scale. The proof of uniqueness then shows that no alternative stationary solution can have different relative weights. Neither the cycle argument nor uniqueness requires aperiodicity.