A Stationary Chain That Is Not Reversible

Stationarity Does Not Require Reversibility

A Markov chain can preserve a distribution even while its transitions have a preferred direction. A three-state cycle makes this distinction especially clear: probability mass is spread uniformly across the states at every time, but each transition moves in the same direction. Reversibility asks for more than preservation of a distribution; it asks whether the stationary chain looks the same when time is reversed.

Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Consider a discrete-time Markov chain \((X_n)_{n\ge0}\) with finite state space \(S\), transition matrix \(P=(p_{ij})\), and filtration \(\mathcal{F}_n=\sigma(X_0,\ldots,X_n)\). The matrix is row-stochastic: \(p_{ij}=\mathbb{P}(X_{n+1}=j\mid X_n=i)\), and each row sums to \(1\). Distributions are row vectors, so the distribution at time \(n+1\) is the distribution at time \(n\) multiplied on the right by \(P\).

Stationarity balances total probability entering and leaving each state. Detailed balance requires a stronger, pair-by-pair balance: the stationary probability flow from \(i\) to \(j\) must equal the flow from \(j\) to \(i\). The difference matters because stationarity can hold through an overall circulation around a cycle, without any pairwise balance.

A Uniform Stationary Distribution on a Directed Cycle

Use the state space \(S=\{0,1,2\}\), with transitions moving deterministically from \(0\) to \(1\), from \(1\) to \(2\), and from \(2\) to \(0\). The transition matrix is

Each row sums to \(1\): the three row sums are \(0+1+0=1\), \(0+0+1=1\), and \(1+0+0=1\). Each column also sums to \(1\). This column check is what makes the uniform distribution stationary.

Proof. Let \(m=|S|\), and let \(\pi\) be the row vector whose \(m\) entries are all \(1/m\). It is a probability distribution because its entries are nonnegative and sum to \(m(1/m)=1\). For each state \(j\), the \(j\)th entry of \(\pi P\) is \((\pi P)_j=\sum_{i\in S}(1/m)p_{ij}=(1/m)\sum_{i\in S}p_{ij}=1/m=\pi_j\), using the column-sum assumption. Thus \(\pi P=\pi\), as required. \(\square\)

The example demonstrates the distinction directly. Each time step moves the state, but if the initial state is uniform, every state is equally likely at every later time. Preservation of the distribution does not imply that the chain retraces its transitions with the same probabilities.

Computing the Reverse Chain

For a stationary chain whose distribution has positive entries, the one-step transition probability of the reversed chain from \(i\) to \(j\) is the conditional probability that the previous state was \(j\), given that the current state is \(i\). The joint probability of being at \(j\) and then moving to \(i\) is \(\pi_jp_{ji}\); the probability of being at \(i\) is \(\pi_i\). Therefore the reversed transition matrix \(P^{\mathrm{rev}}=(p^{\mathrm{rev}}_{ij})\) is given by the following formula.

In particular, detailed balance holds exactly when \(p^{\mathrm{rev}}_{ij}=p_{ij}\) for all \(i,j\): multiplying that equality by \(\pi_i\) gives \(\pi_i p_{ij}=\pi_jp_{ji}\), and dividing the detailed-balance equation by the positive \(\pi_i\) gives the reverse implication.

The reversal formula follows from conditioning, and also identifies what equality of the two transition matrices means. In a reversible chain, the probability of a transition \(i\to j\), when measured under stationarity, matches the probability of the reversed transition \(j\to i\). In the directed cycle, the reversed transition is absent, so equality cannot hold.

Path Probabilities Reveal the Direction

A stationary chain is reversible only if reversing whole paths leaves their probabilities unchanged. To see why detailed balance gives this property, consider a finite path \(x_0,x_1,\ldots,x_n\) under a stationary initial distribution. Its probability is \(\pi_{x_0}\prod_{r=0}^{n-1}p_{x_rx_{r+1}}\). If detailed balance holds, then \(\pi_{x_r}p_{x_rx_{r+1}}=\pi_{x_{r+1}}p_{x_{r+1}x_r}\) at each step. Multiplying these equalities yields \(\pi_{x_0}\prod_{r=0}^{n-1}p_{x_rx_{r+1}}=\pi_{x_n}\prod_{r=0}^{n-1}p_{x_{r+1}x_r}\), because the intermediate factors of \(\pi\) cancel. The right side is the probability of the reversed path \(x_n,\ldots,x_0\) under the same stationary chain. Thus detailed balance implies equality of forward and reversed path probabilities.

A Less Extreme Directed Cycle

The phenomenon does not depend on transitions being deterministic. A chain can allow movement both ways and still prefer one direction. The following example has a uniform stationary distribution because each column sums to \(1\), while unequal clockwise and counterclockwise probabilities prevent detailed balance.

Common Mistake: Confusing Stationarity with Pairwise Balance

A common error is to check \(\pi P=\pi\), observe that the distribution is preserved, and conclude that detailed balance must hold. The first condition adds together all transitions into a state; it does not require each transition to be balanced by its reverse. For the deterministic cycle, the incorrect calculation might write \(\pi_0p_{01}=\tfrac13\) and then claim the reverse side is also \(\tfrac13\) by using \(p_{01}=1\) again. That substitutes the forward transition probability for the reverse probability. The correct reverse term is \(\pi_1p_{10}=\tfrac13\cdot0=0\). Therefore stationarity holds, but detailed balance fails.

When checking a proposed reversible chain, calculate both \(p_{ij}\) and \(p_{ji}\) for the same pair of states. Do not infer pairwise equality from stationarity, and do not confuse a chain and its reverse merely because they share a stationary distribution. The directed examples show that stationary distributions describe how probability is spread across states, while reversibility describes whether the transition dynamics themselves are unchanged by reversing time.