Common Errors About Stationarity and Reversibility

Two Equations That Answer Different Questions

Stationarity and reversibility are related, but they are not interchangeable. Stationarity says that a distribution is unchanged after one transition. Reversibility says, in addition, that the stationary flow from each state to another is balanced by the flow in the opposite direction. Confusing these statements can produce false conclusions even for small chains.

Let \((X_n)_{n\ge0}\) be a discrete-time Markov chain on a finite or countable state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with transition matrix \(P=(p_{ij})\). Its rows sum to \(1\). Use the natural filtration \(\mathcal{F}_n=\sigma(X_0,\ldots,X_n)\). A distribution is a row vector. It is stationary if \(\pi P=\pi\). A stationary distribution \(\pi\) satisfies detailed balance if \(\pi_i p_{ij}=\pi_j p_{ji}\) for every \(i,j\in S\); the chain is reversible with respect to \(\pi\) when these equalities hold.

The distinction is visible by summing the pairwise equations over possible origins. If detailed balance holds, then for each \(j\), \(\sum_i\pi_i p_{ij}=\sum_i\pi_j p_{ji}=\pi_j\), because row \(j\) of \(P\) sums to \(1\). This gives stationarity. The converse does not follow: the stationarity equation for \(j\) sums all incoming flows, so it does not require each pair of opposite flows to match.

Zero Transitions and Matrix Indices

A zero transition probability is part of the transition structure, not a term to omit before comparing flows. If \(p_{ij}=0\) and \(\pi_j>0\), detailed balance forces \(p_{ji}=0\). For a chain with a strictly positive stationary distribution, therefore, an allowed transition \(i\to j\) must have an allowed reverse transition \(j\to i\). When \(\pi\) has zero entries, detailed balance need not impose that same support condition on states outside its positive support; check the equation as written rather than assuming all entries of \(\pi\) are positive.

For additional clarity, when \(\pi_i>0\) the time-reversed transition probability from \(i\) to \(j\) is \(p^*_{ij}=\pi_jp_{ji}/\pi_i\). The indices in \(p_{ji}\) are reversed: the original chain moves from \(j\) to \(i\) in the numerator. Detailed balance says \(p^*_{ij}=p_{ij}\). This formula does not define a reversed row at a state \(i\) with \(\pi_i=0\); do not divide by zero.

What the Cycle Criterion Does—and Does Not—Say

For a finite irreducible chain, the cycle criterion tests whether a strictly positive reversible stationary distribution exists. Irreducibility means that every state can be reached from every other state along a path of transitions with positive probability. A finite closed walk is a sequence \(i_0,i_1,\ldots,i_m=i_0\); its forward product is \(\prod_{r=0}^{m-1}p_{i_r i_{r+1}}\), and its reverse product is \(\prod_{r=0}^{m-1}p_{i_{r+1}i_r}\). Zero factors count: equality is equality of the products including zeros.

Proof. Suppose first that a strictly positive distribution \(\pi\) satisfies detailed balance. Multiplying \(\pi_{i_r}p_{i_r i_{r+1}}=\pi_{i_{r+1}}p_{i_{r+1}i_r}\) over \(r=0,\ldots,m-1\) gives equal products after cancellation of the positive factors \(\pi_{i_r}\); the list of these factors on each side is the same because \(i_m=i_0\). This proves the cycle equalities.

Conversely, suppose all closed-walk equalities hold. First, positive transitions have positive reverse transitions. If \(p_{ij}>0\), irreducibility supplies a positive-probability path from \(j\) back to \(i\). Joining the edge \(i\to j\) to that path gives a closed walk with positive forward product. Its reverse product must also be positive, and that product contains \(p_{ji}\), so \(p_{ji}>0\).

Fix a reference state \(r\). For any positive-edge path \(r=x_0,x_1,\ldots,x_n=i\), assign the ratio \(R=\prod_{\ell=0}^{n-1}(p_{x_\ell x_{\ell+1}}/p_{x_{\ell+1}x_\ell})\). All denominators are positive by the preceding argument. This ratio does not depend on the chosen path: follow one path from \(r\) to \(i\), then follow the other path backwards from \(i\) to \(r\). The result is a closed walk. The cycle equality says its product of forward-to-reverse ratios is \(1\), which says the two path ratios agree. Define \(w_r=1\) and \(w_i=R\) for any such path. Every \(w_i\) is positive. Extending a path to \(i\) by an edge \(i\to j\) gives \(w_j=w_i p_{ij}/p_{ji}\), hence \(w_i p_{ij}=w_jp_{ji}\) for positive edges. If \(p_{ij}=0\), then \(p_{ji}=0\) by the support argument, so this equality holds for those pairs too. Since \(S\) is finite, \(C=\sum_i w_i\) is finite and positive. Set \(\pi_i=w_i/C\); it sums to \(1\) and satisfies detailed balance. Summing detailed balance over \(i\) gives \(\sum_i\pi_i p_{ij}=\pi_j\sum_i p_{ji}=\pi_j\), using row \(j\)'s sum of \(1\). Thus \(\pi\) is stationary. \(\square\)

The finite-state and irreducibility assumptions matter. On a countably infinite state space, cycle equalities can produce positive reversible weights whose sum is infinite; such weights cannot be normalized into a stationary probability distribution. For example, on the integers, a chain that moves to either neighboring integer with probability \(\tfrac12\) has equal forward and reverse products around every closed walk. But stationarity would require \(\pi_j=(\pi_{j-1}+\pi_{j+1})/2\). This recurrence makes \(\pi_j\) affine in \(j\); a nonnegative affine sequence on all integers is either zero or a positive constant, and neither gives a probability distribution. The cycle equalities alone therefore do not establish reversibility under a definition that requires a stationary probability distribution.

Common Errors to Avoid

Wrong calculation: “Since \(\pi P=\pi\), detailed balance holds.” In the first example, stationarity gives \(\pi P=(\tfrac13,\tfrac13,\tfrac13)\), but \(\pi_1p_{12}=\tfrac13\) and \(\pi_2p_{21}=0\). The correction is to check the pairwise equations separately; a summed flow balance does not imply pairwise balance.

Wrong index: Replacing \(\pi_jp_{ji}\) in the detailed-balance equation with \(\pi_jp_{ij}\) compares the same-direction transitions and is not the reversal test. For the reversed kernel, start from \(i\), look at the original transition from \(j\) to \(i\), and use \(p^*_{ij}=\pi_jp_{ji}/\pi_i\) only when \(\pi_i>0\).

Wrong treatment of zeros: Omitting a zero from a pairwise check can hide a failed equality. In the directed three-state example, the zero reverse transition is exactly why detailed balance fails. Conversely, in the reversible example the two zero flows match. Always write both sides of each equation, including zeros.

Wrong use of a cycle test: Do not infer a strictly positive stationary probability distribution from cycle equalities without the finite irreducible hypotheses. In a reducible chain, cycle checks can miss states that cannot carry positive reversible weight. In a countably infinite chain, even positive reversible weights may have infinite total mass and fail to normalize.