Detailed Balance and Reversible Chains

Pairwise Balance and Reversibility

Stationarity describes how probability mass is distributed after one transition: the total mass arriving at each state agrees with its mass before the transition. Reversibility imposes a more detailed condition. It asks whether, for every pair of states, the stationary probability flow from the first state to the second matches the flow in the opposite direction. Keeping these two levels of balance distinct is important: stationarity is a global, state-by-state equation, whereas detailed balance compares individual pairs.

Let \((X_n)_{n\ge0}\) be a Markov chain on a finite or countable state space \(S\), on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with discrete time \(n\ge0\) and row-stochastic transition matrix \(P=(p_{ij})\). Thus \(\sum_{j\in S}p_{ij}=1\) for every \(i\). A distribution is a probability row vector \(\pi=(\pi_i)_{i\in S}\). It is stationary when \(\pi P=\pi\), or, coordinate by coordinate, when \(\sum_{i\in S}\pi_i p_{ij}=\pi_j\) for every \(j\).

The edge-flow terminology includes self-loops: \(f_{ii}=\pi_i p_{ii}\), and the balance equation for \(i=j\) holds automatically. For distinct states, the equation matches the two directed flows. The equations are meaningful even when some \(\pi_i=0\). In that case they do not impose a restriction on every transition out of a zero-mass state. For example, if \(\pi_i=0\) and \(\pi_j>0\), detailed balance requires \(p_{ji}=0\), but does not require \(p_{ij}=0\).

The stationarity equations and the detailed-balance equations have different forms. Stationarity at \(j\) sums all incoming flows, \(\sum_i f_{ij}=\pi_j\). Detailed balance instead compares each incoming flow \(f_{ij}\) with the corresponding outgoing flow \(f_{ji}\) separately. Pairwise equations therefore record more local information than the aggregate stationarity equation. A standard result is that detailed balance with respect to a probability distribution implies that the distribution is stationary; the proof of that implication is treated separately. The calculations below check stationarity directly when it is needed, rather than substituting the implication for the check.

Global Flow Conservation and Stationarity

Proof. For each fixed \(i\), nonnegativity and the row-sum condition give \(\sum_j f_{ij}=\sum_j\pi_i p_{ij}=\pi_i\sum_jp_{ij}=\pi_i\), including when \(\pi_i=0\). For each fixed \(j\), \(\sum_i f_{ij}=\sum_i\pi_i p_{ij}\), which is exactly the \(j\)-th coordinate of the row vector \(\pi P\). Thus \(\sum_i f_{ij}=\pi_j\) for every \(j\) holds exactly when \(\pi P=\pi\). These sums are well-defined for a countable state space because their terms are nonnegative. \(\square\)

This proposition explains the word “global”: the stationarity condition checks a total at each destination after summing over every possible source. Detailed balance is the pairwise condition \(f_{ij}=f_{ji}\). Do not replace one type of equation with the other when doing a calculation. A direct stationarity check uses incoming sums; a direct detailed-balance check compares pairs.

Reversibility and the Reverse Transition Matrix

For a stationary chain with \(\pi_i>0\) at every state under consideration, the reverse transition probability from \(i\) to \(j\) is \(p^*_{ij}=\pi_jp_{ji}/\pi_i\). This formula describes transitions read in the opposite time direction when the chain is stationary. The positivity condition ensures that the division by \(\pi_i\) is defined. If \(\pi\) is stationary, the reverse row sums are one: for a fixed \(i\), \(\sum_j\pi_jp_{ji}/\pi_i=\pi_i/\pi_i=1\), using stationarity at \(i\).

Proof. Fix any \(i,j\in S\). The equality \(p^*_{ij}=p_{ij}\), after substituting the definition of \(p^*_{ij}\), is \(\pi_jp_{ji}/\pi_i=p_{ij}\). Since \(\pi_i>0\), multiplying both sides by \(\pi_i\) gives \(\pi_jp_{ji}=\pi_i p_{ij}\), exactly the detailed-balance equation for this pair. Reversing the algebra shows that this detailed-balance equation implies \(p^*_{ij}=p_{ij}\). Since the argument holds for each pair, \(P^*=P\) if and only if all detailed-balance equations hold. \(\square\)

Common Mistakes and Checks

A frequent error is to test reversibility by comparing \(p_{ij}\) directly with \(p_{ji}\), without weighting by the stationary probabilities. In the two-state example, that incorrect test says \(\tfrac14\ne\tfrac12\), so it labels the chain nonreversible. The correction is to use the detailed-balance equation: \(\pi_1p_{12}=\tfrac23\cdot\tfrac14=\tfrac16\), while \(\pi_2p_{21}=\tfrac13\cdot\tfrac12=\tfrac16\). The flows match, so the chain is reversible with respect to \(\pi\).

Another error is to treat a stationarity equation as though it matched every pair of transitions. The equation at state \(j\) is the single sum \(\sum_i\pi_i p_{ij}=\pi_j\); it does not itself list pairwise comparisons. Conversely, checking only one pair cannot establish the full collection of detailed-balance equations. For finite chains, a practical check is to verify that \(\pi\) is a probability distribution, check the transition row sums, and then compute the required flows for every distinct pair, along with the global column sums when stationarity must be confirmed directly.

When a stationary distribution has zero entries, do not use the reverse-transition formula by dividing by those entries. The pairwise detailed-balance equations remain defined, but the reverse transition formula above requires positive \(\pi_i\) for its row \(i\). In an irreducible finite chain the stationary distribution is positive at every state; without such a condition, the positivity needed for this formula must be checked rather than assumed.