Detailed Balance and Reversibility

From Generator Rates to Reversibility

In a stationary continuous-time Markov chain, the rate of transitions from \(i\) to \(j\) need not match the rate in the opposite direction: the chain may carry a persistent circulation around its state space. Detailed balance is the stronger condition that the stationary flow from \(i\) to \(j\) exactly matches the flow back. This condition can be checked directly from a generator, and in a finite state space it guarantees both stationarity and reversibility.

Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Let \((X_t)_{t\ge0}\) be a time-homogeneous continuous-time Markov chain with finite state space \(S\), generator \(Q=(q_{ij})\), and transition matrix \(P(t)=(p_{ij}(t))=e^{tQ}\). For \(i\ne j\), \(q_{ij}\ge0\) is the instantaneous transition rate from \(i\) to \(j\); \(q_{ii}=-\sum_{j\ne i}q_{ij}\), so every row of \(Q\) sums to zero. Set \(\mathcal{F}_t=\sigma(X_s:0\le s\le t)\) for the natural filtration. The finite-state assumption ensures the chain is nonexplosive. We use the standard finite-state CTMC fact that \(P(t)\) is stochastic and satisfies the semigroup relation \(P(s+t)=P(s)P(t)\).

A probability vector \(\pi=(\pi_i)_{i\in S}\) is written as a row vector. It is stationary if \(\pi P(t)=\pi\) for every \(t\ge0\). When \(X_0\) has distribution \(\pi\), the chain is reversible if for every \(m\ge1\), every \(0\le t_0<t_1<\cdots<t_m\), and states \(i_0,\ldots,i_m\), the joint probability of the sequence of states at those times is unchanged when the order of the states and the time intervals are reversed.

Why Detailed Balance Gives Stationarity and Reversibility

Proof. First suppose detailed balance holds. For each state \(j\), use the detailed balance equations and the zero sum of row \(j\) of \(Q\) to obtain

The equality also holds when some \(\pi_j=0\); no division by a component of \(\pi\) is involved. Thus \(\pi Q=0\). Since \(P(t)=e^{tQ}\), the power series for the matrix exponential gives \(\pi P(t)=\pi\): the constant term is \(\pi\), and every term of positive degree contains \(\pi Q=0\).

It remains to prove reversibility. Let \(D\) be the diagonal matrix with \(D_{ii}=\pi_i\). Detailed balance, including the automatic diagonal identities, says \(DQ=Q^{\mathsf T}D\). Induction on \(n\) gives \(DQ^n=(Q^{\mathsf T})^nD\): it holds for \(n=0\), and multiplying the identity for \(n\) by the relation \(DQ=Q^{\mathsf T}D\) gives the identity for \(n+1\). Taking the matrix-exponential series therefore yields \(DP(t)=P(t)^{\mathsf T}D\). In entries, this is \(\pi_i p_{ij}(t)=\pi_jp_{ji}(t)\).

For ordered times \(t_0<\cdots<t_m\), write \(\Delta_\ell=t_\ell-t_{\ell-1}\). By stationarity and the Markov property, the joint probability of the specified states is \(\pi_{i_0}\prod_{\ell=1}^m p_{i_{\ell-1},i_\ell}(\Delta_\ell)\). Apply the transition-kernel identity to each factor. The factors of \(\pi\) telescope, giving \(\pi_{i_m}\prod_{\ell=1}^m p_{i_\ell,i_{\ell-1}}(\Delta_\ell)\), the probability of the reversed sequence with reversed time intervals. This multiplication argument remains valid if a component of \(\pi\) is zero, because the entrywise identities hold without division. Hence the stationary chain is reversible.

For the converse, take two states \(i\ne j\) and use reversibility at times \(0\) and \(t>0\). It gives \(\pi_i p_{ij}(t)=\pi_jp_{ji}(t)\). For a finite-state CTMC, \(P(0)=I\) and \(P'(0)=Q\), so for distinct states \(p_{ij}(0)=p_{ji}(0)=0\). Divide the equality by \(t\) and let \(t\) decrease to zero; the derivative definition gives \(\pi_iq_{ij}=\pi_jq_{ji}\). This proves both directions. \(\square\)

Thus detailed balance is exactly the generator-level test for reversibility when the chain is stationary. It also proves stationarity directly, without first solving differential equations for \(P(t)\). A stationary distribution need not satisfy detailed balance: stationarity only requires the total inflow and outflow to balance at each state, while detailed balance requires the flow to balance separately on each pair of states.

Checking Detailed Balance in Finite Chains

Common Mistakes and a Practical Test

A common mistake is to treat stationarity and detailed balance as interchangeable. For the directed-cycle example, one might incorrectly infer reversibility from \(\pi Q=0\). The calculation that exposes the error is \(\pi_1q_{12}=\tfrac23\ne\tfrac13=\pi_2q_{21}\): the distribution is stationary, but the pairwise flows do not match. Stationarity checks net flow at each state; reversibility checks every pair of states.

Another error is to compare rates alone, ignoring the stationary masses. The condition is \(\pi_iq_{ij}=\pi_jq_{ji}\), not \(q_{ij}=q_{ji}\). Unequal rates can satisfy detailed balance when the corresponding stationary probabilities compensate, as in the two-state example. For a finite-state candidate, a reliable procedure is to check that it is a probability vector, verify the generator row sums, test each unordered pair once, and then multiply \(\pi Q\) as an independent stationarity check. If all detailed balance equations hold, the theorem supplies stationarity and reversibility.

Zero probabilities require care as well. Detailed balance is an equality of products and remains meaningful when \(\pi_i=0\); it must not be rearranged by dividing by \(\pi_i\) unless \(\pi_i>0\) has been established. In finite state spaces the theorem includes such cases, but a proposed vector with zero mass still has to satisfy every pairwise equation. In applications on infinite state spaces, generator equations can require additional conditions to justify the passage from rates to a stationary process; the finite-state proof here uses the matrix exponential directly.