Time Reversal of a Stationary CTMC

What Does It Mean to Run a CTMC Backward?

A stationary continuous-time Markov chain has the same distribution at every time, but its transitions may look different when the chain is viewed backward. A transition from \(i\) to \(j\) in the reversed chain is governed not just by the forward rate from \(j\) to \(i\), but also by the relative stationary masses of the two states. This tutorial derives that rate and explains when the reversed chain is well-defined.

Let \((X_t)_{t\ge0}\) be a finite-state continuous-time Markov chain on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with state space \(S\), generator \(Q=(q_{ij})\), and transition semigroup \(P(t)=e^{tQ}\). We use the row convention: \(q_{ij}\ge0\) for \(i\ne j\), and every row of \(Q\) sums to zero. The natural filtration is \(\mathcal{F}_t^X=\sigma(X_u:0\le u\le t)\). Assume the initial distribution is a stationary probability \(\pi\), so \(\pi P(t)=\pi\) for every \(t\ge0\). In finite state spaces this is equivalent to \(\pi Q=0\).

Write \(S_\pi=\{i\in S:\pi_i>0\}\). Reversal rates can be defined on \(S_\pi\), but not in general on all of \(S\): conditioning on \(X_t=i\) is not meaningful when \(\pi_i=0\). The next result establishes that \(S_\pi\) is closed under transitions, so the chain stays in this support when it is stationary.

Proof. Fix \(j\notin S_\pi\), so \(\pi_j=0\). The \(j\)th coordinate of \(\pi Q=0\) gives \[ 0=\sum_{i\in S}\pi_iq_{ij}=\sum_{i\ne j}\pi_iq_{ij}, \] because the diagonal term \(\pi_jq_{jj}\) is zero. Every term in the last sum is nonnegative. Therefore each term is zero, and for every \(i\) with \(\pi_i>0\), \(q_{ij}=0\). The chain cannot jump from \(S_\pi\) to its complement. Since it starts in \(S_\pi\) with probability one, it stays there. \(\square\)

The Reversed Semigroup and Generator

For \(i,j\in S_\pi\), define \[ p^*_{ij}(t)=\frac{\pi_jP(t)_{ji}}{\pi_i}. \] The indices matter: the forward transition in the numerator is from \(j\) to \(i\), opposite to the reversed transition from \(i\) to \(j\). The stationary probabilities supply the normalization needed to turn these quantities into transition probabilities.

Proof. We verify the semigroup, the reversed Markov probabilities, and the generator. First, stationarity gives \[ \sum_{j\in S_\pi}p^*_{ij}(t) =\frac{1}{\pi_i}\sum_{j\in S_\pi}\pi_jP(t)_{ji} =\frac{(\pi P(t))_i}{\pi_i}=1. \] Terms outside \(S_\pi\) contribute zero because their stationary masses are zero. Also \(P^*(0)\) is the identity: \(P(0)_{ji}\) is \(1\) if \(j=i\) and \(0\) otherwise. For \(s,t\ge0\), \[ \begin{aligned} \sum_{k\in S_\pi}p^*_{ik}(s)p^*_{kj}(t) &=\frac{\pi_j}{\pi_i}\sum_{k\in S_\pi}P(s)_{ki}P(t)_{jk}\\ &=\frac{\pi_j}{\pi_i}P(s+t)_{ji} =p^*_{ij}(s+t). \end{aligned} \] The middle equality is the semigroup identity, using \(P(t)P(s)=P(s+t)\). Thus \(P^*(t)\) is a stochastic semigroup.

To identify the reversed Markov probabilities, fix \(T>0\) and consider the finite-dimensional distributions of \(Y_s=X_{T-s}\), \(0\le s\le T\). For \(0\le s_0<\cdots<s_m\le T\), let \(i_0,\ldots,i_m\in S_\pi\). Stationarity and the forward Markov property give \[ \mathbb{P}(Y_{s_0}=i_0,\ldots,Y_{s_m}=i_m) =\pi_{i_m}\prod_{\ell=1}^m P(s_\ell-s_{\ell-1})_{i_\ell i_{\ell-1}}. \] Using the definition of \(p^*\), the right side equals \[ \pi_{i_0}\prod_{\ell=1}^m p^*_{i_{\ell-1}i_\ell}(s_\ell-s_{\ell-1}). \] This factorization proves that the reversed finite-dimensional distributions have initial law \(\pi\) and transition semigroup \(P^*(t)\). Finally, for \(i\ne j\), \[ q^*_{ij}=\lim_{h\downarrow0}\frac{p^*_{ij}(h)}{h} =\frac{\pi_jq_{ji}}{\pi_i}, \] since \(P(h)_{ji}/h\to q_{ji}\). For the diagonal, \(p^*_{ii}(h)=P(h)_{ii}\), so \(q^*_{ii}=q_{ii}\). Alternatively, the row sum follows directly from stationarity: \[ \sum_{j\in S_\pi}q^*_{ij} =\frac{1}{\pi_i}\sum_{j\in S_\pi}\pi_jq_{ji} =\frac{(\pi Q)_i}{\pi_i}=0. \] The off-diagonal entries are nonnegative, so \(Q^*\) is a generator. \(\square\)

This theorem concerns transition probabilities and finite-dimensional distributions. There is a path-version subtlety: under the usual right-continuous path convention, the exact process \(X_{T-s}\) need not have right-continuous paths. At a forward jump time \(u\), it takes the post-jump value at reversed time \(T-u\), while just afterward in reversed time it has the pre-jump value. For a right-continuous reversed path version, use \(\widehat Y_s=X_{(T-s)-}\) at interior times \(0<s<T\), with endpoint values chosen as \(\widehat Y_0=X_T\) and \(\widehat Y_T=X_0\). A finite-state CTMC has probability zero of jumping at any fixed deterministic time, so this version has the same finite-dimensional distributions as \(Y\).

Computing Reversed Rates

Zero Stationary Mass and a Common Error

Common mistake. On the directed-cycle example, using the forward rate \(q_{01}=2\) as the reversed rate from \(1\) to \(0\) gives the wrong answer. The formula uses the opposite forward transition and the stationary-mass ratio: \[ q^*_{10}=\frac{\pi_0q_{01}}{\pi_1} =\frac{(3/11)(2)}{6/11}=1, \] not \(2\). The reversed rate describes a transition from \(1\) to \(0\), so its forward counterpart is \(0\to1\), and the ratio accounts for the different stationary frequencies of \(0\) and \(1\).

Time reversal is useful when a model is easier to analyze by following probability flow backward, or when one wants to compare forward and backward transition mechanisms. The formula does not say that the reversed chain is always identical to the forward chain: the cycle example has reversed motion in the opposite direction. Whether the two generators agree is a further condition, treated under detailed balance and reversibility.