Time Reversal of a Stationary Markov Chain

Reading a Stationary Chain Backward

A Markov chain evolves forward by choosing its next state from its current state. If the chain is observed in equilibrium, its past also has a precise probability law when viewed from the present. That law is determined by the joint probabilities of consecutive states. In general, reading the chain backward changes its transition probabilities: the chance that the previous state was \(j\), given that the current state is \(i\), need not equal the forward probability of moving from \(i\) to \(j\).

Let \((X_n)_{n\ge 0}\) be a discrete-time Markov chain on a finite or countable state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with transition matrix \(P=(p_{ij})\). The matrix is row-stochastic, so \(\sum_{j\in S}p_{ij}=1\) for every \(i\). Suppose the chain starts with distribution \(\pi\), a row vector satisfying \(\pi P=\pi\). Then \(X_n\) has distribution \(\pi\) at every time \(n\); this is the stationarity assumption used below.

For a finite window \(X_0,\ldots,X_m\), reading backward gives \(X_m,\ldots,X_0\). The distribution of this reversed window does not depend on where the window starts, by stationarity. We define the reversed chain in law by starting with distribution \(\pi\) and using the transition probabilities obtained from those backward windows. This definition concerns distributions: it does not assert that a one-sided chain already defined on \(\Omega\) has values \(X_{-1},X_{-2},\ldots\) on that same probability space.

Deriving the Reverse Transition Probabilities

For \(i,j\in S_\pi\), the event that the chain is at \(j\) and then moves to \(i\) has probability \(\pi_jp_{ji}\). Also, \(\mathbb{P}(X_n=i)=\pi_i>0\). Thus the conditional probability of the previous state \(j\), given current state \(i\), is

This calculation requires \(\pi_i>0\). It also shows why the order of the indices matters: \(p_{ji}\) is the forward probability of moving from the proposed previous state \(j\) to the current state \(i\).

Proof. First fix \(i\in S_\pi\). Since \(\pi P=\pi\), the \(i\)th component of this equality is \(\pi_i=\sum_{j\in S}\pi_jp_{ji}\). Therefore \[ \sum_{j\in S_\pi}q_{ij} =\frac{1}{\pi_i}\sum_{j\in S_\pi}\pi_jp_{ji} =\frac{\pi_i}{\pi_i}=1. \] Terms with \(j\notin S_\pi\) vanish because \(\pi_j=0\), so restricting the sum to \(S_\pi\) changes nothing. Each \(q_{ij}\) is nonnegative, and if \(q_{ij}>0\), then \(\pi_j>0\); hence the reverse transition stays within \(S_\pi\). Thus \(Q\) is row-stochastic on \(S_\pi\).

For the path formula, the Markov property gives the forward stationary path probability \[ \mathbb{P}(X_0=i_m,\ldots,X_m=i_0) =\pi_{i_m}p_{i_m i_{m-1}}\cdots p_{i_1 i_0}. \] The probability for the reversed chain, by its initial distribution and transition rule, is \[ \pi_{i_0}\prod_{r=1}^{m}q_{i_{r-1}i_r} =\pi_{i_0}\prod_{r=1}^{m} \frac{\pi_{i_r}p_{i_r i_{r-1}}}{\pi_{i_{r-1}}}. \] The factors involving \(\pi\) cancel consecutively: the initial \(\pi_{i_0}\) cancels the first denominator, and each subsequent numerator cancels the next denominator, leaving \(\pi_{i_m}\). The remaining factors are \(p_{i_r i_{r-1}}\), giving the stated forward path probability in reverse order. This proves equality of the finite-dimensional path probabilities. For \(m=0\), the formula reduces to \(\mathbb{P}(Y_0=i_0)=\pi_{i_0}\), so the zero-step case also holds. \(\square\)

The proof identifies more than the probability of one reversed step. It shows that every consecutive block has the right reversed law. In particular, the reverse process is Markov: the probability of its next state, conditional on any positive-probability history ending at \(i\), is \(q_{ij}\). This follows from the path factorization, since the factors for the earlier part of the history cancel when the conditional probability is formed. Conditional probabilities given a history of probability zero are not required to have a specified value.

The reverse chain is stationary as well. For \(j\in S_\pi\), its probability of being at \(j\) after one step when it starts from \(\pi\) is \[ \sum_{i\in S_\pi}\pi_iq_{ij} =\sum_{i\in S_\pi}\pi_jp_{ji} =\pi_j\sum_{i\in S}p_{ji} =\pi_j. \] In the penultimate equality, states outside \(S_\pi\) add zero to the sum. The final sum is row \(j\) of \(P\), so it is \(1\). Thus \(\pi Q=\pi\). Repeating the same one-step argument shows that every \(Y_n\) has distribution \(\pi\).

Worked Path Calculations

Reading the Formula Correctly

The reverse transition is a conditional probability under the stationary joint law. It combines two ingredients: the stationary chance \(\pi_j\) of being at the proposed previous state and the forward chance \(p_{ji}\) of then arriving at the current state. Dividing by \(\pi_i\) conditions on that current state. This also explains why stationary probabilities are essential: without the stationary joint law, the distribution of a previous state can depend on the time and the initial distribution.

Common mistake: reversing the indices incorrectly. In the directed-cycle example, a tempting but wrong calculation is to use \(p_{02}\) for the probability that the previous state was \(2\) given current state \(0\). Since \(p_{02}=0\), this would incorrectly give \(q_{02}=0\). The forward transition that must be used is from \(2\) to \(0\), namely \(p_{20}=1\). The correct calculation is \(q_{02}=\pi_2p_{20}/\pi_0=(\tfrac13\cdot1)/(\tfrac13)=1\). The direction in \(p_{ji}\) runs from the proposed past state \(j\) to the present state \(i\).

Another important distinction is between a stationary chain and a chain whose reversed transition matrix happens to equal its forward matrix. Stationarity gives a well-defined backward law on states with positive stationary probability; it does not require the backward and forward transition probabilities to agree. The directed cycle demonstrates this: the stationary distribution is uniform, but the reverse cycle runs in the opposite direction.

For a finite set of observation times rather than consecutive times, the same path-law principle applies. One can fill in the unobserved times between observations and sum the consecutive-path probabilities over those intermediate states. Equivalently, the resulting probabilities use the corresponding multi-step transition probabilities. The essential point remains that the reversed finite-dimensional laws are fixed by the stationary probabilities of forward paths, read in reverse order.