Self-Adjointness of Reversible Transition Operators

Transition Operators and Weighted Inner Products

A transition matrix acts on probability distributions from the right, but it also acts on functions from the left. That function action turns a Markov chain into a linear operator, so ideas from linear algebra become available. For a reversible chain, the appropriate inner product is weighted by the stationary distribution. Detailed balance then says exactly that the transition operator is self-adjoint in this inner product.

Let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space, and let \((X_n)_{n\ge 0}\) be a discrete-time Markov chain on a finite state space \(S\), with transition matrix \(P=(p_{ij})\). Each row of \(P\) sums to \(1\). Let \(\pi\) be a stationary distribution with full support: \(\pi_i>0\) for every \(i\in S\), \(\sum_i\pi_i=1\), and \(\pi P=\pi\). Full support ensures that the weighted inner product below is positive definite. If a stationary distribution assigns zero mass to some states, those states must be excluded from the space on which this is an inner product, or a full-support stationary distribution must be assumed.

Here \(Pf(i)\) is the conditional average of the next function value given that the chain is currently in state \(i\): if \(X_n=i\), then \(\mathbb{E}[f(X_{n+1})\mid X_n=i]=(Pf)(i)\). This definition concerns the action on functions, not the evolution of a distribution. The latter is written as a row vector \(\mu_{n+1}=\mu_nP\).

Proof. Assume first that detailed balance holds. For any functions \(f,g:S\to\mathbb{R}\), expand the two inner products using the definition of \(P\): \(\langle Pf,g\rangle_\pi=\sum_{i,j}\pi_i p_{ij}f(j)g(i)\). Detailed balance replaces \(\pi_i p_{ij}\) by \(\pi_j p_{ji}\). Reordering the finite sum then gives \(\sum_{j,i}\pi_j p_{ji}f(j)g(i)=\sum_j\pi_j f(j)\sum_i p_{ji}g(i)=\langle f,Pg\rangle_\pi\). Thus \(P\) is self-adjoint.

For the converse, let \(\mathbf{1}_{\{i\}}\) denote the function equal to \(1\) at state \(i\) and \(0\) at all other states. Apply self-adjointness with \(f=\mathbf{1}_{\{j\}}\) and \(g=\mathbf{1}_{\{i\}}\). The left side is \(\langle P\mathbf{1}_{\{j\}},\mathbf{1}_{\{i\}}\rangle_\pi=\pi_i p_{ij}\), since \((P\mathbf{1}_{\{j\}})(i)=p_{ij}\). The right side is \(\langle\mathbf{1}_{\{j\}},P\mathbf{1}_{\{i\}}\rangle_\pi=\pi_j p_{ji}\). Equality for every pair \(i,j\) gives detailed balance. This proves both directions. \(\square\)

The proof shows why the weighting matters: when the terms in one inner product are indexed by transitions \(i\to j\), detailed balance gives the same weight to the reversed transition \(j\to i\). The converse also makes the condition exact: on a finite space with full-support \(\pi\), self-adjointness is not merely a consequence of reversibility; it characterizes detailed balance.

Checking the Identity Directly

For a finite state space, the function calculation can be expressed as a matrix symmetry. Let \(D\) be the diagonal matrix with entries \(D_{ii}=\pi_i\). For a column vector of function values \(f\), \(\langle f,g\rangle_\pi=f^\mathsf{T}Dg\), and the operator acting on functions has matrix \(P\). Detailed balance is \(DP=P^\mathsf{T}D\). Thus \(\langle Pf,g\rangle_\pi=f^\mathsf{T}P^\mathsf{T}Dg=f^\mathsf{T}DPg=\langle f,Pg\rangle_\pi\). Since \(\pi_i>0\), the diagonal matrix \(D^{1/2}\) is invertible, and \(D^{1/2}PD^{-1/2}\) is symmetric: its \((i,j)\) entry is \(\sqrt{\pi_i}p_{ij}/\sqrt{\pi_j}\), which equals its \((j,i)\) entry by detailed balance. This is the ordinary Euclidean matrix form of the same weighted symmetry.

A Consequence: The Operator Does Not Increase the Weighted Norm

Define \(\|f\|_\pi^2=\langle f,f\rangle_\pi=\sum_i\pi_i f(i)^2\). A Markov transition operator with stationary distribution \(\pi\) satisfies \(\|Pf\|_\pi\le\|f\|_\pi\) for every real-valued function \(f\). This contraction property uses stationarity, not detailed balance, so it holds even for a stationary chain that is not reversible.

Proof. For each state \(i\), write \(m_i=(Pf)(i)=\sum_jp_{ij}f(j)\). Since \(\sum_jp_{ij}=1\), \(\sum_jp_{ij}(f(j)-m_i)^2\ge0\). Expanding the left side gives \(\sum_jp_{ij}f(j)^2-2m_i\sum_jp_{ij}f(j)+m_i^2\sum_jp_{ij}=\sum_jp_{ij}f(j)^2-m_i^2\). Hence \((Pf)(i)^2\le\sum_jp_{ij}f(j)^2\). Multiply by \(\pi_i\), sum over \(i\), and use stationarity, \(\sum_i\pi_i p_{ij}=\pi_j\): \(\|Pf\|_\pi^2\le\sum_{i,j}\pi_i p_{ij}f(j)^2=\sum_j\pi_j f(j)^2=\|f\|_\pi^2\). All sums are finite, so rearranging them is valid. Taking nonnegative square roots proves the stated inequality. \(\square\)

When detailed balance also holds, the operator is both self-adjoint and a contraction. These are distinct properties: the contraction proof used only the averaging in each row and stationarity, whereas self-adjointness requires the pairwise balance of opposite flows.

Why Stationarity Alone Is Not Enough

Common pitfall. A tempting but incorrect calculation is to treat \(\pi P=\pi\) as if it implied \(\pi_i p_{ij}=\pi_jp_{ji}\) term by term. Stationarity only says that total incoming mass at each state equals its stationary mass: \(\sum_i\pi_i p_{ij}=\pi_j\). It does not equate the flow on each edge with the flow on its reverse. In the directed-cycle example, the incoming flows sum correctly at every state, yet the flow from \(1\) to \(2\) is \(\tfrac13\) and the reverse flow is zero. The correction is to check pairwise detailed balance, or to test the inner-product identity on indicator functions for every pair of states.