From Instantaneous Balance to Invariance
For a continuous-time Markov chain, the generator describes how probability begins to move over a very short interval. An invariant probability distribution asks for something stronger: if the chain starts with that distribution, its distribution remains unchanged at every later time. In a finite state space, the link between these two ideas is exact. A probability row vector satisfies the generator balance equations if and only if it is invariant under all the transition matrices.
Let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space and let \((X_t)_{t\ge0}\) be a time-homogeneous continuous-time Markov chain with finite state space \(S=\{1,\ldots,d\}\). Write \(P(t)=(p_{ij}(t))\), where \(p_{ij}(t)=\mathbb{P}(X_t=j\mid X_0=i)\). The generator is the matrix \(Q=(q_{ij})\) of transition rates: for \(i\ne j\), \(q_{ij}\ge0\), and \(q_{ii}=-\sum_{j\ne i}q_{ij}\). Thus each row of \(Q\) sums to zero. In the finite-state setting, the transition matrices are \(P(t)=e^{tQ}\).
The distinction is between a condition on rates and a condition on the distribution after any elapsed time. The balance equation \(\pi Q=0\) says that, under weights \(\pi_i\), the rate of probability flow into each state equals the rate flowing out. The result below shows why that instantaneous equality is enough in a finite state space.
Proof. The matrix exponential is defined by the convergent series \(e^{tQ}=\sum_{k=0}^{\infty}t^kQ^k/k!\). Because \(Q\) is a finite matrix, this series converges entrywise for every \(t\), and it can be differentiated term by term. Hence \(P'(t)=P(t)Q\) and \(P(0)=I\). If \(\pi Q=0\), then \(\pi Q^k=0\) for every integer \(k\ge1\): the assertion holds for \(k=1\), and for larger \(k\), \(\pi Q^k=(\pi Q)Q^{k-1}=0\). Therefore the exponential series gives \(\pi P(t)=\pi+\sum_{k=1}^{\infty}t^k\pi Q^k/k!=\pi\) for every \(t\ge0\). Conversely, if \(\pi P(t)=\pi\) for every \(t\ge0\), subtract \(\pi P(0)=\pi\), divide by \(t>0\), and let \(t\) decrease to zero. By the definition \(Q=P'(0)\), the limit is \(\pi Q=0\). This proves both directions. \(\square\)
The same matrix-exponential representation also follows from the forward equation. In finite state space, the transition matrices satisfy \(P'(t)=P(t)Q\), \(P(0)=I\). To see why the solution is unique, suppose \(R(t)\) is any differentiable matrix satisfying these equations. The product rule gives \(\frac{d}{dt}\big(R(t)e^{-tQ}\big)=R(t)Qe^{-tQ}-R(t)Qe^{-tQ}=0\). Thus \(R(t)e^{-tQ}\) is constant and equals \(I\) at zero, so \(R(t)=e^{tQ}\). This explains why the finite-state assumption matters to the argument: it permits the ordinary finite-matrix exponential and its differentiation without domain issues.
Worked Calculations
Worked Example: A Two-State Chain
Consider \(S=\{1,2\}\), with rate \(2\) from state \(1\) to state \(2\), and rate \(1\) from state \(2\) to state \(1\). The generator is \(Q=\begin{bmatrix}-2&2\\1&-1\end{bmatrix}\). Its first row sums to \(-2+2=0\), and its second row sums to \(1-1=0\); its off-diagonal entries are nonnegative.
Let \(\pi=(a,1-a)\). The first coordinate of \(\pi Q=0\) requires \(-2a+(1-a)=0\), so \(1-3a=0\) and \(a=\tfrac13\). The second coordinate then gives \(2a-(1-a)=\tfrac23-\tfrac23=0\). Thus \(\pi=(\tfrac13,\tfrac23)\), whose entries are nonnegative and sum to \(1\).
Here \(Q^2=-3Q\), as direct multiplication gives \(Q^2=\begin{bmatrix}6&-6\\-3&3\end{bmatrix}=-3Q\). Consequently, for \(k\ge1\), \(Q^k=(-3)^{k-1}Q\), and the exponential series reduces to \(P(t)=I+\frac{1-e^{-3t}}{3}Q\). Set \(r=e^{-3t}\). Substitution gives
For \(t\ge0\), each row sums to \(1\): the first has numerator \(1+2r+2-2r=3\), and the second has numerator \(1-r+2+r=3\), each over \(3\). Multiplying by \(\pi\), the first coordinate is \(\tfrac13\cdot\tfrac{1+2r}{3}+\tfrac23\cdot\tfrac{1-r}{3}=\tfrac{1+2r+2-2r}{9}=\tfrac13\). The second is \(\tfrac13\cdot\tfrac{2(1-r)}{3}+\tfrac23\cdot\tfrac{2+r}{3}=\tfrac{2-2r+4+2r}{9}=\tfrac23\). So the explicit transition matrix confirms the theorem’s conclusion.
Worked Example: An Asymmetric Three-State Cycle
Let the chain move from \(1\) to \(2\) at rate \(2\), from \(2\) to \(3\) at rate \(1\), and from \(3\) to \(1\) at rate \(3\), with no other transitions. Its generator is \(Q=\begin{bmatrix}-2&2&0\\0&-1&1\\3&0&-3\end{bmatrix}\). The row sums are \(-2+2+0=0\), \(0-1+1=0\), and \(3+0-3=0\), and every off-diagonal entry is nonnegative.
Write \(\pi=(a,b,c)\). Multiplication by \(Q\) gives \(\pi Q=(-2a+3c,\ 2a-b,\ b-3c)\). Setting each coordinate equal to zero gives \(b=2a\) and \(b=3c\), so \(c=\tfrac23a\). Normalization requires \(a+b+c=a+2a+\tfrac23a=\tfrac{11}{3}a=1\); hence \(a=\tfrac3{11}\), \(b=\tfrac6{11}\), and \(c=\tfrac2{11}\).
Check: The entries sum to \(\tfrac3{11}+\tfrac6{11}+\tfrac2{11}=1\). Also, the three coordinates of \(\pi Q\) are \(-2\cdot\tfrac3{11}+3\cdot\tfrac2{11}=0\), \(2\cdot\tfrac3{11}-\tfrac6{11}=0\), and \(\tfrac6{11}-3\cdot\tfrac2{11}=0\). Therefore \(\pi P(t)=\pi\) for every \(t\ge0\). The rates around the cycle are unequal, so this example also shows that invariance does not require equal transition rates in both directions along each pair of states.
Worked Example: Two Absorbing States and Many Invariant Probabilities
Take \(S=\{1,2,3\}\), where states \(1\) and \(2\) are absorbing, and state \(3\) moves to state \(1\) at rate \(2\). The generator is \(Q=\begin{bmatrix}0&0&0\\0&0&0\\2&0&-2\end{bmatrix}\). Its row sums are \(0\), \(0\), and \(2-2=0\), and its off-diagonal entries are nonnegative.
For \(\pi=(a,b,c)\), the balance equations are \(\pi Q=(2c,0,-2c)=(0,0,0)\), so \(c=0\). The conditions that \(\pi\) be a probability vector then reduce to \(a+b=1\), \(a\ge0\), and \(b\ge0\). Thus every vector \((a,1-a,0)\) with \(0\le a\le1\) satisfies generator balance.
This family really is invariant. With \(r=e^{-2t}\), the transition matrix is \(P(t)=\begin{bmatrix}1&0&0\\0&1&0\\1-r&0&r\end{bmatrix}\). Its first and second rows sum to \(1\), and its third sums to \((1-r)+r=1\). Direct multiplication yields \((a,1-a,0)P(t)=(a,1-a,0)\) for every \(t\ge0\). Different invariant probabilities arise because probability initially on either absorbing state remains there.
Why Invariance Makes the Process Stationary
Invariance also has a process-level meaning. If \(X_0\) has distribution \(\pi\) and \(\pi\) is invariant, then the distribution of \(X_t\) is \(\pi\) for every \(t\ge0\). More fully, all finite-dimensional distributions are unchanged by a common shift in time, provided the times being compared are nonnegative. This property is called stationarity of the process.
Proof. For states \(i_1,\ldots,i_m\), the Markov property and time homogeneity give the joint probability at times \(t_1,\ldots,t_m\) as \((\pi P(t_1))_{i_1}\prod_{\ell=2}^{m}p_{i_{\ell-1}i_\ell}(t_\ell-t_{\ell-1})\). At shifted times the first factor is \((\pi P(t_1+h))_{i_1}=\pi_{i_1}\), by invariance, and each later time difference remains \((t_\ell+h)-(t_{\ell-1}+h)=t_\ell-t_{\ell-1}\). The resulting joint probability is therefore the same as at the original times. Since this equality holds for every state tuple, the joint distributions agree. \(\square\)
This proposition concerns stationarity, not convergence to equilibrium. Generator balance proves that a chain started with \(\pi\) keeps distribution \(\pi\); it does not say that a chain started from an arbitrary state will approach \(\pi\). Nor does it imply that the invariant probability is unique, as the absorbing-state example demonstrates.
A Common Notational Pitfall
With the row-stochastic convention, distributions are row vectors and evolve by \(\mu P(t)\). Generator balance is therefore \(\pi Q=0\), not \(Q\pi^{\mathsf T}=0\). In the two-state example, \(Q\pi^{\mathsf T}=\begin{bmatrix}-2&2\\1&-1\end{bmatrix}\begin{bmatrix}\tfrac13\\\tfrac23\end{bmatrix}=\begin{bmatrix}\tfrac23\\-\tfrac13\end{bmatrix}\), which is not zero. That calculation is not a contradiction: it applies the generator to a column vector, whereas the probability distribution is a row vector. The correct check is \((\tfrac13,\tfrac23)Q=(0,0)\).
For a finite-state problem, a reliable procedure is to first verify that \(Q\) has nonnegative off-diagonal entries and rows summing to zero, then solve \(\pi Q=0\) together with nonnegativity and normalization. Once those conditions hold, the theorem supplies invariance for all \(t\); computing \(e^{tQ}\) is not required to establish it. Directly computing \(P(t)\), as in the two-state example, remains useful as an independent check.
Check Your Understanding
Use row vectors throughout, and check any proposed probability vector by both its coordinates and its total mass.
- For \(Q=\begin{bmatrix}-4&4\\2&-2\end{bmatrix}\), solve \(\pi Q=0\) and normalize \(\pi\).
- Explain why the implication from \(\pi Q=0\) to \(\pi P(t)=\pi\) uses the finite-state matrix exponential.
- For the three-state cycle, verify the second coordinate of \(\pi Q\) using \(\pi=(\tfrac3{11},\tfrac6{11},\tfrac2{11})\).
- For the absorbing-state example, what invariant probability results when \(a=0\), and what does it place mass on?
- Does \(\pi Q=0\) prove that a chain started from every state converges to \(\pi\)? Explain the distinction.