Solving Finite-State Stationary Equations

From Transition Rates to Stationary Equations

For a finite-state continuous-time Markov chain, finding a stationary distribution is a finite-dimensional linear algebra problem. The transition rates determine a generator matrix \(Q\); a stationary distribution, written as a row vector, must satisfy \(\pi Q=0\) and have entries summing to one. The main work is setting up the equations with the correct orientation, solving them, and checking the result.

Throughout, let \((X_t)_{t\ge0}\) be a continuous-time Markov chain on a finite state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). When conditioning on the past, use the natural filtration \(\mathcal{F}_t=\sigma(X_s:0\le s\le t)\). Its generator is \(Q=(q_{ij})\): for distinct states \(i,j\), \(q_{ij}\ge0\) is the rate of a jump from \(i\) to \(j\), and \(q_{ii}=-\sum_{j\ne i}q_{ij}\). Thus each row sums to zero. The transition semigroup is \(P(t)=e^{tQ}\), and distributions are row vectors: an initial distribution \(\mu\) evolves as \(\mu P(t)\).

For the examples below, all chains are finite and irreducible: every state can be reached from every other state by a path of positive-rate transitions. The standard finite-state CTMC result established in the earlier material on stationary distributions says that such a chain has a unique stationary distribution. Consequently, if the normalized generator equations have a unique solution, that solution must be the stationary distribution: a stationary distribution exists, it satisfies the equations, and uniqueness of the normalized solution identifies it. We use this result to identify answers without proving the converse implication \(\pi Q=0\Rightarrow\pi P(t)=\pi\).

Why the Equations Have a Redundancy

For a row vector \(\pi\), the \(j\)th component of \(\pi Q\) is \(\sum_i\pi_iq_{ij}\). The equation for state \(j\) says that the total stationary rate of flow into \(j\), including the diagonal term, balances to zero. In a finite generator, the sum of all components of \(\pi Q\) is zero for every row vector \(\pi\), because

Therefore, once all but one of the component equations hold, the final one holds as well. In practice, solve a convenient subset of the equations and use \(\sum_i\pi_i=1\) to set the scale. For a two-state chain this leaves one balance equation; for three states, it usually leaves two.

Proof. The generator is \(Q=\begin{bmatrix}-\alpha&\alpha\\ \beta&-\beta\end{bmatrix}\). Write \(\pi=(x,y)\), where \(x,y\ge0\) and \(x+y=1\). The first component of \(\pi Q=0\) gives \(-\alpha x+\beta y=0\), so \(\beta y=\alpha x\). Substituting \(y=1-x\) gives \(\beta(1-x)=\alpha x\), hence \((\alpha+\beta)x=\beta\) and \(x=\frac{\beta}{\alpha+\beta}\). Then \(y=1-x=\frac{\alpha}{\alpha+\beta}\). Since \(\alpha+\beta>0\), these values are uniquely determined, nonnegative, and sum to one. The finite irreducible CTMC result stated above identifies this unique solution with the stationary distribution. \(\square\)

Solving a Three-State System

For three states, write one equation for each column of \(Q\), rather than reading rows as equations. If \(\pi=(x,y,z)\), then the equations are the components of \((x,y,z)Q\). Since their sum is automatically zero, only two of the three are independent. A reliable procedure is to write the generator, check its row sums, solve two component equations together with normalization, and substitute into all three components to verify the result.

Proof. By the finite irreducible CTMC result stated above, a stationary distribution \(\widehat{\pi}\) exists. It is normalized and satisfies \(\widehat{\pi}Q=0\), so it solves the displayed system. If that system has exactly one solution, the solution computed from it must equal \(\widehat{\pi}\). Therefore the unique solution is the stationary distribution. \(\square\)

Common Pitfall: Multiplying on the Wrong Side

Because distributions are row vectors in this course, the stationary equations are \(\pi Q=0\), not \(Q\pi=0\). These are different systems. For the two-state example, the incorrect right-nullvector calculation \(Qv=0\) gives \(-2v_1+2v_2=0\), so \(v_1=v_2\); after normalization it suggests \((\frac12,\frac12)\). But this is not stationary: \((\frac12,\frac12)Q=(\frac12,-\frac12)\ne(0,0)\). The correct left-vector calculation gives \((\frac35,\frac25)Q=(0,0)\). Always check the product in the same orientation as the distribution convention.

A second useful check is to distinguish generator entries from transition probabilities. Off-diagonal entries of \(Q\) are rates and need not sum to one; the diagonal entry is the negative total rate out of that state, making the whole row sum to zero. A generator is not a transition matrix. Finally, satisfying the equations is not the same algebraic operation as normalizing: verify both \(\pi Q=0\) and \(\sum_i\pi_i=1\), and check that all entries are nonnegative.