Generator Equations for Stationarity

From Stationarity to a Generator Equation

For a continuous-time Markov chain, stationarity means that its distribution does not change as time passes. The generator records the instantaneous rates of those changes. In a finite state space, differentiating the stationarity condition at time zero gives a compact system of linear equations. Each component of that system says that, at equilibrium, the rate of probability flowing into a state matches the rate flowing out.

Let \((X_t)_{t\ge0}\) be a continuous-time Markov chain on a finite or countable state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with natural filtration \(\mathcal{F}_t=\sigma(X_s:0\le s\le t)\). Write \(P(t)=(p_{ij}(t))\), where \(p_{ij}(t)=\mathbb{P}_i(X_t=j)\). Its generator is \(Q=(q_{ij})\), with \(q_{ij}=\lim_{h\downarrow0}(p_{ij}(h)-\mathbf{1}_{\{i=j\}})/h\) when the limit exists. For \(i\ne j\), \(q_{ij}\ge0\) is the instantaneous rate from \(i\) to \(j\); the diagonal entry is \(q_{ii}=-q_i\), where \(q_i=\sum_{j\ne i}q_{ij}\) is the total rate of leaving \(i\). Thus each generator row sums to zero.

The probability-space definition of stationarity refers to the distribution at each time: if \(X_0\) has distribution \(\pi\), then \(X_t\) has distribution \(\pi P(t)\). The equation \(\pi P(t)=\pi\) is the invariance condition. The generator equations are obtained by differentiating this condition; for finite \(S\), all sums are finite and the differentiation is justified directly.

Proof. Since \(S\) is finite, \(Q\) is a finite matrix and \(P(t)=e^{tQ}\). By the power-series definition of the matrix exponential, \(P(t)=I+tQ+R(t)\), where every entry of \(R(t)/t\) tends to zero as \(t\downarrow0\). Stationarity gives \(\pi P(t)=\pi\), so \((\pi P(t)-\pi)/t=0\) for every \(t>0\). Substituting the expansion gives \(0=\pi Q+\pi R(t)/t\). Taking \(t\downarrow0\) yields \(\pi Q=0\). \(\square\)

Reading the Equations as Rate Balances

Because distributions are row vectors, the \(j\)th component of \(\pi Q\) is \(\sum_{i\in S}\pi_iq_{ij}\). Separating the diagonal term, and using \(q_{jj}=-q_j\), gives a useful state-by-state interpretation.

The left side is the total rate of probability flow into state \(j\): the chain is in state \(i\) with stationary probability \(\pi_i\), and moves from \(i\) to \(j\) at rate \(q_{ij}\). The right side is the rate of flow out of \(j\): the chain is in \(j\) with probability \(\pi_j\), and leaves at rate \(q_j\). The equality is a balance of rates, not an assertion that the probability of entering \(j\) in a time interval equals the probability of leaving it on every individual path.

Proof. For any \(j\), the definition of matrix multiplication and the generator diagonal give \((\pi Q)_j=\sum_{i\ne j}\pi_iq_{ij}+\pi_jq_{jj}=\sum_{i\ne j}\pi_iq_{ij}-\pi_jq_j\). Thus \((\pi Q)_j=0\) exactly when the stated balance holds at \(j\). Applying this identity to each state proves both directions. \(\square\)

The balance equations are necessary for stationarity in a finite state space. They are often used to find candidate stationary probabilities, but the normalization and nonnegativity conditions remain essential. The calculations below illustrate how to form and check the equations; the question of when generator equations alone ensure invariance requires additional care, especially on countable spaces.

Why Countable State Spaces Need Care

For countable \(S\), the component expression \((\pi Q)_j=\sum_i\pi_iq_{ij}\) may involve an infinite sum. The finite-state proof used a finite matrix expansion and finite sums; it cannot simply be repeated without checking convergence and whether differentiation can be interchanged with summation. Even if each individual rate \(q_i\) is finite, the stationary average exit rate \(\sum_i\pi_iq_i\) need not be finite. Also, summing component equations over infinitely many states can discard or conceal probability flow at the boundary “at infinity.”

One useful sufficient condition for the differentiation step is a uniform bound on exit rates: \(\sup_{i\in S}q_i<\infty\). In that case \(Q\) is a bounded operator on summable row vectors, and its exponential series converges in operator norm. If \(\pi P(t)=\pi\), the same difference-quotient argument as in the finite case gives \(\pi Q=0\). Without a condition that justifies the operation, the coordinate equations should not be obtained by formally differentiating an infinite sum. This is a statement about the derivation; it does not assert that bounded rates are necessary for a stationary distribution to satisfy suitable balance equations.

A countable example shows both how the equations can work and why their normalization must be checked. Consider the immigration-death chain on \(S=\{0,1,2,\ldots\}\), with immigration rate \(2\) from \(n\) to \(n+1\), and death rate \(n\) from \(n\) to \(n-1\) for \(n\ge1\). Its exit rate is \(q_0=2\) and \(q_n=2+n\) for \(n\ge1\), so exit rates are not uniformly bounded. The candidate probabilities \(\pi_n=e^{-2}2^n/n!\) are normalized because \(\sum_{n=0}^{\infty}2^n/n!=e^2\). For \(n=0\), the component balance is \(\pi_1\cdot1=\pi_0\cdot2\), since \(\pi_1=2\pi_0\). For \(n\ge1\), incoming flow is \(\pi_{n-1}\cdot2+\pi_{n+1}(n+1)\), while outgoing flow is \(\pi_n(2+n)\). The identities \(\pi_{n-1}\cdot2=n\pi_n\) and \(\pi_{n+1}(n+1)=2\pi_n\) make both sides equal to \((n+2)\pi_n\). These coordinate balances are well-defined and verify the equations state by state. They do not license arbitrary rearrangements of an infinite sum of rates.

The distinction between coordinate equations and a global sum is important. In this example, the expected exit rate under \(\pi\) is \(\sum_n\pi_n(2+n)=4\), which is finite, but in general that quantity needs its own check. Another warning comes from the critical birth-death chain on the nonnegative integers with upward rate \(1\) from every state and downward rate \(1\) from every positive state. The formal balance equations admit constant weights: at zero, \(\pi_1=\pi_0\), and for \(n\ge1\), \(\pi_{n-1}+\pi_{n+1}=2\pi_n\). A nonzero constant sequence cannot be normalized to sum to \(1\), so those formal solutions are not stationary probability distributions. Equations must be paired with the probability constraints and the appropriate conditions for the countable-state chain.

Common mistake. A frequent error is to omit the diagonal outflow term and equate only the off-diagonal entries of a generator column. In the two-state example, this would incorrectly write the state-\(0\) equation as \(2\pi_1=0\), since \(q_{10}=2\) is the only off-diagonal inflow. The correction is to include the diagonal contribution \(\pi_0q_{00}=-3\pi_0\): the actual equation is \(2\pi_1-3\pi_0=0\). Equivalently, incoming rate \(\pi_1\cdot2\) must equal outgoing rate \(\pi_0\cdot3\). The diagonal records total departure and cannot be dropped.