Pitfalls in Infinite-State Generator Equations

Why an Infinite-State Balance Equation Needs a Check

For a finite-state continuous-time Markov chain, solving $\pi Q=0$ and normalizing the resulting vector is a standard route to a stationary distribution. On a countable state space, the same algebra can conceal two different problems: an infinite sum may not be rearrangeable, and a formal solution of the balance equations may not describe the actual transition probabilities. In particular, explosion can carry probability away from the states on which the equations were written.

Let $(X_t)_{t\ge0}$ be a continuous-time Markov chain on a countable state space $S$, with transition semigroup $P(t)=(p_{ij}(t))$ and generator $Q=(q_{ij})$. The rows of a conservative generator sum to zero: $q_{ij}\ge0$ for $j\ne i$, and $q_{ii}=-q_i$, where $q_i=\sum_{j\ne i}q_{ij}$. A probability row vector $\pi$ is invariant if $\pi P(t)=\pi$ for every $t\ge0$. The coordinate balance equation at state $j$ is $\sum_i\pi_iq_{ij}=0$, when that sum is defined.

Definition: For a transition semigroup acting on bounded functions, its generator $A$ has domain $D(A)$ consisting of the bounded functions $f$ for which $(P(t)f-f)/t$ has a limit in the supremum norm as $t\downarrow0$. The limit is $Af$. Pointwise, the formal generator expression is $Qf(i)=\sum_jq_{ij}f(j)$, when this sum exists. For unbounded rates, a pointwise expression need not define an element of $D(A)$.

The distinction matters because invariance is a statement about $P(t)$ for all times, whereas $\pi Q=0$ is an infinitesimal statement. Passing between them requires conditions that justify the derivative and the sums involved. The following result gives a setting where the passage is valid.

Theorem (bounded-rate criterion): Suppose $S$ is countable, $Q$ is conservative, and $\sup_i q_i\le\Lambda<\infty$. For a probability row vector $\pi$, the coordinate equations $\pi Q=0$ are equivalent to invariance, $\pi P(t)=\pi$ for every $t\ge0$.

Proof. If $\Lambda=0$, every rate is zero, $Q=0$, and $P(t)=I$; both statements hold for every probability vector. Now suppose $\Lambda>0$. Define $R=I+Q/\Lambda$. Its off-diagonal entries are nonnegative, its diagonal entries are $1-q_i/\Lambda\ge0$, and each row sums to $1$, so $R$ is a transition matrix. The bounded-rate chain can be constructed by taking potential transition times from a Poisson process of rate $\Lambda$ and, at each such time, updating the state according to $R$; the self-transitions in $R$ allow a potential time to make no state change. Consequently, its transition semigroup is $$ P(t)=e^{-\Lambda t}\sum_{m=0}^{\infty}\frac{(\Lambda t)^m}{m!}R^m. $$ If $\pi Q=0$, then $\pi R=\pi+\pi Q/\Lambda=\pi$, so $\pi R^m=\pi$ for every nonnegative integer $m$. Substitution into the series gives $\pi P(t)=\pi$. Conversely, if $\pi P(t)=\pi$, then $\pi(P(t)-I)/t=0$. Since the rates are bounded, the displayed series gives the generator limit in the space of summable row vectors, with generator $Q=\Lambda(R-I)$. Taking the limit as $t\downarrow0$ yields $\pi Q=0$. The sums defining $\pi Q$ are legitimate: $\sum_i\pi_iq_i\le\Lambda$, and the total absolute sum of the entries of $\pi Q$ is at most $2\Lambda$. $\square$

A Countable Example Where Balance Works

Worked Example: A Reflected Birth–Death Chain with Bounded Rates

Take $S=\{0,1,2,\ldots\}$. From state $0$, the chain jumps to $1$ at rate $\tfrac12$. From each state $n\ge1$, it jumps to $n+1$ at rate $\tfrac12$ and to $n-1$ at rate $1$. Thus $q_0=\tfrac12$, and $q_n=\tfrac32$ for $n\ge1$; the generator rows sum to zero.

Try $\pi_n=2^{-(n+1)}$. Its total mass is $\sum_{n\ge0}2^{-(n+1)}=1$. At state $0$, incoming rate-weighted mass is $\pi_1\cdot1=\tfrac14$, and outgoing rate-weighted mass is $\pi_0\cdot\tfrac12=\tfrac14$. For $j\ge1$, incoming mass is $\pi_{j-1}\cdot\tfrac12+\pi_{j+1}\cdot1$. Since $\pi_{j-1}=2\pi_j$ and $\pi_{j+1}=\tfrac12\pi_j$, this equals $\pi_j+\tfrac12\pi_j=\tfrac32\pi_j$, exactly the outgoing mass $\pi_jq_j$. Hence every coordinate of $\pi Q$ is zero.

Check the criterion. The maximum exit rate is $\Lambda=\tfrac32$. The associated matrix $R=I+Q/\Lambda$ has row $0$ equal to $(\tfrac23,\tfrac13,0,\ldots)$; for $n\ge1$, row $n$ assigns $\tfrac23$ to $n-1$, $\tfrac13$ to $n+1$, and zero to $n$. At coordinate $0$, $\sum_i\pi_iR_{i0}=\tfrac12\cdot\tfrac23+\tfrac14\cdot\tfrac23=\tfrac12=\pi_0$. At coordinate $j\ge1$, the incoming mass is $\pi_{j-1}\cdot\tfrac13+\pi_{j+1}\cdot\tfrac23=\tfrac23\pi_j+\tfrac13\pi_j=\pi_j$. Thus $\pi R=\pi$, and the theorem gives $\pi P(t)=\pi$ for all $t$. This verifies invariance, not merely the formal balance equations.

Explosion Can Defeat a Normalized Formal Solution

A rate matrix can have conservative rows while its minimal chain is explosive: infinitely many jumps can occur in finite time. If the process is sent to a cemetery state $\Delta$ at explosion, the transition probabilities on the original states lose mass before that time. Balance equations written only for the original states do not record this loss.

Worked Example: A Formal Probability That Loses Mass at Explosion

Let $S=\mathbb{Z}\cup\{\Delta\}$. For each $n\in\mathbb{Z}$, let the only jump from $n$ be to $n+1$, at rate $q_n=2^{|n|}$. Let $\Delta$ be absorbing, with $q_{\Delta\Delta}=0$, and send the chain to $\Delta$ if infinitely many jumps occur in finite time. Each row of $Q$ sums to zero: $q_{n,n+1}=2^{|n|}$, $q_{nn}=-2^{|n|}$, and all other entries in that row are zero.

Define $\pi_\Delta=0$ and $\pi_n=1/(3\cdot2^{|n|})$. The normalizing constant follows from $\sum_{n\le0}2^{-|n|}=\sum_{n\le0}2^n=2$ and $\sum_{n\ge1}2^{-|n|}=\sum_{n\ge1}2^{-n}=1$; therefore $\sum_n\pi_n=(2+1)/3=1$. At any $j\in\mathbb{Z}$, the only incoming transition comes from $j-1$, so \[ (\pi Q)_j=\pi_{j-1}2^{|j-1|}-\pi_j2^{|j|}=\tfrac13-\tfrac13=0. \] At $\Delta$, there are no incoming jumps in a finite number of transitions, so $(\pi Q)_\Delta=0$ as well.

Nevertheless, starting from any integer $n$, the time to explosion is a sum of independent exponential holding times with means $2^{-|k|}$ over $k=n,n+1,\ldots$. The sum of these means is finite: it has only finitely many terms with $k\le0$, and its positive-index tail is a geometric series. The explosion time therefore has finite expectation and is finite almost surely. Under initial distribution $\pi$, explosion also occurs almost surely. Hence for some finite $t$, $\mathbb{P}_{\pi}(\text{explosion by }t)>0$, and $(\pi P(t))_\Delta>0=\pi_\Delta$. Thus $\pi P(t)\ne\pi$, despite the normalized coordinatewise solution of $\pi Q=0$.

Check the failed summation. The total absolute sum of the entries in the rows weighted by $\pi$ is $\sum_{n\in\mathbb Z}\pi_n(2q_n)=\sum_{n\in\mathbb Z}\tfrac23=\infty$. So one cannot sum all coordinate equations and rearrange the resulting double series as if it were absolutely convergent. The formal cancellation at each state has not accounted for probability flowing to $\Delta$ through infinitely many jumps.

Generator Domains and Invalid Differentiation

Even without using an explosion counterexample, a generator calculation must respect its domain. For a bounded function $f$, the formal expression $Qf(i)$ may be finite at each state while $Qf$ is unbounded. In that case $f$ cannot belong to the supremum-norm generator domain defined above: a limit in supremum norm of bounded functions must itself be bounded. It is then invalid to invoke the generator identity for that $f$ without additional justification.

Worked Example: A Bounded Function Outside the Generator Domain

Consider the pure-death chain on $S=\{0,1,2,\ldots\}$ that jumps from $n\ge1$ to $n-1$ at rate $n$, with $0$ absorbing. Each row of its generator sums to zero: $q_{n,n-1}=n$, $q_{nn}=-n$ for $n\ge1$, and the row at $0$ is zero. This chain is non-explosive, since a path starting at $n$ can make at most $n$ jumps.

Set $f(n)=(-1)^n$, a bounded function with supremum norm $1$. The formal generator expression is $Qf(0)=0$, and for $n\ge1$, \[ Qf(n)=n\bigl(f(n-1)-f(n)\bigr) =n\bigl((-1)^{n-1}-(-1)^n\bigr) =-2n(-1)^n. \] Its absolute value is $2n$, which is unbounded. Therefore $f\notin D(A)$: if the difference quotients converged in supremum norm, their limit $Af$ would be bounded, whereas their pointwise generator values are unbounded. The chain itself is non-explosive; the failure here is specifically a domain failure, not a failure of the sample paths to exist for all times.

Check: At $n=1$, $Qf(1)=2$; at $n=2$, $Qf(2)=-4$; and at $n=3$, $Qf(3)=6$. These values follow directly from the formula and confirm that no bounded function can equal $Qf$ on all states.

A Practical Validity Check

When a proposed probability vector is obtained from infinite-state generator equations, separate the algebraic calculation from the claim of invariance. First verify that $\pi$ is nonnegative and sums to one, and that every generator row has the required signs and sums to zero. Then inspect how each coordinate sum $\sum_i\pi_iq_{ij}$ is defined and whether the sums can be rearranged. The stronger bound $\sum_i\pi_iq_i<\infty$ ensures absolute summability of the full weighted generator array, since $\sum_{i,j}\pi_i|q_{ij}|=2\sum_i\pi_iq_i$. It does not by itself replace a theorem connecting the generator equations to the transition semigroup.

Finally, check the process itself. If it can explode, a conservative-looking row of $Q$ does not guarantee that probability remains among the ordinary states at every time. If the rates are bounded, the criterion proved above supplies a direct justification. With unbounded rates, identify a theorem whose hypotheses apply, including the relevant generator domain and summability conditions; do not infer invariance from coordinate algebra alone.

Common mistake. A tempting calculation is $\sum_j\sum_i\pi_iq_{ij}=\sum_i\pi_i\sum_jq_{ij}=0$. The second equality uses the zero row sums of $Q$, but the exchange of sums is valid only with suitable convergence, such as absolute summability. In the explosive example, the absolute sum is infinite. The correction is to establish a convergence condition before rearranging, and separately to verify non-explosion or another valid invariance criterion. A formal cancellation is not a proof that probability is preserved.

Check Your Understanding

For each question, distinguish a coordinatewise generator calculation from a statement about the transition semigroup.

In the bounded-rate theorem, why does $\pi Q=0$ imply $\pi R=\pi$, and why does that imply invariance for every $t\ge0$?
For the reflected birth–death example, compute $\pi_2$ and verify the coordinate balance equation at state $2$.
In the explosive example, compute $(\pi Q)_j$ from its one incoming and one outgoing term. What feature of the path behavior still prevents invariance?
For the pure-death chain, evaluate $Qf(4)$ for $f(n)=(-1)^n$. Does the chain's non-explosion place $f$ in the generator domain?
State a summability condition that permits rearranging the weighted generator array, and explain why the condition is relevant to a formal sum over all states.