Stationarity on a Countable Birth-Death Chain
For a finite chain, stationary probabilities are found by solving a finite system of balance equations. On a countable state space, the equations are still local for a nearest-neighbor chain, but there are infinitely many of them. The reflecting boundary at state \(0\) provides a crucial constraint: it forces the net probability flow across the first edge to be zero. The stationary equations then force zero net flow across every edge, giving a simple product formula. Whether that formula describes a stationary distribution depends on whether its total mass is finite.
Consider a discrete-time Markov chain \((X_n)_{n\ge0}\) on the state space \(S=\{0,1,2,\ldots\}\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). From state \(i\), let \(a_i\) be the probability of moving to \(i+1\), and, for \(i\ge1\), let \(b_i\) be the probability of moving to \(i-1\). Set \(b_0=0\), and let \(r_i=1-a_i-b_i\) be the probability of staying at \(i\). Assume \(a_i>0\) for every \(i\ge0\), \(b_i>0\) for every \(i\ge1\), and \(a_i+b_i\le1\). Thus the chain is irreducible, and the boundary at \(0\) is reflecting: it has no transition to a state below \(0\). The transition matrix \(P=(p_{ij})\) has \(p_{i,i+1}=a_i\), \(p_{i,i-1}=b_i\) when \(i\ge1\), \(p_{ii}=r_i\), and all other entries zero. Each row sums to \(a_i+b_i+r_i=1\).
A stationary distribution is a row vector \(\pi=(\pi_0,\pi_1,\ldots)\) with \(\pi_i\ge0\), \(\sum_{i\ge0}\pi_i=1\), and \(\pi P=\pi\). In particular, its coordinate equations are \(\pi_j=\sum_{i\ge0}\pi_i p_{ij}\). An invariant measure satisfies the same coordinate equations and nonnegative-mass condition, but need not have total mass one.
Why the Stationary Equations Give a Product
The stationary equation at state \(j\ge1\) accounts for transitions from its two neighbors and for the probability of staying there: $$ \pi_j=\pi_{j-1}a_{j-1}+\pi_jr_j+\pi_{j+1}b_{j+1}. $$ At state \(0\), only a stay at \(0\) or a move from \(1\) can enter: $$ \pi_0=\pi_0r_0+\pi_1b_1. $$ To obtain the product formula correctly, it is important to track flow across edges rather than rearrange the equations as if the birth and death probabilities at one state were the same.
Proof. For \(i\ge0\), define the net flow across the edge from \(i\) to \(i+1\) by \(J_i=\pi_i a_i-\pi_{i+1}b_{i+1}\). For any stationary measure \(\pi\), the equation at \(j\ge1\), together with \(r_j=1-a_j-b_j\), gives $$ \pi_{j-1}a_{j-1}+\pi_{j+1}b_{j+1}=\pi_j(a_j+b_j). $$ Rearranging this equality gives $$ \pi_{j-1}a_{j-1}-\pi_jb_j=\pi_ja_j-\pi_{j+1}b_{j+1}, $$ so \(J_{j-1}=J_j\) for each \(j\ge1\). Thus all edge flows have the same value. At the boundary, \(r_0=1-a_0\), and the equation at \(0\) gives \(\pi_0a_0=\pi_1b_1\), so \(J_0=0\). Consequently \(J_i=0\) for every \(i\ge0\), or \(\pi_i a_i=\pi_{i+1}b_{i+1}\). Since all the \(a_i\) and \(b_{i+1}\) are positive, iteration yields \(\pi_n=\pi_0w_n\). A stationary distribution must have \(\pi_0>0\): if \(\pi_0=0\), this recurrence forces every \(\pi_n=0\), contradicting total mass one. Normalization therefore requires \(1=\pi_0 Z\), which is possible exactly when \(Z<\infty\); in that case \(\pi_0=1/Z\) and the stated formula is necessary. Conversely, if \(Z<\infty\), the weights \(\pi_n=w_n/Z\) sum to one and satisfy \(\pi_i a_i=\pi_{i+1}b_{i+1}\) on every edge. For \(j\ge1\), these identities give $$ \pi_{j-1}a_{j-1}+\pi_{j+1}b_{j+1} =\pi_jb_j+\pi_ja_j, $$ which, after adding \(\pi_jr_j\), is the stationary equation at \(j\). The identity on the first edge gives the equation at \(0\) as well. This proves sufficiency, and the necessary formula proves uniqueness. \(\square\)
The weights are determined by the ratios of birth to death probabilities on adjacent states. The probabilities of holding do not appear in the formula, although they affect how the chain moves in time. A finite \(Z\) gives a probability distribution; an infinite \(Z\) still leaves positive invariant weights, but they cannot be normalized to have total mass one.
Worked Example: A Reflecting Chain with Geometric Stationary Probabilities
Let \(a_i=\tfrac13\) for all \(i\ge0\), let \(b_i=\tfrac23\) for \(i\ge1\), and let \(b_0=0\). At state \(0\), \(r_0=1-\tfrac13=\tfrac23\); at each state \(i\ge1\), \(r_i=1-\tfrac13-\tfrac23=0\). Thus the boundary row has entries \(\tfrac23,\tfrac13\), and every other row has entries \(\tfrac23,\tfrac13\) in the columns for moving down and up; each row sums to \(1\).
For every \(k\ge0\), \(a_k/b_{k+1}=(\tfrac13)/(\tfrac23)=\tfrac12\). Hence \(w_0=1\), and for \(n\ge1\), $$ w_n=\left(\frac12\right)^n. $$ The total weight is the geometric sum \(Z=\sum_{n=0}^{\infty}(\tfrac12)^n=2\), so \(\pi_n=\tfrac12(\tfrac12)^n=2^{-(n+1)}\).
Check: The masses sum to \(\sum_{n=0}^{\infty}2^{-(n+1)}=1\). At state \(0\), the incoming mass is \(\pi_0r_0+\pi_1b_1=(\tfrac12)(\tfrac23)+(\tfrac14)(\tfrac23)=\tfrac13+\tfrac16=\tfrac12=\pi_0\). At any state \(j\ge1\), the incoming mass is \(\pi_{j-1}(\tfrac13)+\pi_{j+1}(\tfrac23)\). Since \(\pi_{j-1}=2\pi_j\) and \(\pi_{j+1}=\tfrac12\pi_j\), this is \((\tfrac23)\pi_j+(\tfrac13)\pi_j=\pi_j\).
Worked Example: Summable Weights with State-Dependent Death Probabilities
Set \(a_i=\tfrac14\) for every \(i\ge0\), and for \(i\ge1\) set \(b_i=\tfrac14\,\tfrac{i+2}{i}\), with \(b_0=0\). The first death probabilities are \(b_1=\tfrac34\), \(b_2=\tfrac12\), and \(b_3=\tfrac5{12}\). At state \(0\), \(r_0=\tfrac34\), so that row sums to \(\tfrac14+\tfrac34=1\). At state \(i\ge1\), \(a_i+b_i=\tfrac14+\tfrac14\,\tfrac{i+2}{i}=\tfrac12+\tfrac1{2i}\le1\), since \(i\ge1\); therefore \(r_i=\tfrac12-\tfrac1{2i}\ge0\), and each row sums to \(1\).
For \(i\ge0\), the death probability at \(i+1\) is \(b_{i+1}=\tfrac14\,\tfrac{i+3}{i+1}\). Thus $$ \frac{a_i}{b_{i+1}}=\frac{\tfrac14}{\tfrac14\,\tfrac{i+3}{i+1}}=\frac{i+1}{i+3}. $$ Multiplying the ratios from \(i=0\) through \(i=n-1\) gives $$ w_n=\prod_{i=0}^{n-1}\frac{i+1}{i+3} =\frac{1\cdot2}{(n+1)(n+2)} =\frac{2}{(n+1)(n+2)}. $$ This also gives \(w_0=1\) when \(n=0\). Since \(\frac{1}{(n+1)(n+2)}=\frac1{n+1}-\frac1{n+2}\), the partial sum through \(n=N\) is \(2(1-\tfrac1{N+2})\), which tends to \(2\). Therefore \(Z=2\) and \(\pi_n=\tfrac1{(n+1)(n+2)}\).
Check: The partial sums of \(\pi\) are \(1-\tfrac1{N+2}\), tending to \(1\). On the edge from \(i\) to \(i+1\), \(\pi_i a_i=\tfrac1{(i+1)(i+2)}\cdot\tfrac14\), while $$ \pi_{i+1}b_{i+1} =\frac{1}{(i+2)(i+3)}\cdot\frac14\frac{i+3}{i+1} =\frac{1}{4(i+1)(i+2)}. $$ The edge flows agree. The theorem’s proof then ensures every stationary equation holds, including the boundary equation.
Worked Example: Positive Invariant Weights That Cannot Be Normalized
Let \(a_i=\tfrac13\) for every \(i\ge0\), and \(b_i=\tfrac13\,\tfrac{i+1}{i}\) for \(i\ge1\), with \(b_0=0\). Here \(b_1=\tfrac23\), and at state \(i\ge1\), $$ a_i+b_i=\frac13+\frac13\frac{i+1}{i}=\frac23+\frac1{3i}\le1. $$ At \(i=1\) the sum is \(1\); at larger \(i\) it is less than \(1\). Also \(a_0=\tfrac13\), so \(r_0=\tfrac23\). All rows are probability distributions: their nonzero entries are the specified birth and death probabilities and the remaining nonnegative holding probability.
For \(i\ge0\), \(b_{i+1}=\tfrac13\,\tfrac{i+2}{i+1}\), so $$ \frac{a_i}{b_{i+1}}=\frac{i+1}{i+2}. $$ The product telescopes to \(w_n=1/(n+1)\), including \(w_0=1\). Its total mass diverges: in each block \(2^m\le n+1<2^{m+1}\), there are \(2^m\) terms, each at least \(1/2^{m+1}\), so the block contributes at least \(\tfrac12\). There are infinitely many disjoint such blocks, hence \(Z=\infty\) and no stationary distribution exists.
Check: The positive weights \(w_n=1/(n+1)\) do satisfy the edge identities: \(w_i a_i=1/[3(i+1)]\) and $$ w_{i+1}b_{i+1} =\frac1{i+2}\cdot\frac13\frac{i+2}{i+1} =\frac1{3(i+1)}. $$ Thus the failure is not a failure of the local stationary equations; it is the impossibility of scaling these weights to total mass one.
Interpreting the Criterion
The theorem separates two tasks. The stationary equations determine the relative masses \(w_n\); summability decides whether those relative masses can be made into probabilities. If \(Z<\infty\), the chain has a stationary distribution given by the normalized weights. If \(Z=\infty\), no stationary distribution exists under the stated assumptions, even though the same product still gives an invariant measure. In the language of recurrence classification, the irreducible chain is positive recurrent exactly when this stationary distribution exists; that connection is a standard result from the earlier material on positive recurrence and stationary distributions.
A common error is to write \(\pi_{j+1}b_{j+1}-\pi_ja_j=\pi_ja_j-\pi_{j-1}a_{j-1}\) and then claim the boundary equation makes the right-hand side zero. The boundary equation instead gives \(\pi_1b_1=\pi_0a_0\); it does not assert \(\pi_1a_1=\pi_0a_0\). The correct reasoning is to define \(J_i=\pi_i a_i-\pi_{i+1}b_{i+1}\), derive \(J_{j-1}=J_j\) from the equation at \(j\ge1\), and use the equation at \(0\) to establish \(J_0=0\). This fixes every edge flow at zero without conflating the birth and death probabilities.
When applying the criterion, check the boundary separately, verify that the specified transition probabilities give valid rows, form each adjacent ratio \(a_i/b_{i+1}\), and only then test the resulting series for convergence. A product formula alone is not evidence that a stationary probability distribution exists: normalization is an essential part of the conclusion.
Check Your Understanding
For each question, distinguish the local balance equations from the requirement that total mass be one.
- For a reflecting birth-death chain, write the stationary equation at state \(0\) and explain why it forces zero flow across the edge from \(0\) to \(1\).
- If \(a_i=\tfrac14\) and \(b_{i+1}=\tfrac12\) for every \(i\ge0\), calculate \(w_n\) and determine whether the weights can be normalized.
- Suppose the product weights satisfy every edge identity but have infinite sum. What can be concluded about an invariant measure, and what can be concluded about a stationary distribution?
- For \(a_i=\tfrac13\) and \(b_{i+1}=\tfrac13\,\tfrac{i+2}{i+1}\), calculate \(w_1,w_2,w_3\) directly from the ratios and compare them with the product formula.
- Why does the argument require \(a_i>0\) and \(b_{i+1}>0\) to derive the stated ratio formula and uniqueness conclusion?