Stationary Probabilities In A Finite-Capacity Model

Finite Capacity Changes the Boundary, Not the Weighting Method

A finite-capacity model has only finitely many possible population levels. When the system is at its upper capacity, a birth cannot move it to a higher state. This boundary changes the generator, but the stationary probabilities can still be computed successively: each state's weight is the previous weight multiplied by the upward rate divided by the downward rate on that edge. The remaining task is to normalize all the weights, including the one at capacity.

Work on a probability space $(\Omega,\mathcal{F},\mathbb{P})$, and let $(X_t)_{t\ge0}$ be a continuous-time Markov chain with state space $S=\{0,1,\ldots,K\}$, where $K\ge1$. From state $i$, the chain moves to $i+1$ at rate $\lambda_i$ when $i<K$, and to $i-1$ at rate $\mu_i$ when $i>0$. Assume $\lambda_i>0$ for $0\le i<K$ and $\mu_i>0$ for $1\le i\le K$. Set $\lambda_K=0$ and $\mu_0=0$. The generator $Q=(q_{ij})$ has $q_{i,i+1}=\lambda_i$, $q_{i,i-1}=\mu_i$, and $q_{ii}=-(\lambda_i+\mu_i)$, with all other entries zero.

Definition: A probability row vector $\pi=(\pi_0,\ldots,\pi_K)$ is a stationary distribution if, when $X_0$ has distribution $\pi$, $X_t$ has distribution $\pi$ for every $t\ge0$. For this finite-state chain, the transition matrix is $P(t)=e^{tQ}$, so stationarity means $\pi P(t)=\pi$ for every $t\ge0$.

Stationarity is an invariance property, not by itself a claim about what happens from every initial distribution as time grows. For a finite-capacity chain, the product formula gives the stationary probabilities directly and makes the effect of the capacity explicit: the weight at $K$ is included, but no upward rate out of $K$ is used.

The Product-Weight Formula

Theorem: Under the positive-rate assumptions above, define $$ w_0=1,\qquad w_i=\prod_{r=0}^{i-1}\frac{\lambda_r}{\mu_{r+1}} \quad (1\le i\le K),\qquad Z_K=\sum_{i=0}^{K}w_i. $$ Then the unique stationary distribution is $\pi_i=w_i/Z_K$ for $0\le i\le K$. Equivalently, the weights can be computed successively using $w_{i+1}=w_i\lambda_i/\mu_{i+1}$ for $0\le i<K$.

Proof. First, $w_i>0$ for every $i$ by the positive-rate assumptions, so $Z_K$ is finite and positive and $\pi$ is a probability row vector. The product definition gives $w_{i+1}\mu_{i+1}=w_i\lambda_i$; hence $\pi_i\lambda_i=\pi_{i+1}\mu_{i+1}$ on every adjacent pair. For an interior state $j$, the $j$th component of $\pi Q$ is $\pi_{j-1}\lambda_{j-1}+\pi_{j+1}\mu_{j+1}-\pi_j(\lambda_j+\mu_j)$, which is zero by the two adjacent equalities. At state 0 it is $-\pi_0\lambda_0+\pi_1\mu_1=0$, and at state $K$ it is $\pi_{K-1}\lambda_{K-1}-\pi_K\mu_K=0$. Thus $\pi Q=0$. Since $P(t)=e^{tQ}=\sum_{m=0}^{\infty}t^mQ^m/m!$, the equality $\pi Q=0$ implies $\pi Q^m=0$ for every $m\ge1$, and consequently $\pi P(t)=\pi$. This proves that the displayed vector is stationary.

To prove uniqueness, let $\nu$ be any stationary probability row vector. The power-series definition of $P(t)$ gives $\lim_{t\downarrow0}(P(t)-I)/t=Q$. From $\nu P(t)=\nu$, dividing by $t$ and taking this limit gives $\nu Q=0$. For each edge from $i$ to $i+1$, write $J_i=\nu_i\lambda_i-\nu_{i+1}\mu_{i+1}$. The state-0 equation in $\nu Q=0$ gives $J_0=0$. At each interior state $i$, the equation is $J_{i-1}-J_i=0$, so induction gives $J_i=0$ for every $0\le i<K$. Therefore $\nu_{i+1}=\nu_i\lambda_i/\mu_{i+1}$, and repeated substitution yields $\nu_i=\nu_0w_i$. Since $\nu$ has total mass one, $\nu_0Z_K=1$, so $\nu_i=w_i/Z_K$. This proves uniqueness and the theorem. $\square$

The equalities $\pi_i\lambda_i=\pi_{i+1}\mu_{i+1}$ are called detailed balance across the edge joining $i$ and $i+1$: stationary flow in each direction along that edge agrees. The proof shows why these equalities work here, including at the boundaries. They should be checked against the actual rates on each edge, not guessed from a familiar special case.

Finite-capacity consequence: If the rates below capacity agree with those of a larger model, the finite model's unnormalized weights through $K$ are the same successive products. Its probabilities are obtained by normalizing only $w_0,\ldots,w_K$. At capacity, set the upward transition rate to zero; do not add an upward edge to the weight formula.

Worked Calculations

Worked Example: State-Dependent Rates with Capacity Three

Let $S=\{0,1,2,3\}$, with capacity $K=3$, birth rates $\lambda_0=3,\lambda_1=1,\lambda_2=1$, and death rates $\mu_1=2,\mu_2=2,\mu_3=6$. At capacity $\lambda_3=0$, and at state 0, $\mu_0=0$. Compute the stationary distribution.

The generator, in state order $0,1,2,3$, is $$ Q=\begin{bmatrix} -3&3&0&0\\ 2&-3&1&0\\ 0&2&-3&1\\ 0&0&6&-6 \end{bmatrix}. $$ Its row sums are $-3+3=0$, $2-3+1=0$, $2-3+1=0$, and $6-6=0$, as required for a generator.

Starting with $w_0=1$, compute each successive weight using the rate ratio on that edge: $$ w_1=1\cdot\frac{3}{2}=\frac32,\qquad w_2=\frac32\cdot\frac{1}{2}=\frac34,\qquad w_3=\frac34\cdot\frac{1}{6}=\frac18. $$ Thus $Z_3=1+\frac32+\frac34+\frac18=\frac{8+12+6+1}{8}=\frac{27}{8}$. Normalizing gives $$ \pi=\left(\frac{1}{27/8},\frac{3/2}{27/8},\frac{3/4}{27/8},\frac{1/8}{27/8}\right) =\left(\frac8{27},\frac{12}{27},\frac6{27},\frac1{27}\right). $$ The entries sum to $(8+12+6+1)/27=1$.

Check by edge flows: On the first edge, $\pi_0\lambda_0=(8/27)3=24/27$ and $\pi_1\mu_1=(12/27)2=24/27$. On the second, $\pi_1\lambda_1=12/27$ and $\pi_2\mu_2=(6/27)2=12/27$. On the third, $\pi_2\lambda_2=6/27$ and $\pi_3\mu_3=(1/27)6=6/27$. Each edge balances.

Worked Example: A Finite-Capacity Queue with State-Dependent Death Rates

Consider states $0,1,2,3$, with capacity 3, birth rate $\lambda_i=2$ for $i=0,1,2$, and death rate $\mu_i=i$ for $i=1,2,3$. Find the stationary probabilities, including the probability that the system is full.

Here $\lambda_3=0$, $\mu_0=0$, and the generator is $$ Q=\begin{bmatrix} -2&2&0&0\\ 1&-3&2&0\\ 0&2&-4&2\\ 0&0&3&-3 \end{bmatrix}. $$ The row sums are $-2+2=0$, $1-3+2=0$, $2-4+2=0$, and $3-3=0$. The successive weights are $$ w_0=1,\qquad w_1=1\cdot\frac21=2,\qquad w_2=2\cdot\frac22=2,\qquad w_3=2\cdot\frac23=\frac43. $$ Their sum is $Z_3=1+2+2+\frac43=\frac{3+6+6+4}{3}=\frac{19}{3}$. Therefore $$ \pi=\left(\frac3{19},\frac6{19},\frac6{19},\frac4{19}\right). $$ The probabilities sum to $(3+6+6+4)/19=1$; in particular, the probability of being at capacity is $\pi_3=4/19$.

Check: The three upward flows are $(3/19)2=6/19$, $(6/19)2=12/19$, and $(6/19)2=12/19$. The corresponding downward flows are $(6/19)1=6/19$, $(6/19)2=12/19$, and $(4/19)3=12/19$. Each pair agrees, confirming the normalized weights.

Worked Example: Constant Rates and the Full-State Probability

Take capacity $K=3$, birth rate $\lambda_i=2$ for $i=0,1,2$, and death rate $\mu_i=3$ for $i=1,2,3$. Compute the stationary distribution and the probability of being full.

The generator is $$ Q=\begin{bmatrix} -2&2&0&0\\ 3&-5&2&0\\ 0&3&-5&2\\ 0&0&3&-3 \end{bmatrix}. $$ Its row sums are $-2+2=0$, $3-5+2=0$, $3-5+2=0$, and $3-3=0$. The successive weights are $$ w_0=1,\qquad w_1=1\cdot\frac23=\frac23,\qquad w_2=\frac23\cdot\frac23=\frac49,\qquad w_3=\frac49\cdot\frac23=\frac8{27}. $$ The normalizing sum is $Z_3=1+\frac23+\frac49+\frac8{27}=\frac{27+18+12+8}{27}=\frac{65}{27}$. Hence $$ \pi=\left(\frac{27}{65},\frac{18}{65},\frac{12}{65},\frac8{65}\right), \qquad \pi_3=\frac8{65}. $$ The entries sum to $(27+18+12+8)/65=1$. The upward edge flows are $54/65,36/65,24/65$; the downward edge flows are $(18/65)3=54/65$, $(12/65)3=36/65$, and $(8/65)3=24/65$. Thus all three detailed-balance checks hold.

Common Mistake: Mixing Weights from Different Recursions

A weight calculation must use the same successive rule at every step. In the first example, the correct weights are $1,\frac32,\frac34,\frac18$, because the ratios are $\frac32,\frac12,\frac16$. If one mistakenly writes $w_2=\frac12$ but then retains $w_3=\frac18$, the list $1,\frac32,\frac12,\frac18$ is inconsistent: its last entry was not obtained by multiplying the stated $w_2$ by the next ratio. Its sum is $1+\frac32+\frac12+\frac18=\frac{25}{8}$, but normalizing this hybrid list does not produce a stationary distribution.

Propagating the mistaken $w_2=\frac12$ consistently instead gives $w_3=\frac12\cdot\frac16=\frac1{12}$, so the resulting incorrect weights would be $1,\frac32,\frac12,\frac1{12}$, with sum $1+\frac32+\frac12+\frac1{12}=\frac{37}{12}$. The correction is to return to the rate ratios: $w_2=\frac32\cdot\frac12=\frac34$, followed by $w_3=\frac34\cdot\frac16=\frac18$. Then $Z_3=\frac{27}{8}$, as computed above. This illustrates why checking each adjacent edge is more informative than checking the normalization alone: a list can sum to one after normalization and still use the wrong weights.

Two boundary checks prevent another common error. At state 0 there is no downward transition, and at capacity $K$ there is no upward transition to $K+1$. The boundary conventions $\mu_0=0$ and $\lambda_K=0$ encode these facts. A blocked arrival may be relevant when interpreting a physical system, but it does not create a transition to a new state and does not appear as an outgoing rate in this generator.

Check Your Understanding

For each question, keep the state order explicit and check the rates and normalization.

For capacity $K=2$, let $\lambda_0=4,\lambda_1=2$, $\mu_1=2,\mu_2=4$, with $\lambda_2=0$ and $\mu_0=0$. Find the three weights and the stationary distribution.
For the first worked example, calculate the stationary flow in each direction across the edge between states 1 and 2.
Why does the product formula include $w_K$ but not a factor involving an upward rate out of state $K$?
If a proposed list of weights satisfies the normalization after rescaling but fails one adjacent edge-flow equality, can it be stationary? Explain using the theorem's proof.
For constant birth rate $\lambda_i=1$ below capacity $K=2$ and death rates $\mu_1=1,\mu_2=2$, compute the weights, normalizing sum, and probability of being at capacity.

Stationary Probabilities In A Finite-Capacity Model

What You'll Learn