Rate Diagrams And Generator Entries

From a Rate Diagram to a Generator

A birth-death process moves among ordered states by single-step increases and decreases. A rate diagram records the rates attached to those possible moves; the generator collects the same information in a matrix, with one additional entry in each row that records the total rate of leaving that state. Constructing that diagonal entry correctly is essential: a generator is not a transition-probability matrix, and its rows sum to zero rather than one.

Let \((X_t)_{t\ge0}\) be a continuous-time Markov chain on a finite state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Write \(P(t)=(p_{ij}(t))\), where \(p_{ij}(t)=\mathbb{P}(X_t=j\mid X_0=i)\). Its generator \(Q=(q_{ij})\) is given by the infinitesimal rates, when the limits exist:

Thus an off-diagonal entry is the instantaneous rate of transition from \(i\) to \(j\). The diagonal entry is not a transition rate to a different state. For a finite-state chain, the rows of \(P(t)\) sum to one, and consequently each row of \(Q\) sums to zero. In particular, \(q_{ii}=-\sum_{j\ne i}q_{ij}\). The total exit rate from \(i\) is the nonnegative number \(-q_{ii}\).

The phrase “wherever allowed” matters. On \(S=\{0,1,\ldots,N\}\), there is no state \(-1\) or \(N+1\), so the rates for arrows out of the state space are set to zero: \(\mu_0=0\) and \(\lambda_N=0\). A boundary may also be absorbing or have some other allowed behavior, depending on its rates. The diagram specifies which transitions are possible; the generator records their rates and the negative sum of the rates out of each state.

Theorem (Rate-diagram construction). Suppose a finite-state continuous-time Markov chain has only the birth and death transitions shown in a rate diagram. At each state \(i\), let the rates of the allowed arrows be \(\lambda_i\) and \(\mu_i\), setting a rate to zero when its corresponding arrow is absent. Then its generator entries are \(q_{i,i+1}=\lambda_i\), \(q_{i,i-1}=\mu_i\), \(q_{ii}=-(\lambda_i+\mu_i)\), and \(q_{ij}=0\) for all other \(j\). Conversely, these entries give exactly the specified rates and no other transitions, and every row sums to zero.

Proof. Fix a state \(i\). By the definition of an off-diagonal generator entry, the rate on the arrow \(i\to i+1\) is \(q_{i,i+1}=\lambda_i\), and the rate on \(i\to i-1\) is \(q_{i,i-1}=\mu_i\). Every other move to a distinct state is excluded by the rate diagram, so its rate, and therefore its off-diagonal generator entry, is zero. The row-sum condition for a finite-state generator gives \(q_{ii}=-\sum_{j\ne i}q_{ij}=-(\lambda_i+\mu_i)\), where absent arrows have rate zero. This proves the stated form. Conversely, in the displayed form the two possible off-diagonal entries have precisely the specified rates, all other off-diagonal entries vanish, and the row sum is \(\lambda_i+\mu_i-(\lambda_i+\mu_i)=0\). Since \(i\) was arbitrary, the claims hold in every row. \(\square\)

Reading the Entries and Checking the Matrix

There are two routine checks when converting a diagram. First, each off-diagonal entry must correspond to an arrow with the same rate; do not place a rate on the diagonal. Second, the diagonal entry must be the negative sum of all off-diagonal entries in that row. It is therefore zero at an absorbing state, negative at any state with a positive exit rate, and never positive.

For a countably infinite state space, the same entry rule defines a conservative \(Q\)-matrix when the off-diagonal rates in each row have finite sum: the diagonal is the negative of that sum, so each row sums to zero. This algebraic row-sum property alone does not guarantee that the matrix generates a conservative continuous-time Markov chain. Nonexplosion is also required. The examples here are finite-state, where this countable-state qualification does not arise.

What the Generator Does to a Function

Besides displaying individual rates, \(Q\) describes the instantaneous change in the expected value of a function of the state. For a real-valued function \(f\) on a finite state space, the generator acts by matrix multiplication:

In this expression, a term for an absent arrow is zero. The differences show why the diagonal is negative: it subtracts the current value \(f(i)\) once for each possible outgoing rate. Equivalently, the finite-state generator definition gives \(\lim_{h\downarrow0}(\mathbb{E}_i[f(X_h)]-f(i))/h=(Qf)(i)\).

Proof. Since \(S\) is finite, write \(\mathbb{E}_i[f(X_h)]=\sum_{j\in S}p_{ij}(h)f(j)\). Subtract \(f(i)\), and use \(\sum_jp_{ij}(h)=1\), to obtain \[ \mathbb{E}_i[f(X_h)]-f(i) =\sum_{j\in S}\bigl(p_{ij}(h)-\mathbf{1}_{\{i=j\}}\bigr)f(j). \] Divide by \(h>0\) and take \(h\downarrow0\). The sum has finitely many terms, so the limit can be taken term by term; each coefficient converges to \(q_{ij}\) by the definition of the generator. The limit is \(\sum_jq_{ij}f(j)=(Qf)(i)\). For a birth-death chain, substitute \(q_{i,i+1}=\lambda_i\), \(q_{i,i-1}=\mu_i\), and \(q_{ii}=-(\lambda_i+\mu_i)\). The result is \(\lambda_i(f(i+1)-f(i))+\mu_i(f(i-1)-f(i))\), with absent-arrow terms zero. \(\square\)

A Common Error: Putting the Exit Rate in the Wrong Place

A frequent mistake is to put a positive total exit rate on the diagonal, or to use a zero diagonal because there is no arrow from a state to itself. Neither convention gives the generator. For example, in the first example, state \(1\) has a death rate \(1\) and a birth rate \(3\). A wrong row would be \((1,4,3,0)\), using a positive diagonal entry \(4\). Its row sum is \(1+4+3=8\), not zero, and its diagonal cannot represent a transition rate to a distinct state. The correct row is \((1,-4,3,0)\), whose sum is \(1-4+3=0\), and whose diagonal is the negative total exit rate.

Another incorrect calculation would set \(q_{11}=-\lambda_1=-3\), forgetting the death arrow. The resulting row \((1,-3,3,0)\) sums to \(1\), so it fails the generator check. The correction is to include every outgoing arrow before negating: \(-(\mu_1+\lambda_1)=-(1+3)=-4\). In general, read the off-diagonal entries first, sum them, and only then write the diagonal as the negative of that sum.

Once a generator has been built, its entries can be checked in both directions: each allowed arrow appears exactly once as an off-diagonal rate, and the diagonal balances all outgoing rates. The total exit rate is \(-q_{ii}\), not \(q_{ii}\); it is a nonnegative rate, whereas the diagonal generator entry is nonpositive.