Kolmogorov Equations for Birth-Death Models

Introduction

A birth-death process is a continuous-time model in which the state records a count, and each transition changes that count by exactly one. It can represent, for example, the number of active claims in a hypothetical portfolio, the number of people receiving a service, or the number of units in a system. The model is simple to state, but its forward equations require careful attention to which transitions enter a state and which leave it.

The key technique is to write the derivative of each state probability as incoming probability flow minus outgoing probability flow. This tutorial develops that rule for a finite state space, explains the boundary equations, and solves systems where the probabilities can be found explicitly.

Definition: A finite birth-death process \(X(t)\) has state space \(\{0,1,\ldots,N\}\). From state \(j\), it can jump only to \(j+1\) at intensity \(\lambda_j\) or to \(j-1\) at intensity \(\mu_j\). The rates are constant in time and nonnegative. At the boundaries, set \(\mu_0=0\) and \(\lambda_N=0\), so the process cannot leave the state space. All other transitions have intensity zero.

The process is assumed to be a time-homogeneous continuous-time Markov chain: conditional on its current state, its future transition behavior does not depend on its earlier path. Since there are finitely many states and all rates are finite, the process cannot make infinitely many jumps in a finite time interval. Write

p_j(t)=P(X(t)=j\mid X(0)=i),

when the initial state \(i\) is fixed. The initial conditions are \(p_i(0)=1\) and \(p_j(0)=0\) for \(j\ne i\). The starting state affects the initial conditions, not the form of the forward equations.

Forward Equations by State

For an interior state \(j\), where \(1\le j\le N-1\), probability can enter from state \(j-1\) by a birth or from state \(j+1\) by a death. It can leave state \(j\) by a birth or a death. Thus its forward equation is

p_j'(t)=\lambda_{j-1}p_{j-1}(t)+\mu_{j+1}p_{j+1}(t) -(\lambda_j+\mu_j)p_j(t).

The first two terms are incoming flow. The final term is outgoing flow: while the process is in state \(j\), its total rate of leaving is \(\lambda_j+\mu_j\). At the boundaries, omit the nonexistent incoming or outgoing transitions. Equivalently, the boundary equations are

p_0'(t)=\mu_1p_1(t)-\lambda_0p_0(t), \qquad p_N'(t)=\lambda_{N-1}p_{N-1}(t)-\mu_Np_N(t).

These equations use the row-vector convention for the transition matrix: if \(Q\) is the generator, its entry \(q_{jk}\) is the transition intensity from state \(j\) to state \(k\), and the forward equation is \(p'(t)=p(t)Q\). In particular, the derivative at destination state \(j\) gathers flows from states that can enter \(j\). This is the forward equation, not the backward equation, in which the derivative is organized by the starting state.

Why the Flow Equation Holds

The short-time transition probabilities explain both the incoming terms and the negative sign on the outgoing term. Over a small interval of length \(h\), a process in state \(j\) stays there with probability \(1-(\lambda_j+\mu_j)h+o(h)\). A process in state \(j-1\) moves to \(j\) with probability \(\lambda_{j-1}h+o(h)\), and one in state \(j+1\) moves to \(j\) with probability \(\mu_{j+1}h+o(h)\). Here \(o(h)\) denotes terms whose ratio to \(h\) tends to zero as \(h\) tends to zero. The probability of two or more jumps in this short interval is also \(o(h)\).

p_j(t+h)=p_j(t)\{1-(\lambda_j+\mu_j)h\} +\lambda_{j-1}p_{j-1}(t)h +\mu_{j+1}p_{j+1}(t)h+o(h).

Subtract \(p_j(t)\), divide by \(h\), and let \(h\) tend to zero. The result is the stated differential equation. At a boundary, the same argument applies after removing transitions that cannot occur. This justification depends on the model allowing only nearest-neighbor jumps; with additional transitions, the equation must include their incoming and outgoing rates too.

Solving a Two-State Birth-Death Model

Worked Example: A Two-State Process with Positive Rates

A hypothetical system has states 0 and 1. It moves from 0 to 1 at constant rate \(\lambda_0=2\) per year and from 1 to 0 at constant rate \(\mu_1=1\) per year. It starts in state 1. Find both state probabilities at \(t=\ln 2/3\) years.

Method. Write the boundary forward equation and use \(p_0(t)+p_1(t)=1\). This avoids solving two dependent equations separately. In a two-state model the equation is \(p_0'=\mu_1p_1-\lambda_0p_0\), so

p_0'=1-3p_0,\qquad p_0(0)=0.

Multiplying by the integrating factor \(e^{3t}\) gives \((e^{3t}p_0)'=e^{3t}\). Integrating from 0 to \(t\),

e^{3t}p_0(t)-p_0(0)=\frac{e^{3t}-1}{3}, \qquad p_0(t)=\frac{1-e^{-3t}}{3}.

At \(t=\ln 2/3\), \(e^{-3t}=e^{-\ln 2}=1/2\). Therefore \(p_0(t)=(1-1/2)/3=1/6\), and \(p_1(t)=1-p_0(t)=5/6\). The values are nonnegative and sum to 1, as required.

Check. Differentiating the solution gives \(p_0'=e^{-3t}\). The right side of the forward equation is \(p_1-2p_0=1-3p_0=e^{-3t}\), so the equation and initial condition are both satisfied. A common mistake is to assign the rate \(\lambda_0\) to flow into state 0; in fact, the flow into 0 comes from state 1 and has rate \(\mu_1\).

The same two-state equation also illustrates why conditions on the rates matter. With initial state 1, let \(a=\lambda_0+\mu_1\). If \(a>0\), then

p_0(t)=\frac{\mu_1}{a}(1-e^{-at}), \qquad p_1(t)=1-p_0(t).

The limiting probability of state 0 is \(\mu_1/a\). If both rates are zero, then \(a=0\), the process stays in its initial state, and \(p_0(t)=0\) for all \(t\). The limiting expression \(\mu_1/a\) must not be used in that case. This zero-rate case is a useful reminder to check denominators and assumptions before quoting a limiting value.

A Three-State System and Its Solution

Worked Example: Solving a Three-State Birth-Death Model

A hypothetical count process has states 0, 1, and 2. At either state 0 or state 1, the birth rate is 1 per year; at either state 1 or state 2, the death rate is 1 per year. The boundary rates are \(\mu_0=0\) and \(\lambda_2=0\). The process starts in state 0. Find the probabilities at \(t=\ln 2\) years.

Method. Write the three forward equations, then solve the constant-coefficient system. The generator, with rows as starting states and columns as destination states, is

Q= \begin{pmatrix} -1&1&0\\ 1&-2&1\\ 0&1&-1 \end{pmatrix}.

The equations and initial conditions are

p_0'=-p_0+p_1,\qquad p_1'=p_0-2p_1+p_2,\qquad p_2'=p_1-p_2, \qquad (p_0(0),p_1(0),p_2(0))=(1,0,0).

Solving the system (for example, by decomposing the generator into its constant mode and its modes with rates 1 and 3) gives

p_0(t)=\frac13+\frac12e^{-t}+\frac16e^{-3t},\quad p_1(t)=\frac13-\frac13e^{-3t},\quad p_2(t)=\frac13-\frac12e^{-t}+\frac16e^{-3t}.

At \(t=\ln 2\), \(e^{-t}=1/2\) and \(e^{-3t}=1/8\). Hence

p_0=\frac13+\frac14+\frac1{48}=\frac{29}{48},\quad p_1=\frac13-\frac1{24}=\frac7{24}=\frac{14}{48},\quad p_2=\frac13-\frac14+\frac1{48}=\frac5{48}.

The probabilities sum to \((29+14+5)/48=1\), and each is nonnegative. As an additional check, at \(t=0\) the formulas give \((1,0,0)\). Substitution into the three differential equations also verifies the solution: for example, \(p_0'=-e^{-t}/2-e^{-3t}/2=-p_0+p_1\). The other two equations follow by differentiating their displayed expressions. A common mistake is to use \(-p_1\), rather than \(-2p_1\), as the outgoing term in the middle-state equation; both possible jumps out of state 1 must be counted.

Boundary Cases: A Pure-Birth Example

Worked Example: An Upper Boundary with No Births

A hypothetical process has states 0, 1, and 2, starts at 0, and has birth rates \(\lambda_0=2\) and \(\lambda_1=1\) per year. There are no deaths, and state 2 is the upper boundary, so \(\lambda_2=0\). Find the state probabilities at \(t=\ln 2\) years.

Method. Solve successively from the lowest state. The forward equations are \(p_0'=-2p_0\), \(p_1'=2p_0-p_1\), and \(p_2'=p_1\), with initial values \((1,0,0)\). The first equation gives \(p_0=e^{-2t}\). For the second, the integrating factor is \(e^t\):

(e^tp_1)'=2e^{-t}, \qquad p_1(t)=2(e^{-t}-e^{-2t}).

The last equation can be integrated using \(p_2(0)=0\):

p_2(t)=\int_0^t2(e^{-s}-e^{-2s})\,ds =1-2e^{-t}+e^{-2t}.

At \(t=\ln 2\), \(e^{-t}=1/2\) and \(e^{-2t}=1/4\). Thus \(p_0=1/4\), \(p_1=2(1/2-1/4)=1/2\), and \(p_2=1-1+1/4=1/4\). They sum to 1. Also, \(p_2'=2e^{-t}-2e^{-2t}=p_1\), as required. State 2 is not left by a birth: setting \(\lambda_2=0\) is essential to the boundary equation.

Common Exam Traps and Practical Checks

A forward equation is easiest to audit by listing, for each state, its possible incoming and outgoing transitions before writing any algebra. In particular, distinguish the rate attached to a transition from the probability of occupying its starting state: the probability flow from \(j\) to \(j+1\) is \(\lambda_jp_j(t)\), not simply \(\lambda_j\).

A deliberately incorrect equation for state 0 in a two-state model is \(p_0'=\lambda_0p_0-\mu_1p_1\). It reverses the signs and describes outflow as inflow. The correction is \(p_0'=\mu_1p_1-\lambda_0p_0\): state 1 supplies incoming flow to 0, while state 0 supplies outgoing flow. For any proposed solution, check the initial conditions, verify its derivative against the forward equation, and confirm that its probabilities remain nonnegative and sum to 1.

Key takeaway: In a finite birth-death model, the derivative of each state probability is the sum of the adjacent-state flows into it minus its own total outgoing flow. At states 0 and \(N\), omit transitions that cannot occur. Always check rate conditions before using a closed-form limit.

Check Your Understanding

For each problem, write the relevant forward equation before calculating.

A two-state process starts in state 1, with \(\lambda_0=0.8\) and \(\mu_1=0.2\) per year. Find \(p_0\) and \(p_1\) at \(t=\ln 2\) years. (Answer: \(p_0=0.1000,\ p_1=0.9000\).)
A process has states 0, 1, and 2, with \(\lambda_0=0.4,\lambda_1=1.5,\mu_1=0.5,\mu_2=2\). At an instant its state probabilities are \((0.2,0.5,0.3)\). Find the instantaneous derivative of the state-1 probability. (Answer: \(p_1'=-0.32\) per year.)
A pure-death process has states 0, 1, and 2, with \(\mu_1=1,\mu_2=2\) per year, and starts in state 2. Find the probabilities at \(t=\ln 2\) years. (Answer: \((p_0,p_1,p_2)=(0.25,0.50,0.25)\).)
For a process with states 0 through 3, write the boundary forward equation for state 3 when \(\lambda_3=0\). State the source of each term. (Answer: \(p_3'=\lambda_2p_2-\mu_3p_3\); births from state 2 enter, and deaths from state 3 leave.)
A two-state process starts in state 1 and has \(\lambda_0=\mu_1=0\). State the probability of state 0 at every time, and explain why the positive-rate limiting formula cannot be used. (Answer: \(p_0(t)=0\) for all \(t\); the denominator \(\lambda_0+\mu_1\) is zero, and the process remains in state 1.)

What You'll Learn