State-Dependent Event Rates in Countable Models

Rates Determine Both the Waiting Time and the Next State

In a countable-state continuous-time Markov chain, the event rates can depend on the current state. A queue with many customers may have a different total event rate from an empty queue; a population may produce births and deaths at rates that change with its size. To describe the dynamics at a state \(i\), we need two pieces of information: how long the chain waits before leaving \(i\), and where it goes when it leaves. The off-diagonal entries of the generator encode both.

Let \((X_t)_{t\ge0}\) be a time-homogeneous continuous-time Markov chain on a countable state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with natural filtration \(\mathcal{F}_t=\sigma(X_s:0\le s\le t)\). Its generator is \(Q=(q_{ij})\). We use the convention that \(q_{ij}\ge0\) for \(j\ne i\), each exit rate \(q_i=\sum_{j\ne i}q_{ij}\) is finite, and \(q_{ii}=-q_i\), so each generator row sums to zero. The rates are time-homogeneous, but they may vary with \(i\).

The first-exit destination is not the same as the state observed at a fixed later time: after leaving \(i\), the chain might make further jumps before that observation. The law below concerns the first exit only. Its proof also handles the main technical issue for countably many possible destinations: individual small-time expansions cannot simply be summed.

Proof. Fix \(i\), and write \(S_i(t)=\mathbb{P}_i(T_i>t)\) for the probability of still being in \(i\) without a first exit by time \(t\). For \(j\ne i\), define \(F_j(t)=\mathbb{P}_i(T_i\le t,\ J_i=j)\). While the chain has not exited, it is at \(i\). By the Markov property, conditional on survival up to time \(s\), its instantaneous first-exit rate to \(j\) is \(q_{ij}\), just as when it starts at \(i\). Thus the first-exit probability into \(j\) has rate \(q_{ij}S_i(s)\) at time \(s\), giving \(F_j(t)=q_{ij}\int_0^t S_i(s)\,ds\). This relation concerns first exits, not all transitions observed at time \(t\).

For a finite \(t\), the disjoint events \(\{T_i\le t,J_i=j\}\), over the countable set \(j\ne i\), comprise the event \(\{T_i\le t\}\). Nonnegativity allows us to sum their probabilities and interchange the sum and integral by Tonelli's theorem:

Since \(S_i(0)=1\), this integral equation makes \(S_i\) absolutely continuous and gives \(S_i'(t)=-q_iS_i(t)\) for almost every \(t\). The product \(e^{q_it}S_i(t)\) is absolutely continuous and has derivative zero almost everywhere, so it is constant; its value at zero is \(1\). Hence \(S_i(t)=e^{-q_it}\). In particular, if \(q_i=0\), then \(S_i(t)=1\) for every \(t\), so no exit occurs. If \(q_i>0\), then \(T_i<\infty\) almost surely, and \(F_j(t)=\frac{q_{ij}}{q_i}(1-e^{-q_it})\). This factors as \(\mathbb{P}_i(J_i=j)\mathbb{P}_i(T_i\le t)\), proving the stated destination probabilities and independence. \(\square\)

Proof. For \(q_i>0\), nonnegativity and the definition of the exit rate give \(\sum_{j\ne i}r_{ij}=\sum_{j\ne i}q_{ij}/q_i=q_i/q_i=1\). The first-exit law identifies each \(r_{ij}\) as the probability of destination \(j\). If \(q_i=0\), every nonnegative \(q_{ij}\) must be zero, and the theorem says the chain does not exit. \(\square\)

Thus the total rate controls the time scale, while normalizing the off-diagonal rates determines the destination. A row with the same destination proportions but a different total rate gives the same jump choices and different waiting times.

Why Countable State Spaces Require Care

When there are countably many possible destinations, a tempting argument is to write a small-time expansion for every destination and add the remainders. That step is not generally valid. For example, for \(0<h<1\), define \(a_j(h)=h^2\) for \(1\le j\le\lfloor h^{-2}\rfloor\), and \(a_j(h)=0\) otherwise. For every fixed \(j\), \(a_j(h)/h=h\) for all sufficiently small \(h\), so \(a_j(h)=o(h)\). But \(\sum_{j\ge1}a_j(h)=h^2\lfloor h^{-2}\rfloor\), which approaches \(1\), not \(0\). This demonstrates why destination-by-destination \(o(h)\) terms cannot be summed without a separate bound.

The first-exit proof avoids that problem. It first obtains the exact integral expression for each destination, then sums nonnegative integrands. The finite total-rate assumption \(q_i<\infty\) makes the sum equal to \(q_i\int_0^tS_i(s)\,ds\). The holding-time conclusion is therefore justified without assuming an aggregate small-time expansion.

Common mistake. At state \(n=3\) in the state-dependent queue example, one might report the next-state probabilities as \(1\) for an arrival and \(6\) for a departure, treating rates themselves as probabilities. Those numbers sum to \(7\), not \(1\), so they cannot be probabilities. The correction is to divide each rate by the total rate \(7\): the probabilities are \(1/7\) and \(6/7\), which sum to \(1\). The total rate \(7\) remains the exponential holding-time parameter; it is not itself a destination probability.

These calculations matter whenever event rates change with system size. A rate table supports two distinct questions: how quickly the next event occurs, and which event occurs next. Keeping those roles separate helps prevent errors in queueing, population, and other countable-state models. It also makes explicit the zero-rate case, where the chain remains at its state and no next destination exists.