Generator Calculations for Event-Driven Models

From Event Rates to Changes in a Function

In an event-driven model, the state changes when one of several possible events occurs. Each event has a rate that may depend on the current state, and its effect on a function of the state is the difference between the function after the event and before it. Organizing calculations around those differences gives a direct way to obtain generator actions without first solving for the transition probabilities at positive times.

Let \((X_t)_{t\ge0}\) be a continuous-time Markov chain on a finite or countable state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Use the natural filtration \(\mathcal{F}_t=\sigma(X_s:0\le s\le t)\). Its generator is \(Q=(q_{ij})\), with \(q_{ij}\ge0\) for \(i\ne j\) and rows summing to zero. For a function \(f:S\to\mathbb{R}\), the generator action is

The second expression follows from \(q_{ii}=-\sum_{j\ne i}q_{ij}\). It emphasizes that a transition matters through its change in \(f\), not just the destination value. On a finite state space the sum is finite. On a countable state space we use the expression when the sum of absolute values of the terms is finite. For unbounded \(f\), a pointwise rate calculation alone does not assert that \(\mathbb{E}_i[f(X_t)]\) is differentiable or that its derivative equals \(\mathbb{E}_i[Qf(X_t)]\); those conclusions require additional conditions.

The following result justifies this event-by-event calculation. If several event types lead to the same new state, their rates must be added to obtain the corresponding off-diagonal entry of the generator.

Proof. Fix \(i\). For each destination \(j\ne i\), collect all event types \(r\) for which \(T_r(i)=j\), and define \(q_{ij}\) as the sum of their rates. Event types with \(T_r(i)=i\) have zero change in \(f\), so they add nothing. Grouping the remaining terms by destination gives \(\sum_r a_r(i)(f(T_r(i))-f(i))=\sum_{j\ne i}q_{ij}(f(j)-f(i))\). Set \(q_{ii}=-\sum_{j\ne i}q_{ij}\). The row then sums to zero, and \(\sum_{j\in S}q_{ij}f(j)=\sum_{j\ne i}q_{ij}(f(j)-f(i))\). This is precisely the generator action. \(\square\)

A Finite-State Event Model

Consider a system with capacity two. Its state \(i\in\{0,1,2\}\) is the number of items present. Arrivals occur at rate \(2\) when the system is not full, and services occur at rate \(1\) when it is not empty. An arrival increases the state by one; a service decreases it by one. The rates are event rates, not probabilities: for example, the arrival rate \(2\) at state \(0\) does not mean that an arrival has probability \(2\).

Generator actions also apply to indicator functions. For \(f(i)=\mathbf{1}_{\{2\}}(i)\), the value is \(1\) at state \(2\) and \(0\) elsewhere. The action measures the rate of change in this indicator, separating transitions into the target state from transitions out of it.

A Countable-State Population Model

Now let \(S=\{0,1,2,\ldots\}\), where \(X_t\) is a population size. Immigrants arrive at rate \(\lambda>0\), increasing the population by one, and each individual dies independently at rate \(\delta>0\), decreasing the population by one. Thus, at population \(n\), the total immigration rate is \(\lambda\) and the total death rate is \(\delta n\). At \(n=0\) the death rate is zero. Every state has finite total exit rate. The model is nonexplosive: over a finite time interval there are finitely many immigration events almost surely, and, starting with finitely many individuals, each individual can die at most once.

The rates are \(q_{n,n+1}=\lambda\), \(q_{n,n-1}=\delta n\) for \(n\ge1\), and \(q_{n,n}=-(\lambda+\delta n)\), with \(q_{0,0}=-\lambda\). The row at \(0\) sums to \(-\lambda+\lambda=0\). For \(n\ge1\), the row sums to \(\lambda+\delta n-(\lambda+\delta n)=0\). For a function with finite differences at each state, the event formula is

and \(Qf(0)=\lambda(f(1)-f(0))\). The death term at zero is omitted, rather than evaluated at a nonexistent state \(-1\).

Proof. For \(n\ge1\), the change in \(f(n)=n\) under immigration is \((n+1)-n=1\), and under death it is \((n-1)-n=-1\). Therefore \(Qf(n)=\lambda(1)+\delta n(-1)=\lambda-\delta n\). At \(n=0\), \(Qf(0)=\lambda(1-0)=\lambda\), also given by the formula. For \(g(n)=n^2\), immigration changes the function by \((n+1)^2-n^2=2n+1\), while death changes it by \((n-1)^2-n^2=-2n+1\). Hence, for \(n\ge1\), \(Qg(n)=\lambda(2n+1)+\delta n(-2n+1)=-2\delta n^2+(2\lambda+\delta)n+\lambda\). At \(n=0\), \(Qg(0)=\lambda(1^2-0^2)=\lambda\), matching the same expression at zero. This proves both formulas on the entire state space. \(\square\)

Interpretation and a Common Mistake

For a function \(f\), \(Qf(i)\) is the rate-weighted sum of changes in \(f\) from state \(i\). In the finite occupancy example, \(Qf(i)\) for \(f(i)=i\) is a local drift calculation. In the population model, \(Qf(n)=\lambda-\delta n\) compares the immigration and death contributions. These are generator actions; interpreting them as derivatives of expected values requires suitable conditions on the process and function. In particular, for an unbounded function such as \(n^2\), finiteness of the pointwise rate sum does not by itself justify interchanging a derivative and expectation.

Common mistake: dropping the diagonal term but also forgetting to subtract the current function value. In the finite model, at state \(1\) with \(f(i)=i\), the incorrect calculation using only off-diagonal entries as \(q_{10}f(0)+q_{12}f(2)\) gives \(1(0)+2(2)=4\). This omits the holding-rate contribution \(q_{11}f(1)=-3\). The correct matrix calculation is \(1(0)-3(1)+2(2)=1\), or, equivalently, the event-difference calculation \(1(0-1)+2(2-1)=-1+2=1\). Use either complete form; do not mix the off-diagonal rates with the matrix formula while omitting its diagonal term.

Event-based calculations are useful whenever a model is specified by mechanisms rather than a complete matrix. For each state, identify every event, its rate, and its effect on the chosen function. If different events lead to the same destination, their rates add in the generator matrix, but their separate contributions to \(Qf\) still sum to the same total. This makes the calculation both systematic and easy to check.