Events Are Not Always Jumps
A mechanism can register an event without changing the state. A clock may ring and instruct a system to remain where it is; an attempted move may be blocked; or several event types may produce the same state. Such events can matter to a model of the mechanism, but they are not jumps of the state-valued process. Counting them as jumps gives the wrong holding rate and the wrong jump probabilities.
Let \((X_t)_{t\ge0}\) be a continuous-time Markov chain on a finite or countable state space \(S\), defined on a probability space \((\Omega,\mathcal F,\mathbb P)\), with right-continuous paths. Its generator is \(Q=(q_{ij})\). For \(j\ne i\), \(q_{ij}\ge0\) is the rate of a transition from \(i\) to \(j\), and the exit rate is \(q_i=\sum_{j\ne i}q_{ij}\). When \(q_i<\infty\), the diagonal entry is \(q_{ii}=-q_i\), so each generator row sums to zero. If \(q_i=0\), state \(i\) is absorbing: the process has no departure from it.
The diagonal value \(r_{ii}=0\) describes the jump chain, not the probability that \(X_t=i\) at a later fixed time. A process can leave \(i\) and return before time \(t\), or simply not yet have left. Thus \(P_{ii}(t)=\mathbb P_i(X_t=i)\) can be positive even though an actual jump never takes the process from \(i\) to \(i\).
Why the Jump-Chain Diagonal Is Zero
For a state \(i\) with \(q_i>0\), the standard holding-time and jump-probability description of a CTMC says that the holding time in \(i\) is exponential with rate \(q_i\), and, at departure, the destination \(j\ne i\) has probability \(q_{ij}/q_i\). The destination is a different state by the meaning of departure. The following argument derives these probabilities directly when the rates are produced by competing event clocks.
Proof. Let \(A\) be the time of the first event of any type that changes the state. For each event type with \(\phi_\ell(i)\ne i\), its clock has probability \(e^{-a_\ell t}\) of not ringing by time \(t\). Independence therefore gives \(\mathbb P_i(A>t)=\prod_{\ell:\phi_\ell(i)\ne i}e^{-a_\ell t}=e^{-q_i t}\). Hence \(A\) is exponential with rate \(q_i\) when \(q_i>0\). For a particular type \(\ell\) that changes the state to \(j\), the probability that it is the first state-changing event and occurs in the interval \([t,t+dt)\) has density \(a_\ell e^{-q_i t}\,dt\). Summing this density over the types that send \(i\) to \(j\) gives \(q_{ij}e^{-q_i t}\,dt\). Integrating over \(t\ge0\) gives \(q_{ij}/q_i\), since \(\int_0^\infty e^{-q_i t}\,dt=1/q_i\). Every event counted in this destination calculation changes the state, so \(j\ne i\). If \(q_i=0\), there are no state-changing clocks and no next jump. \(\square\)
In this construction, clocks whose events leave the state unchanged may ring before the first actual jump. They do not alter the time of that jump: the event clocks that can change the state still have their original rates. Equivalently, the process may be constructed by ignoring no-change events and using only the state-changing rates. This conclusion concerns state-dependent exponential event clocks; an arbitrary event mechanism with history-dependent rates need not have this simple description.
Worked Rate-Table Corrections
Worked Example: A Two-State Chain with No-Change Events
The state space is \(S=\{0,1\}\). In either state, a flip event occurs at rate \(2\), and a separate event that leaves the current state unchanged occurs at rate \(3\). Find the generator and the jump-chain probabilities.
At state \(0\), the flip sends the process to \(1\), so \(q_{01}=2\); the rate-\(3\) event does not change the state and contributes nothing to \(q_0\). Thus \(q_0=2\) and \(q_{00}=-2\). At state \(1\), the flip sends the process to \(0\), so \(q_{10}=2\), \(q_1=2\), and \(q_{11}=-2\). Therefore
Check: The first row sums to \(-2+2=0\), and the second row sums to \(2-2=0\). The jump-chain probabilities are \(r_{01}=2/2=1\) and \(r_{10}=2/2=1\), with \(r_{00}=r_{11}=0\); each jump-chain row sums to \(1\).
The time until the next actual change is exponential with rate \(2\), not \(5\). The rate-\(5\) clock would describe the time until either type of event rings, including the events that do not change the state. Treating that as the CTMC holding rate and also retaining only the flip as an off-diagonal rate would give a row sum of \(-5+2=-3\), not zero. The generator correction is to exclude the no-change events from the exit rate.
Worked Example: Several Event Types with the Same Destination
On \(S=\{0,1,2\}\), state \(0\) has three event types: rate \(1\) leaves it at \(0\), rate \(2\) sends it to \(1\), and rate \(1\) sends it to \(2\). From state \(1\), a rate-\(1\) event sends it to \(0\), while a rate-\(4\) event leaves it at \(1\). From state \(2\), a rate-\(3\) event sends it to \(1\), while a rate-\(2\) event leaves it at \(2\). Construct the generator and jump-chain rows.
At state \(0\), the state-changing rate is \(2+1=3\), giving \(q_{01}=2\), \(q_{02}=1\), and \(q_{00}=-3\). At state \(1\), only the rate-\(1\) event changes state, so \(q_{10}=1\), \(q_{11}=-1\), and \(q_{12}=0\). At state \(2\), only the rate-\(3\) event changes state, so \(q_{21}=3\), \(q_{22}=-3\), and \(q_{20}=0\). Thus
Check: The generator row sums are \(-3+2+1=0\), \(1-1+0=0\), and \(0+3-3=0\). The exit rates are \(q_0=3\), \(q_1=1\), and \(q_2=3\). The jump-chain rows are \((0,\tfrac23,\tfrac13)\), \((1,0,0)\), and \((0,1,0)\); their sums are \(0+\tfrac23+\tfrac13=1\), \(1+0+0=1\), and \(0+1+0=1\).
At state \(0\), the total rate of all event types is \(1+2+1=4\), but the probability of the next actual jump being to \(1\) is \(2/3\), not \(2/4\): the rate-\(1\) no-change event is not a possible destination of the jump chain. The holding rate is \(3\), and the two genuine destinations have probabilities proportional to their rates, \(2\) and \(1\).
Worked Example: Boundary Events in a Capped Process
Consider \(S=\{0,1,2\}\). A birth attempt occurs at rate \(2\) and sends \(n\) to \(\min(n+1,2)\). A death attempt occurs at rate \(1\) and sends \(n\) to \(\max(n-1,0)\). At a boundary, one of these attempts leaves the state unchanged. Find the corrected generator.
At state \(0\), the birth attempt sends \(0\) to \(1\) at rate \(2\), while the death attempt leaves the process at \(0\). Hence \(q_{01}=2\), \(q_{00}=-2\), and \(q_{02}=0\). At state \(1\), the birth and death attempts send the process to \(2\) and \(0\) at rates \(2\) and \(1\). Hence \(q_{10}=1\), \(q_{12}=2\), and \(q_{11}=-3\). At state \(2\), the birth attempt leaves the process at \(2\), while the death attempt sends it to \(1\) at rate \(1\). Hence \(q_{21}=1\), \(q_{22}=-1\), and \(q_{20}=0\). The generator is
Check: The row sums are \(-2+2+0=0\), \(1-3+2=0\), and \(0+1-1=0\). The exit rates are \(2,3,1\), respectively. The jump-chain rows are \((0,1,0)\), \((\tfrac13,0,\tfrac23)\), and \((0,1,0)\); their sums are \(1\), \(\tfrac13+0+\tfrac23=1\), and \(1\). In particular, at state \(2\) the two attempted-event rates sum to \(3\), but the actual exit rate is only \(1\).
Why the Distinction Matters
The state process records states over time, not every internal event of a proposed mechanism. If the goal is to model the internal event record as well, that record can be tracked separately. But when constructing the CTMC generator for \(X_t\), a no-change event has no off-diagonal destination and contributes no exit rate. Including it can make the generator row fail to sum to zero, or can produce a formally valid but incorrect table if an erroneous diagonal is adjusted to match.
Common mistake. Suppose at state \(0\) a rate-\(1\) event leaves the state unchanged and a rate-\(2\) event sends it to \(1\). A common incorrect calculation is to set \(q_0=1+2=3\), \(q_{01}=2\), and \(q_{00}=-3\), then report a jump probability \(r_{01}=2/3\). This incorrectly treats the no-change event as a departure. The corrected calculation is \(q_{01}=2\), \(q_0=2\), \(q_{00}=-2\), so \(r_{01}=2/2=1\), and \(r_{00}=0\). The generator row now sums to \(-2+2=0\), and the jump-chain row sums to \(0+1=1\).
There is no contradiction between a zero jump-chain diagonal and a positive probability of being in the same state at a later time. For instance, in the two-state example, the chain may make two jumps and return to its starting state. Such a return is a sequence of jumps through a different state; it is not a self-transition at a jump time. Keeping this distinction straight is essential whenever rates are built from event descriptions.
Check Your Understanding
For each question, distinguish the rate of an attempted event from the rate of an actual change of state.
- In state \(i\), two event types of rates \(3\) and \(2\) send the process to \(j\ne i\), and a rate-\(4\) event leaves it at \(i\). What are \(q_i\), \(q_{ij}\), and the jump probability to \(j\)?
- Why must the diagonal entry of a generator be the negative sum of its off-diagonal entries, rather than a positive self-transition rate?
- A state has two possible actual destinations, with rates \(1\) and \(3\). Give its jump-chain probabilities and verify they sum to \(1\).
- Can \(P_{ii}(t)\) be positive even when \(r_{ii}=0\)? Explain how the process can be back in state \(i\) at time \(t\).
- At a boundary, an attempted move is blocked and leaves the state unchanged. Which entries in the generator and jump-chain row does that attempt affect?