What a Short Interval Reveals
For a continuous-time Markov chain, transition probabilities over a short interval are close to those over no time at all. The first-order change records how likely the chain is to move from its current state to a different one. This is the basis for interpreting the off-diagonal entries of a generator as jump rates. The probability of staying in the current state also has a first-order expansion, but obtaining it requires accounting for all the ways to leave.
Let \((X_t)_{t\ge0}\) be a time-homogeneous continuous-time Markov chain on a finite or countable state space \(S\), defined on a probability space \((\Omega,\mathcal F,\mathbb P)\). Write \(p_{ij}(h)=\mathbb P(X_{s+h}=j\mid X_s=i)\) for its transition probability over an interval of length \(h\ge0\); time homogeneity makes this independent of \(s\). For each \(h\), the matrix \(P(h)=(p_{ij}(h))\) is row-stochastic: its entries are nonnegative and each row sums to \(1\). Also \(P(0)=I\), so \(p_{ij}(0)=0\) when \(i\ne j\), and \(p_{ii}(0)=1\).
The ratio defining \(q_{ij}\) is meaningful for \(h>0\), and its limit concerns \(h\downarrow0\). When \(q_{ij}\) exists, the definition says \(p_{ij}(h)=q_{ij}h+o(h)\): the remainder divided by \(h\) tends to zero. For a state \(i\), the corresponding probability of staying put is not \(q_{ii}h\). It starts at \(1\), so its expansion has the form \(1+q_{ii}h+o(h)\).
The Finite-State Expansion
Proof. For \(i\ne j\), \(p_{ij}(0)=0\). The definition of the right derivative gives \([p_{ij}(h)-p_{ij}(0)]/h\to q_{ij}\), which is exactly \(p_{ij}(h)=q_{ij}h+o(h)\). For a diagonal entry, the same derivative definition at \(p_{ii}(0)=1\) gives \(p_{ii}(h)=1+q_{ii}h+o(h)\). Because \(S\) is finite, the row-sum identity \(1=\sum_{j\in S}p_{ij}(h)\) can be differentiated term by term at zero: the derivative of the constant row sum is \(0=\sum_{j\in S}q_{ij}\). Rearranging gives \(q_{ii}=-\sum_{j\ne i}q_{ij}\), and substitution into the diagonal expansion gives the stated formula. \(\square\)
Thus, from state \(i\), the short-interval probability of ending at a particular different state \(j\) is \(q_{ij}h+o(h)\). In a finite state space, the probability of ending anywhere other than \(i\) is $$ 1-p_{ii}(h)=\left(\sum_{j\ne i}q_{ij}\right)h+o(h). $$ The coefficient is the sum of the off-diagonal rates out of \(i\). This statement concerns the state at the interval’s endpoint. It does not by itself count how many jumps occurred inside the interval; a path could leave and return before the endpoint.
For a process with filtration \(\mathcal F_s=\sigma(X_u:0\le u\le s)\), the Markov property gives the same short-time description conditional on the observed history. In particular, for each \(j\in S\), $$ \mathbb E[\mathbf 1_{\{X_{s+h}=j\}}\mid\mathcal F_s]=p_{X_sj}(h)\quad\text{almost surely}. $$ This conditional expectation is defined even when a particular event \(\{X_s=i\}\) has probability zero; the displayed equality is an almost-sure statement, not a claim about conditioning on a zero-probability event.
Proof. The Markov property gives the conditional expectation identity. On the event \(\{X_s=i\}\), its right-hand side equals the deterministic number \(p_{ij}(h)\). Applying the finite-state theorem’s expansions to that number gives the two stated conditional probabilities. Since the Markov identity holds almost surely, the resulting conditional statements also hold almost surely. \(\square\)
Worked Examples
Worked Example: A Two-State Chain
Consider a two-state chain on \(S=\{1,2\}\) with the following transition semigroup, for \(t\ge0\):
Each row sums to \(1\): the first has sum \(\tfrac13+\tfrac23e^{-3t}+\tfrac23-\tfrac23e^{-3t}=1\); the second has sum \(\tfrac13-\tfrac13e^{-3t}+\tfrac23+\tfrac13e^{-3t}=1\). Its entries are nonnegative for \(t\ge0\). To see the semigroup structure, let \(\Pi\) have both rows equal to \((\tfrac13,\tfrac23)\). Then \(\Pi^2=\Pi\), and \(P(t)=\Pi+e^{-3t}(I-\Pi)\). The identities \(\Pi(I-\Pi)=(I-\Pi)\Pi=0\) and \((I-\Pi)^2=I-\Pi\) give \(P(s)P(t)=P(s+t)\).
For the rate from \(1\) to \(2\), divide the corresponding entry by \(h>0\): $$ \frac{p_{12}(h)}h=\frac23\frac{1-e^{-3h}}h\longrightarrow\frac23\cdot3=2. $$ Thus \(q_{12}=2\). Similarly, \(p_{21}(h)/h=\tfrac13(1-e^{-3h})/h\to1\), so \(q_{21}=1\). The diagonal expansion from state \(1\) is \(p_{11}(h)=1-2h+o(h)\); from state \(2\) it is \(p_{22}(h)=1-h+o(h)\).
Check: The diagonal rates are \(q_{11}=-2\) and \(q_{22}=-1\), matching the negative sums of the off-diagonal rates in their rows. At a positive interval such as \(h=\tfrac13\), the probability of going from \(1\) to \(2\) is \(\tfrac23(1-e^{-1})\), not \(2\). The rate is the limiting probability-per-unit-time as the interval shrinks.
Worked Example: Three States with Equal Total Exit Rates
Construct a chain on \(S=\{1,2,3\}\) using a Poisson clock of rate \(4\). At each clock event, update the state according to the row-stochastic matrix $$ R= \begin{bmatrix} 0&\tfrac12&\tfrac12\\ \tfrac14&0&\tfrac34\\ \tfrac12&\tfrac12&0 \end{bmatrix}. $$ The row sums are \(0+\tfrac12+\tfrac12=1\), \(\tfrac14+0+\tfrac34=1\), and \(\tfrac12+\tfrac12+0=1\). Let \(N_h\) count clock events in an interval of length \(h\). Independence of the clock and the updates gives the exact transition matrix $$ P(h)=e^{-4h}\sum_{m=0}^{\infty}\frac{(4h)^m}{m!}R^m. $$ Each \(R^m\) is row-stochastic: products of row-stochastic matrices have row sums \(1\). The nonnegative Poisson weights sum to \(e^{-4h}e^{4h}=1\), so every row of \(P(h)\) sums to \(1\).
The probability of two or more clock events is \(1-e^{-4h}(1+4h)=O(h^2)\). Therefore the terms with \(m\ge2\) contribute only \(O(h^2)\), and $$ P(h)=e^{-4h}\bigl(I+4hR\bigr)+O(h^2) =I+4h(R-I)+O(h^2). $$ Here \(e^{-4h}=1-4h+O(h^2)\); multiplying gives the displayed first-order matrix. Hence $$ 4(R-I)= \begin{bmatrix} -4&2&2\\ 1&-4&3\\ 2&2&-4 \end{bmatrix}. $$ For instance, \(p_{12}(h)=2h+O(h^2)\), \(p_{23}(h)=3h+O(h^2)\), and \(p_{33}(h)=1-4h+O(h^2)\). Thus the rates from state \(1\) to states \(2,3\) are \(2,2\); those from state \(2\) to states \(1,3\) are \(1,3\); and those from state \(3\) to states \(1,2\) are \(2,2\).
Check: The first-order coefficients in each row sum to \(-4+2+2=0\), \(1-4+3=0\), and \(2+2-4=0\). The exact transition matrix has rows summing to \(1\), and the first-order expansion is consistent with that exact property.
Worked Example: The Poisson Process on a Countable State Space
Let \((N_t)_{t\ge0}\) be a Poisson process of rate \(3\), with state space \(S=\{0,1,2,\ldots\}\). Starting from state \(n\), the probability of being at \(n+k\) after an interval of length \(h\ge0\) is $$ p_{n,n+k}(h)=e^{-3h}\frac{(3h)^k}{k!},\qquad k=0,1,2,\ldots. $$ For each \(n\), the row sums to \(e^{-3h}\sum_{k=0}^\infty(3h)^k/k!=e^{-3h}e^{3h}=1\).
For \(k=1\), \(p_{n,n+1}(h)/h=3e^{-3h}\to3\), so \(q_{n,n+1}=3\). For each fixed \(k\ge2\), the ratio \(p_{n,n+k}(h)/h=3^kh^{k-1}e^{-3h}/k!\to0\). Thus the only nonzero off-diagonal rate from \(n\) is \(q_{n,n+1}=3\). The stay probability is \(p_{nn}(h)=e^{-3h}=1-3h+o(h)\).
Check: The probability of ending at a state different from \(n\) is \(1-e^{-3h}=3h+o(h)\), agreeing with the single off-diagonal rate. For example, the probability of two or more arrivals is \(1-e^{-3h}(1+3h)=O(h^2)\), so it does not change the first-order rate.
Countable State Spaces and a Common Pitfall
The finite-state proof differentiates a finite row sum. On a countably infinite state space, knowing each individual limit \(p_{ij}(h)/h\to q_{ij}\) for \(j\ne i\) does not, by itself, justify exchanging a limit with the infinite sum over destinations. To conclude that the diagonal rate is the negative sum of all off-diagonal rates, one needs an additional condition that controls the contribution from the infinitely many destinations. One sufficient condition is that the total exit-probability ratio \((1-p_{ii}(h))/h\) converges to the sum of the individual off-diagonal limits. In the Poisson example this identity can be checked directly from the exact row probabilities. The finite-state theorem does not require this extra qualification because its sum has only finitely many terms.
Common mistake: treating a diagonal rate as a probability. Suppose the rates out of state \(1\) to states \(2\) and \(3\) are \(2\) and \(1\). A wrong calculation is \(p_{11}(h)\approx q_{11}h=-3h\). It omits the initial probability \(p_{11}(0)=1\), and can even give a negative value. The correct first-order calculation is \(q_{11}=-(2+1)=-3\), followed by $$ p_{11}(h)=1+q_{11}h+o(h)=1-3h+o(h). $$ The row then has first-order probabilities \(1-3h\), \(2h\), and \(h\), whose sum is \(1\) up to terms \(o(h)\).
Small-time expansions are useful because they turn transition behavior into local quantities that can be compared across models and used to build differential equations for transition probabilities. The expansions require the stated right derivatives; they are not automatic merely because a matrix of transition probabilities has been written down. When rates are obtained from a model, check the row sums of its transition probabilities first, then take the limits, and finally verify that the diagonal first-order change balances the off-diagonal changes whenever the finite-state result applies.
Check Your Understanding
Use the definitions and expansions above. When a question refers to a limit, take \(h\downarrow0\).
- If \(p_{12}(h)=5h+o(h)\), what is the rate \(q_{12}\), and what does that rate say about a short interval?
- In a finite-state chain, the rates from state \(i\) to two other states are \(4\) and \(2\). Find \(q_{ii}\) and the first-order expansion of \(p_{ii}(h)\).
- Why can the finite-state row-sum proof not be applied term by term to an infinite sum without an additional condition?
- For the Poisson process example, calculate the rate from \(n\) to \(n+2\), and explain why a two-arrival event does not affect the first-order exit rate.
- Identify the error in writing \(p_{ii}(h)\approx q_{ii}h\) when \(q_{ii}<0\).