The Generator and the Small-Time Expansion

Why the Generator Describes Short Intervals

For a continuous-time Markov chain, the transition semigroup records probabilities over every time interval, while the generator records their instantaneous rates of change at time zero. The connection between them is local: over a short interval of length \(h\), the probability of moving from one state to another is approximately a rate multiplied by \(h\). The diagonal entries describe the matching first-order loss from the starting state.

Let \((X_t)_{t\ge0}\) be a time-homogeneous, finite-state continuous-time Markov chain on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with state space \(S\). Its transition probabilities are \(p_{ij}(t)=\mathbb{P}(X_t=j\mid X_0=i)\), understood as transition-kernel values defined for every starting state \(i\). Write \(P(t)=(p_{ij}(t))\). Each \(P(t)\) is row-stochastic: its entries are nonnegative and every row sums to \(1\).

Here \(o(h)\) is entrywise; because \(S\) is finite, there are only finitely many remainders to control. The generator definition gives the expansion one entry at a time. The stochastic rows of \(P(h)\) additionally force the familiar rate constraints: off-diagonal entries of \(Q\) are nonnegative, and each row of \(Q\) sums to zero.

Proof. Fix \(i,j\). By the definition of \(q_{ij}\), the quotient \((p_{ij}(h)-\mathbf{1}_{\{i=j\}})/h\) tends to \(q_{ij}\). Subtracting \(q_{ij}\) from this quotient shows that \(r_{ij}(h)=p_{ij}(h)-\mathbf{1}_{\{i=j\}}-h q_{ij}\) satisfies \(r_{ij}(h)/h\to0\). This proves the entrywise expansion. If \(j\ne i\), then \(p_{ij}(h)\ge0\), so \(q_{ij}=\lim_{h\downarrow0}p_{ij}(h)/h\ge0\). Finally, for each \(h\), \(\sum_jp_{ij}(h)=1\). Subtracting \(\sum_j\mathbf{1}_{\{i=j\}}=1\), dividing by \(h\), and taking the limit gives \(\sum_j q_{ij}=0\); the sum has finitely many terms, so its limit is the sum of the limits. Therefore \(q_{ii}=-\sum_{j\ne i}q_{ij}=-q_i\). \(\square\)

Reading the Off-Diagonal and Diagonal Entries

For distinct states \(i\) and \(j\), the expansion says \(p_{ij}(h)=h q_{ij}+o(h)\). Thus \(q_{ij}\) is the first-order rate of transition from \(i\) to \(j\). The total rate of leaving \(i\) is \(q_i=\sum_{j\ne i}q_{ij}\), and the diagonal expansion is \(p_{ii}(h)=1-hq_i+o(h)\).

There is an important distinction in that last expression. \(p_{ii}(h)\) is the probability of being in \(i\) at the end of the interval; it is not, in general, the probability of making no jump during the interval. A chain can leave \(i\) and return before time \(h\). For a finite-state CTMC with exponential holding times, the probability of no jump from \(i\) during the interval is \(e^{-q_i h}\), which has the same first-order expansion \(1-q_i h+o(h)\). Return paths affect endpoint probabilities only at smaller order than \(h\).

Proof. Let \(\Lambda=\max_{i\in S}q_i\), which is finite because \(S\) is finite. When \(q_i>0\), the probability that the first jump from \(i\) occurs by time \(h\) and goes to \(j\) is \(\int_0^h e^{-q_i s}q_{ij}\,ds=q_{ij}(1-e^{-q_i h})/q_i\). For \(x\ge0\), \(0\le x-(1-e^{-x})=\int_0^x(1-e^{-u})\,du\le\int_0^x u\,du=x^2/2\), where \(1-e^{-u}\le u\). Taking \(x=q_i h\) shows that this first-jump probability differs from \(q_{ij}h\) by at most \(q_{ij}q_i h^2/2\). The probability of a first jump by time \(h\) is at most \(\Lambda h\), since \(1-e^{-q_i h}\le q_i h\le\Lambda h\). Given a first jump, the conditional probability of a second jump before time \(h\) is at most \(\Lambda h\), by the same bound at the state then occupied. Thus the probability of at least two jumps is at most \(\Lambda^2h^2\). For \(j\ne i\), the event that the endpoint is \(j\) and the event that the first jump is to \(j\) can differ only on paths with at least two jumps. This proves the claimed first-order interpretation. When \(q_i=0\), all \(q_{ij}=0\) for \(j\ne i\), and the holding time is infinite, so the assertion also holds in that case. \(\square\)

Worked Short-Time Expansions

A Common Mistake and Why the Expansion Matters

Common mistake. From the exit rate \(q_i\), one might write \(p_{ii}(h)=e^{-q_i h}\) exactly. That formula gives the probability of no jump from \(i\), not necessarily the probability of ending in \(i\). In the two-state example above, the wrong calculation would set \(p_{aa}(h)=e^{-h}\). The correction is to include paths that leave and return: \(p_{aa}(h)=1-h+o(h)\), and the exactly-two-jump paths alone contribute \(h^2+O(h^3)\). The exact no-jump formula and the endpoint probability agree to first order, but need not agree at a fixed positive \(h\).

The small-time expansion is the bridge from rate specifications to transition probabilities over short intervals. It also explains why each generator row must sum to zero: to first order, probability lost from the diagonal is exactly distributed among the off-diagonal destinations. The expansion is local; it does not by itself give a formula for \(P(t)\) at arbitrary times. That calculation requires additional arguments about the semigroup.