From a Short First Interval to a Differential Equation
For a continuous-time Markov chain, the generator describes what can happen during a very short interval. The backward Kolmogorov equation connects those local rates to transition probabilities over an arbitrary time horizon. Its derivation begins by asking where the chain is after a short initial interval, and then using the Markov property for the time that remains.
Throughout, let \((X_t)_{t\ge0}\) be a time-homogeneous continuous-time Markov chain on a finite state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Use the natural filtration \(\mathcal{F}_t=\sigma(X_s:0\le s\le t)\). Write \(\mathbb{P}_i\) for probabilities when \(X_0=i\), and define the transition probabilities and transition matrix by \(p_{ij}(t)=\mathbb{P}_i(X_t=j)\) and \(P(t)=(p_{ij}(t))\). Matrices are row-stochastic, so each row of \(P(t)\) sums to \(1\).
Let \(Q=(q_{ij})\) be the generator. For \(i\ne j\), \(q_{ij}\ge0\) is the rate of a transition from \(i\) to \(j\); the diagonal entry is \(q_{ii}=-\sum_{j\ne i}q_{ij}\). Thus every row of \(Q\) sums to \(0\). We use the finite-state small-time expansion \(P(h)=I+hQ+o(h)\) as \(h\downarrow0\), where the remainder divided by \(h\) tends to zero entrywise. The transition semigroup satisfies the Chapman–Kolmogorov identity \(P(s+t)=P(s)P(t)\).
The word “backward” reflects the role of the starting state: the first index \(i\) is varied by the rates \(q_{ik}\), while the remaining transition probability is \(p_{kj}(t)\). To see this directly, fix \(h>0\) and condition on \(X_h\). Given \(X_0=i\), the probability of \(X_h=k\) is \(p_{ik}(h)\); after that, the probability of reaching \(j\) a further time \(t\) later is \(p_{kj}(t)\), by the Markov property. Therefore
This conditioning is valid for each starting state \(i\); it does not require conditioning on an event of probability zero. In matrix form the identity is \(P(t+h)=P(h)P(t)\). Subtract \(P(t)\), divide by \(h\), and use the small-time expansion for the first interval. This gives the right-hand derivative. To establish the ordinary derivative at positive times, the left-hand quotient must also be checked.
Proof. Since \(S\) is finite, the matrix limit \((P(h)-I)/h\to Q\) can be multiplied by any fixed matrix, and entrywise limits commute with the finite sums in matrix multiplication. For \(t\ge0\), the right-hand quotient is $$ \frac{P(t+h)-P(t)}{h} =\frac{P(h)-I}{h}P(t)\ \longrightarrow\ QP(t) \qquad (h\downarrow0). $$ For \(t>0\) and \(0<h<t\), Chapman–Kolmogorov also gives \(P(t)=P(h)P(t-h)\). Thus the left-hand quotient is $$ \frac{P(t)-P(t-h)}{h} =\frac{P(h)-I}{h}P(t-h). $$ The transition semigroup is continuous in time, so \(P(t-h)\to P(t)\) as \(h\downarrow0\). Combining this with \((P(h)-I)/h\to Q\) shows that the left-hand quotient also tends to \(QP(t)\). The two one-sided limits agree, proving differentiability and \(P'(t)=QP(t)\). At \(t=0\), \(P(0)=I\), since \(p_{ij}(0)=\mathbb{P}_i(X_0=j)=1\) if \(i=j\) and \(0\) otherwise. The right-hand quotient at zero is \((P(h)-I)/h\to Q\). Taking the \((i,j)\) entry of the matrix equation gives the stated component equation. \(\square\)
Worked Applications
Worked Example: A Two-State Switching Chain
Consider \(S=\{1,2\}\), with rate \(2\) from state \(1\) to state \(2\), and rate \(1\) from state \(2\) to state \(1\). The generator is $$ Q=\begin{bmatrix}-2&2\\1&-1\end{bmatrix}. $$ Before applying the equation, check the generator rows: \(-2+2=0\) and \(1-1=0\).
For each terminal state \(j\in\{1,2\}\), the backward equation reads $$ p'_{1j}(t)=-2p_{1j}(t)+2p_{2j}(t),\qquad p'_{2j}(t)=p_{1j}(t)-p_{2j}(t). $$ The initial condition \(P(0)=I\) gives \(p_{11}(0)=1\), \(p_{12}(0)=0\), \(p_{21}(0)=0\), and \(p_{22}(0)=1\). Substituting these values into the equations gives $$ P'(0)=QP(0)= \begin{bmatrix}-2&2\\1&-1\end{bmatrix} \begin{bmatrix}1&0\\0&1\end{bmatrix} = \begin{bmatrix}-2&2\\1&-1\end{bmatrix}. $$ For example, \(p'_{12}(0)=-2(0)+2(1)=2\), and \(p'_{21}(0)=1(1)-1(0)=1\).
Check: The derivative of the sum of the two entries in row \(1\) at zero is \(-2+2=0\); in row \(2\) it is \(1-1=0\). These are the generator row sums, as required for transition probabilities whose rows sum to \(1\).
Worked Example: Reaching an Absorbing State
Let \(S=\{1,2,3\}\). From state \(1\), the chain jumps to state \(2\) at rate \(2\) and to state \(3\) at rate \(1\). From state \(2\), it jumps to state \(1\) at rate \(1\) and to state \(3\) at rate \(3\). State \(3\) is absorbing. The generator is $$ Q=\begin{bmatrix}-3&2&1\\1&-4&3\\0&0&0\end{bmatrix}. $$ Its row sums are \(-3+2+1=0\), \(1-4+3=0\), and \(0+0+0=0\).
Focus on the probability of being in state \(3\) after time \(t\). The equation for the third column of \(P(t)\) is $$ p'_{13}(t)=-3p_{13}(t)+2p_{23}(t)+p_{33}(t),\quad p'_{23}(t)=p_{13}(t)-4p_{23}(t)+3p_{33}(t),\quad p'_{33}(t)=0. $$ The initial values from \(P(0)=I\) are \(p_{13}(0)=0\), \(p_{23}(0)=0\), and \(p_{33}(0)=1\). Consequently, $$ p'_{13}(0)=-3(0)+2(0)+1(1)=1,\qquad p'_{23}(0)=1(0)-4(0)+3(1)=3,\qquad p'_{33}(0)=0. $$ In particular, starting from state \(1\), the probability of being in state \(3\) grows initially at rate \(1\), exactly the direct jump rate from \(1\) to \(3\).
Check: The initial derivative of the full transition matrix is \(QP(0)=Q\), so the derivatives in its third column must be \(q_{13}=1\), \(q_{23}=3\), and \(q_{33}=0\). They match the values computed from the backward equations.
Worked Example: Two Sequential Jumps
Take \(S=\{1,2,3\}\), with a jump from \(1\) to \(2\) at rate \(4\), a jump from \(2\) to \(3\) at rate \(2\), and state \(3\) absorbing. Then $$ Q=\begin{bmatrix}-4&4&0\\0&-2&2\\0&0&0\end{bmatrix}. $$ The row sums are \(-4+4+0=0\), \(0-2+2=0\), and \(0+0+0=0\).
For the probability of being in state \(3\), the backward equations are $$ p'_{13}(t)=-4p_{13}(t)+4p_{23}(t),\qquad p'_{23}(t)=-2p_{23}(t)+2p_{33}(t),\qquad p'_{33}(t)=0. $$ Initially \(p_{13}(0)=0\), \(p_{23}(0)=0\), and \(p_{33}(0)=1\). Hence \(p'_{13}(0)=0\) and \(p'_{23}(0)=2\). Differentiating the first equation once more gives $$ p''_{13}(0)=-4p'_{13}(0)+4p'_{23}(0) =-4(0)+4(2)=8. $$ Thus the initial growth of \(p_{13}(t)\) is second order: \(p_{13}(t)=4t^2+o(t^2)\). The coefficient can also be checked from the two required jumps. For small \(t\), the probability of a first jump at time \(s\) and then a second jump before \(t\) is $$ \int_0^t 4e^{-4s}\bigl(1-e^{-2(t-s)}\bigr)\,ds. $$ For \(0\le s\le t\), the integrand is \(8(t-s)+O(t^2)\) as \(t\downarrow0\), so the integral is \(8\int_0^t(t-s)\,ds+O(t^3)=4t^2+O(t^3)\). This agrees with the coefficient obtained from the backward equation.
Check: The first derivative \(p'_{13}(0)=q_{13}=0\) correctly records that there is no direct jump from state \(1\) to state \(3\). The second derivative calculation and the two-jump calculation both give the same leading term, \(4t^2\).
Why the Diagonal Rate Matters
The diagonal entries of \(Q\) account for the probability that the chain remains in its current state during the short interval. They are not optional corrections: they make each generator row sum to zero and ensure that the backward equation tracks the loss of probability from a starting state as well as gains from other states.
Common mistake. In the two-state example, one might count only jumps out of state \(1\) and write \(p'_{11}(0)=q_{12}p_{21}(0)\), omitting the no-jump contribution. That gives \(2(0)=0\), which is wrong. The correct calculation includes the diagonal term: $$ p'_{11}(0)=q_{11}p_{11}(0)+q_{12}p_{21}(0) =(-2)(1)+2(0)=-2. $$ This negative derivative is necessary: over a short interval the probability of still being in state \(1\) decreases at rate \(2\). More generally, conditioning on the first short interval must include every possible state at its end, including the original state.
A useful consistency check follows by summing the equation over the terminal state \(j\). Since each row of \(P(t)\) sums to \(1\), the derivative of each row sum must be zero; the generator row sums ensure exactly that.
Proof. Since \(S\) is finite, interchange the finite sums and use the backward equation: $$ \sum_{j\in S}p'_{ij}(t) =\sum_{j\in S}\sum_{k\in S}q_{ik}p_{kj}(t) =\sum_{k\in S}q_{ik}\left(\sum_{j\in S}p_{kj}(t)\right). $$ Each row of \(P(t)\) sums to \(1\), so the expression equals \(\sum_{k\in S}q_{ik}=0\), the sum of row \(i\) of the generator. Thus the derivative of the total probability is zero. \(\square\)
Check Your Understanding
Use the first-interval conditioning identity and the initial condition to check your reasoning.
- For a two-state chain with rates \(3\) from state \(1\) to state \(2\) and \(2\) from state \(2\) to state \(1\), write \(Q\) and the backward equation for \(p_{12}(t)\).
- Explain why \(P(0)=I\), including the values of \(p_{ij}(0)\) when \(i\ne j\).
- For the sequential-jump chain in the third example, why is \(p'_{13}(0)=0\), even though \(p_{13}(t)\) becomes positive for \(t>0\)?
- In the absorbing-state example, calculate \(p'_{23}(0)\) directly from the third-column equation.
- What term is omitted if one conditions only on jumps to a different state during the first short interval? Why can that omission give an incorrect derivative?