Condition on the Final Interval
To find how transition probabilities change over time, examine what can happen in a short interval at the end of the time period. Probability can arrive at a state from other states, or leave it for another state. The forward Kolmogorov equation makes this balance precise: it expresses the time derivative of a transition probability using the probabilities already present at time \(t\) and the rates of transitions during the final short interval.
Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Let \((X_t)_{t\ge0}\) be a time-homogeneous continuous-time Markov chain with finite state space \(S\), and use its natural filtration \(\mathcal{F}_t=\sigma(X_u:0\le u\le t)\). For \(i,j\in S\), define the transition kernel by \(p_{ij}(t)=\mathbb{P}_i(X_t=j)\), where \(\mathbb{P}_i\) is the law of the chain started at \(i\). This defines the kernel for every state, including states that have probability zero under some other initial distribution. Write \(P(t)=(p_{ij}(t))\); it is row-stochastic, so each row sums to \(1\). The Markov property gives \(\mathbb{P}(X_{s+t}=j\mid\mathcal{F}_s)=p_{X_s,j}(t)\) almost surely.
The generator is the finite matrix \(Q=(q_{ij})\) defined by the small-time limits \(q_{ij}=\lim_{h\downarrow0}(p_{ij}(h)-\mathbf{1}_{\{i=j\}})/h\). Its off-diagonal entries are nonnegative transition rates and each row sums to zero. In particular, for \(i\ne j\), \(p_{ij}(h)=q_{ij}h+o(h)\), while \(p_{ii}(h)=1+q_{ii}h+o(h)\). Equivalently, \(P(h)=I+hQ+o(h)\) entrywise. Because \(S\) is finite, these entrywise remainders can be used in each of the finite sums below.
The key step is conditioning on the state at time \(t\), just before the final interval of length \(h\). By the Chapman–Kolmogorov equation, $$ p_{ij}(t+h)=\sum_{k\in S}p_{ik}(t)p_{kj}(h). $$ This uses transition kernels \(\mathbb{P}_k(X_h=j)\), not conditional probabilities given an event that might have probability zero. Subtract \(p_{ij}(t)\), divide by \(h>0\), and let \(h\downarrow0\). The proof also establishes the continuity needed to obtain the ordinary derivative at positive times.
Proof. In matrix form, Chapman–Kolmogorov gives \(P(t+h)=P(t)P(h)\). Hence $$ \frac{P(t+h)-P(t)}{h} =P(t)\frac{P(h)-I}{h}\longrightarrow P(t)Q $$ as \(h\downarrow0\), by the defining small-time limit for \(Q\). This proves the right derivative, including at \(t=0\). To obtain the left derivative at \(t>0\), first note that \(P(t-h)\to P(t)\) as \(h\downarrow0\): the right-increment formula gives \(P(t)-P(t-h)=P(t-h)(P(h)-I)\), whose entries tend to zero because the entries of \(P(t-h)\) are bounded by \(1\) and \(P(h)-I\to0\). Now $$ \frac{P(t)-P(t-h)}{h} =P(t-h)\frac{P(h)-I}{h}\longrightarrow P(t)Q. $$ Thus the left and right derivatives agree for \(t>0\), proving the matrix equation. Taking its \((i,j)\) entry gives the stated sum. \(\square\)
Reading the Equation as Probability Flow
For a fixed destination state \(j\), separate the term \(k=j\) from the sum. Define the exit rate from \(j\) as \(q_j=-q_{jj}=\sum_{k\ne j}q_{jk}\). Then $$ \frac{d}{dt}p_{ij}(t) =\sum_{k\ne j}p_{ik}(t)q_{kj}-p_{ij}(t)q_j. $$
The first term adds flow into \(j\): for each other state \(k\), the probability of being at \(k\) at time \(t\) is multiplied by the rate \(q_{kj}\) of moving from \(k\) to \(j\). The second term subtracts flow out of \(j\): probability \(p_{ij}(t)\) is multiplied by the total exit rate from \(j\). If \(j\) is absorbing, then \(q_j=0\) and there is no outgoing term. If no other state can enter \(j\), then the incoming sum is zero. The formula covers both cases without special conventions.
Proof. Sum the forward equation over \(j\). Since \(S\) is finite, interchange the finite sums: $$ \sum_{j\in S}\frac{d}{dt}p_{ij}(t) =\sum_{k\in S}p_{ik}(t)\sum_{j\in S}q_{kj} =\sum_{k\in S}p_{ik}(t)\cdot0=0, $$ because every row of the generator sums to zero. Thus the total probability in row \(i\) has zero derivative. Since \(P(t)\) is a transition matrix, that total is \(1\), consistent with the calculation. \(\square\)
The same equation describes the distribution of a chain with an arbitrary initial distribution. If \(\mu\) is a row probability vector and \(\alpha(t)=\mu P(t)\), then \(\alpha(t)\) is the row distribution at time \(t\), and differentiation gives \(\alpha'(t)=\alpha(t)Q\). Indeed, multiplying the forward equation on the left by \(\mu\) gives \(\mu P'(t)=\mu P(t)Q\), which is exactly this identity. Thus the transition-probability equation also yields probability-flow equations for the state probabilities of a particular population or system.
Worked Flow Calculations
Worked Example: A Symmetric Two-State Chain
Consider \(S=\{0,1\}\), with transitions in either direction at rate \(1\). Its generator is $$ Q=\begin{bmatrix}-1&1\\1&-1\end{bmatrix}. $$ Both row sums are \(-1+1=0\) and \(1-1=0\), as required. For this chain the number of jumps by time \(t\) has a Poisson distribution with mean \(t\). Starting at \(0\), the chain is at \(0\) after an even number of jumps and at \(1\) after an odd number. Summing the even and odd terms of the Poisson series gives \(p_{00}(t)=(1+e^{-2t})/2\) and \(p_{01}(t)=(1-e^{-2t})/2\); symmetry gives the other row.
Take \(t=(\log 2)/2\), so \(e^{-2t}=1/2\). The transition matrix at that time is $$ P(t)=\begin{bmatrix}\tfrac34&\tfrac14\\\tfrac14&\tfrac34\end{bmatrix}. $$ Each row sums to \(1\). Multiply by \(Q\): $$ P(t)Q= \begin{bmatrix} -\tfrac34+\tfrac14&\tfrac34-\tfrac14\\ -\tfrac14+\tfrac34&\tfrac14-\tfrac34 \end{bmatrix} = \begin{bmatrix}-\tfrac12&\tfrac12\\\tfrac12&-\tfrac12\end{bmatrix}. $$ For example, the rate of change of \(p_{01}\) is \(\tfrac34\cdot1+\tfrac14\cdot(-1)=\tfrac12\): flow arrives from state \(0\) at rate \(1\), while probability at state \(1\) flows out at rate \(1\). Direct differentiation gives \(p'_{01}(t)=e^{-2t}\), which at the chosen time is \(1/2\), agreeing with the forward equation. Each derivative row sums to zero, as the conservation result requires.
Worked Example: Instantaneous Flow in a Three-State Cycle
Let \(S=\{0,1,2\}\), with rates \(2\) from \(0\) to \(1\), \(1\) from \(1\) to \(2\), and \(3\) from \(2\) to \(0\). The generator is $$ Q=\begin{bmatrix}-2&2&0\\0&-1&1\\3&0&-3\end{bmatrix}. $$ The row sums are \(-2+2+0=0\), \(0-1+1=0\), and \(3+0-3=0\). Start from the distribution \(\mu=(\tfrac12,\tfrac13,\tfrac16)\), whose entries sum to \(1\). The state distribution is \(\alpha(t)=\mu P(t)\), so its initial derivative is \(\alpha'(0)=\mu Q\), since \(P(0)=I\) and the forward equation gives \(P'(0)=Q\).
Compute each component: $$ \begin{aligned} \alpha'_0(0)&=\tfrac12(-2)+\tfrac13(0)+\tfrac16(3)=-1+\tfrac12=-\tfrac12,\\ \alpha'_1(0)&=\tfrac12(2)+\tfrac13(-1)+\tfrac16(0)=1-\tfrac13=\tfrac23,\\ \alpha'_2(0)&=\tfrac12(0)+\tfrac13(1)+\tfrac16(-3)=\tfrac13-\tfrac12=-\tfrac16. \end{aligned} $$ The flow interpretation gives the same values. State \(0\) loses probability at rate \(\tfrac12\cdot2=1\) and gains it from state \(2\) at rate \(\tfrac16\cdot3=\tfrac12\), for net change \(-\tfrac12\). State \(1\) gains \(\tfrac12\cdot2=1\) and loses \(\tfrac13\cdot1=\tfrac13\), for net change \(2/3\). State \(2\) gains \(\tfrac13\cdot1=1/3\) and loses \(\tfrac16\cdot3=1/2\), for net change \(-1/6\). As a check, \(-\tfrac12+\tfrac23-\tfrac16=0\); the total probability has no instantaneous change.
Worked Example: A Two-Stage Chain with an Absorbing State
Let \(S=\{1,2,3\}\), where the chain moves from \(1\) to \(2\) at rate \(1\), from \(2\) to \(3\) at rate \(1\), and state \(3\) is absorbing. The generator is $$ Q=\begin{bmatrix}-1&1&0\\0&-1&1\\0&0&0\end{bmatrix}. $$ The row sums are \(-1+1+0=0\), \(0-1+1=0\), and \(0+0+0=0\). Starting from state \(1\), the probability of still being in state \(1\) at time \(t\) is \(e^{-t}\). To be in state \(2\), the first transition must occur at some time \(s\in[0,t]\), followed by no second transition before \(t\). Therefore $$ p_{12}(t)=\int_0^t e^{-s}e^{-(t-s)}\,ds=te^{-t}. $$ The remaining probability is in state \(3\). Starting from \(2\), the probabilities of states \(2\) and \(3\) are \(e^{-t}\) and \(1-e^{-t}\), respectively. Thus $$ P(t)=\begin{bmatrix} e^{-t}&te^{-t}&1-(1+t)e^{-t}\\ 0&e^{-t}&1-e^{-t}\\ 0&0&1 \end{bmatrix}. $$ The first row sums to \(e^{-t}+te^{-t}+1-(1+t)e^{-t}=1\); the second sums to \(e^{-t}+1-e^{-t}=1\), and the third sums to \(1\).
At \(t=\log 2\), write \(r=\log 2\). Then \(e^{-t}=1/2\), so $$ P(t)=\begin{bmatrix} \tfrac12&\tfrac r2&\tfrac{1-r}{2}\\ 0&\tfrac12&\tfrac12\\ 0&0&1 \end{bmatrix}. $$ For any row \((x,y,z)\), multiplication by \(Q\) gives \((-x,x-y,y)\). Applying this to each row, $$ P(t)Q= \begin{bmatrix} -\tfrac12&\tfrac{1-r}{2}&\tfrac r2\\ 0&-\tfrac12&\tfrac12\\ 0&0&0 \end{bmatrix}. $$ Direct differentiation of the first row of \(P(t)\) gives \((-e^{-t},(1-t)e^{-t},te^{-t})\), which at \(t=r\) equals \((-\tfrac12,\tfrac{1-r}{2},\tfrac r2)\). The second row differentiates to \((0,-e^{-t},e^{-t})\), giving \((0,-\tfrac12,\tfrac12)\) at the same time. The third row is constant. These derivatives verify the matrix calculation and the flow balance, including zero outflow from the absorbing state.
A Common Error and the Scope of the Derivation
A frequent mistake is to count only flow out of the destination state and omit the incoming flow. In the symmetric two-state example at \(t=(\log2)/2\), the probability of being in state \(1\) when starting from \(0\) is \(1/4\), and the exit rate from state \(1\) is \(1\). Counting only departures would incorrectly give \(p'_{01}(t)=-1/4\). The correct calculation includes arrivals from state \(0\): $$ p'_{01}(t)=p_{00}(t)q_{01}+p_{01}(t)q_{11} =\tfrac34\cdot1+\tfrac14\cdot(-1)=\tfrac12. $$ The positive net derivative is sensible: at this time more probability is at state \(0\) than at state \(1\), so the rate of flow into state \(1\) exceeds the rate flowing back out.
The assumptions matter. The equation here uses a time-homogeneous transition semigroup and a finite state space with a finite generator. In particular, the final-interval transition probabilities have the same small-time rates \(Q\) at every time \(t\). If rates depend on time, the fixed-matrix equation as stated need not apply; the final-interval rates must then be evaluated at the time of that interval. Also, finiteness lets us pass from the entrywise generator limits through finite sums without additional conditions. The derivation is not a proof for arbitrary infinite-state chains, where controlling infinite sums and possible explosion requires further hypotheses.
The practical procedure is direct: check that the generator has nonnegative off-diagonal rates and rows summing to zero; identify the state probabilities at time \(t\); then, for each destination, add all incoming rate-weighted probabilities and subtract the destination's probability times its exit rate. The row-sum check on the resulting derivatives is a useful final safeguard: their sum must be zero.
Check Your Understanding
For each question, keep the row-vector convention and interpret derivatives as net probability flow.
- For a finite-state time-homogeneous chain, what identity conditions on the state at time \(t\) to express \(p_{ij}(t+h)\) using the last interval of length \(h\)?
- In the three-state cycle, identify separately the incoming and outgoing contributions to \(\alpha'_2(0)\).
- Suppose \(j\) is absorbing. Which terms vanish in the flow formula for \(\frac{d}{dt}p_{ij}(t)\), and why?
- For the two-stage chain, use the displayed formula for \(P(t)\) to verify that the derivative of its first row sums to zero.
- Explain why defining \(p_{ij}(t)\) as \(\mathbb{P}_i(X_t=j)\) avoids an undefined conditional probability for a state that has zero probability under a chosen initial law.