Chapman-Kolmogorov Composition and Differential Equations

Why Composition Produces Differential Equations

A transition probability over a long interval can be calculated by conditioning on the state at an intermediate time. This composition rule is more than a way to break a calculation into pieces: when the intermediate time is moved, the short-interval transition probabilities determine how the whole transition matrix changes. That connection yields the forward and backward Kolmogorov equations.

The distinction between the two equations is the time being varied. The forward equation changes the terminal time; the backward equation changes the starting time. With row probabilities and the matrix convention used here, the generator appears on different sides of the transition matrix. Keeping both the time argument and the matrix order straight is essential.

Definition: Consider a finite-state, time-inhomogeneous continuous-time Markov chain. Let \(P(s,t)\) be its transition matrix from time \(s\) to time \(t\), where \(s\leq t\), and let \(P_{ij}(s,t)\) be the probability of being in state \(j\) at time \(t\), given state \(i\) at time \(s\). Matrices use row-to-column probabilities, so each row of \(P(s,t)\) sums to 1. The generator \(Q(t)=[q_{ij}(t)]\) has transition intensities \(q_{ij}(t)\geq 0\) for \(i\ne j\), and diagonal entries \(q_{ii}(t)=-\sum_{j\ne i}q_{ij}(t)\). Assume the intensities are continuous at the times where derivatives are taken and the chain is well-defined.

For \(s\leq u\leq t\), the Chapman–Kolmogorov composition rule is

$$ P(s,t)=P(s,u)P(u,t). $$

The multiplication sums over the possible state at time \(u\): the \((i,j)\) entry is \(\sum_k P_{ik}(s,u)P_{kj}(u,t)\). The intermediate state is the link between the two intervals.

From a Short Interval to the Forward Equation

The generator describes transitions over a short interval. For small positive \(h\), the transition matrix from \(t\) to \(t+h\) satisfies \(P(t,t+h)=I+hQ(t)+o(h)\), where \(I\) is the identity matrix and \(o(h)\) denotes terms that become negligible relative to \(h\) as \(h\) tends to zero.

Hold the starting time \(s\) fixed and apply composition at the intermediate time \(t\). Then \(P(s,t+h)=P(s,t)P(t,t+h)\). Subtract \(P(s,t)\), divide by \(h\), and take the limit. This gives the forward equation:

\frac{\partial}{\partial t}P(s,t)=P(s,t)Q(t).

The generator is on the right because the additional short-interval transition occurs after the transition from \(s\) to \(t\). This equation describes the rate of change in the columns of the transition matrix as the terminal time advances.

Worked Example: Advancing the Terminal Time

In a hypothetical two-state model, a covered life is either active (state 1) or disabled (state 2). Time is measured in years. At a particular terminal time \(t\), suppose the transition matrix from a fixed starting time \(s\) is

P(s,t)= \begin{pmatrix} 0.70&0.30\\ 0.20&0.80 \end{pmatrix}, \qquad Q(t)= \begin{pmatrix} -0.40&0.40\\ 0.30&-0.30 \end{pmatrix}.

The generator means the active-to-disabled intensity is \(0.40\) per year and the disabled-to-active intensity is \(0.30\) per year at time \(t\). Use the forward equation to find the instantaneous derivative of \(P(s,t)\) with respect to \(t\).

Method. Since the terminal time changes, calculate \(P(s,t)Q(t)\), in that order:

\begin{aligned} \frac{\partial}{\partial t}P(s,t) &= \begin{pmatrix} 0.70&0.30\\ 0.20&0.80 \end{pmatrix} \begin{pmatrix} -0.40&0.40\\ 0.30&-0.30 \end{pmatrix}\\ &= \begin{pmatrix} 0.70(-0.40)+0.30(0.30)&0.70(0.40)+0.30(-0.30)\\ 0.20(-0.40)+0.80(0.30)&0.20(0.40)+0.80(-0.30) \end{pmatrix}\\ &= \begin{pmatrix} -0.19&0.19\\ 0.16&-0.16 \end{pmatrix}. \end{aligned}

For example, the probability of being disabled at \(t\), given active status at \(s\), has instantaneous derivative \(0.19\) per year at this time. Each derivative row sums to zero, as required because each transition-probability row continues to sum to 1. Common mistake avoided: multiplying \(Q(t)P(s,t)\) would put the generator on the wrong side for a terminal-time derivative.

From Composition to the Backward Equation

Now hold the terminal time \(t\) fixed and move the starting time \(s\). For a small positive \(h\), composition at \(s+h\) gives \(P(s,t)=P(s,s+h)P(s+h,t)\). The short-interval matrix is \(P(s,s+h)=I+hQ(s)+o(h)\). Thus, to first order, \(P(s+h,t)\) is obtained by removing that initial short interval:

\begin{aligned} P(s+h,t) &=\bigl(I+hQ(s)+o(h)\bigr)^{-1}P(s,t)\\ &=\bigl(I-hQ(s)+o(h)\bigr)P(s,t). \end{aligned}

The second line follows because multiplying \(I+hQ(s)\) by \(I-hQ(s)\) gives \(I+o(h)\). Subtract \(P(s,t)\), divide by \(h\), and let \(h\) tend to zero. The backward equation follows:

\frac{\partial}{\partial s}P(s,t)=-Q(s)P(s,t).

Here the generator is on the left, and the minus sign appears because increasing the starting time removes an initial piece of the interval. The forward and backward equations therefore describe different changes; they are not interchangeable versions of the same multiplication.

Worked Example: Moving the Starting Time

For a hypothetical active/disabled model, assume time is measured in years and the transition matrix from \(s\) to a fixed future time \(t\) is

P(s,t)= \begin{pmatrix} 0.60&0.40\\ 0.10&0.90 \end{pmatrix}, \qquad Q(s)= \begin{pmatrix} -0.20&0.20\\ 0.50&-0.50 \end{pmatrix}.

The intensities at the starting time are \(0.20\) per year from active to disabled and \(0.50\) per year from disabled to active. Find the derivative with respect to \(s\), keeping \(t\) fixed.

Method. Apply \(-Q(s)P(s,t)\), multiplying the generator on the left:

\begin{aligned} Q(s)P(s,t) &= \begin{pmatrix} (-0.20)(0.60)+(0.20)(0.10)&(-0.20)(0.40)+(0.20)(0.90)\\ (0.50)(0.60)+(-0.50)(0.10)&(0.50)(0.40)+(-0.50)(0.90) \end{pmatrix}\\ &= \begin{pmatrix} -0.10&0.10\\ 0.25&-0.25 \end{pmatrix},\\ \frac{\partial}{\partial s}P(s,t) &= \begin{pmatrix} 0.10&-0.10\\ -0.25&0.25 \end{pmatrix}. \end{aligned}

Each row of the derivative again sums to zero. In particular, increasing the starting time changes the probability of ending disabled from an active start at rate \(-0.10\) per year at this point. Common mistake avoided: omitting the minus sign would reverse the effect of removing the initial interval. This is a transition-model calculation; no interest or discounting assumption is involved.

Why the Intermediate Time Does Not Matter

The composition \(P(s,u)P(u,t)\) equals \(P(s,t)\) for every allowed split time \(u\). Differentiating that product with respect to \(u\) makes the connection between the two Kolmogorov equations explicit. The first factor changes at its terminal time, while the second changes at its starting time:

\begin{aligned} \frac{\partial}{\partial u}\bigl(P(s,u)P(u,t)\bigr) &= \bigl(P(s,u)Q(u)\bigr)P(u,t) + P(s,u)\bigl(-Q(u)P(u,t)\bigr)\\ &=0. \end{aligned}

The terms cancel because matrix multiplication is associative, so both terms contain the same ordered product \(P(s,u)Q(u)P(u,t)\), with opposite signs. This is a useful consistency check: shifting the split point cannot change the overall transition probabilities.

Worked Example: Checking Split-Time Consistency

Consider a hypothetical two-state model with intensities at an intermediate time \(u\) of \(0.50\) per year from state 1 to state 2 and \(0.25\) per year from state 2 to state 1. Suppose the two interval transition matrices at the split are

P(s,u)= \begin{pmatrix} 0.80&0.20\\ 0.30&0.70 \end{pmatrix}, \qquad P(u,t)= \begin{pmatrix} 0.60&0.40\\ 0.10&0.90 \end{pmatrix}, \qquad Q(u)= \begin{pmatrix} -0.50&0.50\\ 0.25&-0.25 \end{pmatrix}.

First calculate the composed transition matrix:

P(s,u)P(u,t) = \begin{pmatrix} 0.80(0.60)+0.20(0.10)&0.80(0.40)+0.20(0.90)\\ 0.30(0.60)+0.70(0.10)&0.30(0.40)+0.70(0.90) \end{pmatrix} = \begin{pmatrix} 0.50&0.50\\ 0.25&0.75 \end{pmatrix}.

Next calculate the contribution from changing the first factor. The forward equation gives

\begin{aligned} P(s,u)Q(u) &= \begin{pmatrix} -0.35&0.35\\ 0.025&-0.025 \end{pmatrix},\\ \bigl(P(s,u)Q(u)\bigr)P(u,t) &= \begin{pmatrix} -0.175&0.175\\ 0.0125&-0.0125 \end{pmatrix}. \end{aligned}

The contribution from changing the second factor is, by the backward equation,

\begin{aligned} Q(u)P(u,t) &= \begin{pmatrix} -0.25&0.25\\ 0.125&-0.125 \end{pmatrix},\\ P(s,u)\bigl(-Q(u)P(u,t)\bigr) &= \begin{pmatrix} 0.175&-0.175\\ -0.0125&0.0125 \end{pmatrix}. \end{aligned}

Adding these two derivative contributions gives the zero matrix, entry by entry. The composed matrix is stochastic: both rows sum to 1, and its entries are nonnegative. Common mistake avoided: the two derivatives must use the same \(Q(u)\) at the split time, with the forward contribution on the first factor and the negative backward contribution on the second.

How to Use the Equations Reliably

For a time-inhomogeneous model, \(Q(t)\) can vary with calendar time, so the generator used in a derivative is evaluated at the time being varied. The equations do not say that a single fixed generator applies across the entire interval. Their boundary condition is \(P(s,s)=I\): over a zero-length interval, the state remains its starting state with probability 1.

A practical check is to examine row sums. Since every row of \(Q(t)\) sums to zero, either differential equation implies that each derivative row sums to zero. This is necessary for the rows of \(P(s,t)\) to remain probability distributions. It does not, by itself, verify every entry or establish that a proposed generator has nonnegative off-diagonal intensities.

These relationships matter whenever a transition probability is assembled across multiple periods, including hypothetical disability or care-state models. Composition lets an analyst insert a convenient intermediate time; the differential equations describe how transition probabilities respond when an endpoint changes. The most common errors are using the wrong generator time, reversing matrix order, or forgetting the minus sign in the backward equation.

Key takeaway: With row-to-column transition probabilities, the forward equation is \(\partial P(s,t)/\partial t=P(s,t)Q(t)\), while the backward equation is \(\partial P(s,t)/\partial s=-Q(s)P(s,t)\). Their opposite signs and matrix positions ensure Chapman–Kolmogorov composition is independent of its intermediate time.

Check Your Understanding

Use row-to-column probabilities and the time argument shown in each question. Rates are per year; report derivative matrices as given.

At a terminal time, \(P(s,t)=\begin{pmatrix}0.80&0.20\\0.40&0.60\end{pmatrix}\) and \(Q(t)=\begin{pmatrix}-0.30&0.30\\0.20&-0.20\end{pmatrix}\). Find the forward derivative. (Answer: \(\begin{pmatrix}-0.20&0.20\\0&0\end{pmatrix}\))
At a starting time, \(P(s,t)=\begin{pmatrix}0.75&0.25\\0.20&0.80\end{pmatrix}\) and \(Q(s)=\begin{pmatrix}-0.40&0.40\\0.10&-0.10\end{pmatrix}\). Find the backward derivative. (Answer: \(\begin{pmatrix}0.22&-0.22\\-0.055&0.055\end{pmatrix}\))
Two interval matrices are \(A=\begin{pmatrix}0.90&0.10\\0.20&0.80\end{pmatrix}\) and \(B=\begin{pmatrix}0.70&0.30\\0.40&0.60\end{pmatrix}\). Calculate the composed transition matrix \(AB\). (Answer: \(\begin{pmatrix}0.67&0.33\\0.46&0.54\end{pmatrix}\))
Write the derivative with respect to the split time \(u\) of \(P(s,u)P(u,t)\), and state why it is zero. (Answer: \(P(s,u)Q(u)P(u,t)-P(s,u)Q(u)P(u,t)=0\); the two terms cancel.)
For a valid generator, what must each row sum to, and what does this imply about each row sum of either transition-matrix derivative? (Answer: Each generator row sums to 0, so each derivative row sums to 0.)

What You'll Learn