Transition Kernels and Chapman-Kolmogorov

From Conditional Laws to Transition Kernels

A transition kernel records the distribution of a process at a later time, conditional on its present state. The Chapman–Kolmogorov identity says that this law can be calculated either directly or by passing through an intermediate time. Its content is simple, but its hypotheses matter: a conditional distribution along one process determines a kernel only up to states that process visits with probability zero.

Work on a filtered probability space $(\Omega,\mathcal{F},(\mathcal{F}_t)_{t\geq0},\mathbb{P})$ carrying a Brownian motion $B_t$. Consider a diffusion $X_t$ with values in a standard Borel state space $E$, such as $\mathbb{R}$ or a finite set. When we refer to an Itô SDE, its drift is denoted by $\mu$ and its diffusion coefficient by $\sigma$, so its differential form is $dX_t=\mu(t,X_t)\,dt+\sigma(t,X_t)\,dB_t$. Each Itô integral $\int_0^t H_s\,dB_s$ is defined for a progressively measurable integrand $H$ with $\mathbb{E}\int_0^T H_s^2\,ds<\infty$ on the time interval in question; local versions are defined by stopping when this integrability holds. The integrals against time used below are ordinary integrals of probability measures, not Itô integrals.

Definition: A transition kernel from time $s$ to time $t$, where $s\leq t$, is a function $P_{s,t}(x,A)$ such that (i) for each $x\in E$, $A\mapsto P_{s,t}(x,A)$ is a probability measure on the Borel sets of $E$, and (ii) for each Borel set $A$, $x\mapsto P_{s,t}(x,A)$ is Borel measurable. For a Markov process, the kernel describes its conditional law: $\mathbb{P}(X_t\in A\mid\mathcal{F}_s)=P_{s,t}(X_s,A)$ almost surely.

Equivalently, for every bounded Borel function $f:E\to\mathbb{R}$, the Markov property gives $\mathbb{E}[f(X_t)\mid\mathcal{F}_s]=\int_E f(y)P_{s,t}(X_s,dy)$. If the process is time-homogeneous, its transition law depends only on the elapsed time: we write $P_h(x,A)$ for a transition over an interval of length $h$. A kernel need not have a density with respect to Lebesgue measure; for example, a deterministic process has a point-mass transition law.

The Chapman–Kolmogorov Identity

Theorem: Let $s\leq u\leq t$, and suppose the process has transition kernels satisfying the Markov property at these times. Then $$ P_{s,t}(x,A)=\int_E P_{u,t}(y,A)\,P_{s,u}(x,dy). $$ If the Markov property is specified for a family of processes started from every state $x$, the identity holds for every $x\in E$. For a single process with a fixed initial distribution, it is guaranteed at $X_s$ almost surely, and therefore for $x$ only for $P(X_s\in dx)$-almost every state.

Proof. Fix a Borel set $A$. The conditional expectation of $\mathbf{1}_{\{X_t\in A\}}$ given $\mathcal{F}_s$ can be computed by conditioning first at $u$. The tower property and the Markov property give $\mathbb{P}(X_t\in A\mid\mathcal{F}_s)=\mathbb{E}[P_{u,t}(X_u,A)\mid\mathcal{F}_s]$. The function $y\mapsto P_{u,t}(y,A)$ is bounded and Borel measurable. Applying the Markov property from $s$ to $u$, now to this function, yields $\mathbb{E}[P_{u,t}(X_u,A)\mid\mathcal{F}_s]=\int_E P_{u,t}(y,A)P_{s,u}(X_s,dy)$. The Markov property from $s$ to $t$ also identifies the first conditional probability as $P_{s,t}(X_s,A)$. Thus the two kernel expressions agree at $X_s$ almost surely. If the Markov property holds under a law $\mathbb{P}^x$ for every initial state $x$, the same argument under each $\mathbb{P}^x$, where $X_s=x$ at the initial time (or using the Markov family from time $s$), gives the identity for every such $x$. Without that all-starting-states assumption, the argument proves only equality at the random state $X_s$, hence only almost-everywhere equality in its distribution. $\square$

The right-hand side is a probability kernel: it first draws an intermediate state $y$ using $P_{s,u}(x,dy)$, then uses $P_{u,t}(y,A)$ for the remaining transition. The next verification records why this two-stage construction is itself a kernel.

Proposition: If $K(x,dy)$ and $L(y,A)$ are transition kernels on $E$, then $M(x,A)=\int_E L(y,A)K(x,dy)$ is a transition kernel.

Proof. For fixed $x$, the set function $A\mapsto M(x,A)$ is nonnegative, and $M(x,E)=\int_E 1\,K(x,dy)=1$. If $A_1,A_2,\ldots$ are pairwise disjoint Borel sets, countable additivity of $L(y,\cdot)$ gives $L(y,\bigcup_n A_n)=\sum_n L(y,A_n)$. By monotone convergence, $M(x,\bigcup_n A_n)=\sum_n\int_E L(y,A_n)K(x,dy)=\sum_n M(x,A_n)$. Thus $M(x,\cdot)$ is a probability measure. For fixed $A$, $y\mapsto L(y,A)$ is bounded and measurable. The kernel integration property implies that $x\mapsto\int_E L(y,A)K(x,dy)$ is measurable; this property follows first for indicator functions from the definition of a kernel, then for nonnegative simple functions by addition, and then for bounded nonnegative measurable functions by increasing simple approximation. Hence $M$ meets both requirements. $\square$

Worked Compositions

Worked Example: Composing Brownian Transition Densities

For Brownian motion started at $x$ at time $s$, the increment $B_t-B_s$ is normal with mean $0$ and variance $t-s$. Its transition density is $p_{s,t}(x,z)=\dfrac{1}{\sqrt{2\pi(t-s)}}\exp\big(-\dfrac{(z-x)^2}{2(t-s)}\big)$. Take $s<u<t$, and write $a=u-s>0$, $b=t-u>0$. Show that composing the density over the two intervals gives the direct density over length $a+b$.

The composed density at $z$ is $\int_{\mathbb{R}}p_{s,u}(x,y)p_{u,t}(y,z)\,dy$. To evaluate it, set $m=(bx+az)/(a+b)$. Expanding the squares gives the identity $\dfrac{(y-x)^2}{a}+\dfrac{(z-y)^2}{b}=\dfrac{a+b}{ab}(y-m)^2+\dfrac{(z-x)^2}{a+b}$. Indeed, the coefficient of $y^2$ on both sides is $1/a+1/b$, the coefficient of $y$ on both sides is $-2x/a-2z/b$, and the constant terms agree after substituting $m$.

\begin{aligned} \int_{\mathbb{R}}p_{s,u}(x,y)p_{u,t}(y,z)\,dy &=\frac{1}{2\pi\sqrt{ab}}\exp\left(-\frac{(z-x)^2}{2(a+b)}\right) \int_{\mathbb{R}}\exp\left(-\frac{a+b}{2ab}(y-m)^2\right)\,dy\\ &=\frac{1}{2\pi\sqrt{ab}}\exp\left(-\frac{(z-x)^2}{2(a+b)}\right) \sqrt{\frac{2\pi ab}{a+b}}\\ &=\frac{1}{\sqrt{2\pi(a+b)}}\exp\left(-\frac{(z-x)^2}{2(a+b)}\right) =p_{s,t}(x,z). \end{aligned}

The integral in the second line is the Gaussian integral: for $c>0$, $\int_{\mathbb{R}}e^{-c(y-m)^2/2}\,dy=\sqrt{2\pi/c}$, here with $c=(a+b)/(ab)$. Check: the composed density has variance parameter $a+b=(u-s)+(t-u)=t-s$, and its normalizing factor is $1/\sqrt{2\pi(t-s)}$, exactly the direct Brownian density.

Worked Example: A Finite-State Approximation

Let the state space be $E=\{0,1,2\}$. Suppose the transition matrix from the first time to the intermediate time is $A$, and from the intermediate time to the final time is $C$. Rows are starting states and columns are ending states. Compute the two-stage kernel from the matrices $$ A=\begin{pmatrix} \tfrac12&\tfrac12&0\\ 0&\tfrac12&\tfrac12\\ 0&0&1 \end{pmatrix}, \qquad C=\begin{pmatrix} 1&0&0\\ \tfrac14&\tfrac12&\tfrac14\\ 0&\tfrac13&\tfrac23 \end{pmatrix}. $$

For finite states the kernel integral is a sum over intermediate states, so the composed matrix is $AC$. Its first row is $\tfrac12(1,0,0)+\tfrac12(\tfrac14,\tfrac12,\tfrac14)=(\tfrac58,\tfrac14,\tfrac18)$. Its second row is $\tfrac12(\tfrac14,\tfrac12,\tfrac14)+\tfrac12(0,\tfrac13,\tfrac23)=(\tfrac18,\tfrac5{12},\tfrac{11}{24})$. Its third row is $0(1,0,0)+0(\tfrac14,\tfrac12,\tfrac14)+1(0,\tfrac13,\tfrac23)=(0,\tfrac13,\tfrac23)$. Therefore $$ AC=\begin{pmatrix} \tfrac58&\tfrac14&\tfrac18\\ \tfrac18&\tfrac5{12}&\tfrac{11}{24}\\ 0&\tfrac13&\tfrac23 \end{pmatrix}. $$

Check: each entry is nonnegative. The first row sums to $\tfrac58+\tfrac14+\tfrac18=1$; the second sums to $\tfrac18+\tfrac5{12}+\tfrac{11}{24}=\tfrac3{24}+\tfrac{10}{24}+\tfrac{11}{24}=1$; and the third sums to $0+\tfrac13+\tfrac23=1$. Thus the composition still has probability rows, as the proposition requires.

Worked Example: A Time-Dependent Drift

Let $X_t=B_t+t^2$, with $B_0=0$. This is an Itô diffusion with drift $\mu(t,x)=2t$ and diffusion coefficient $\sigma(t,x)=1$; the stochastic integral in its SDE is $\int_0^t 1\,dB_s=B_t$, whose integrand is progressively measurable and satisfies $\mathbb{E}\int_0^T1^2\,ds=T<\infty$. Both coefficients are Lipschitz in $x$ and satisfy linear growth on each bounded time interval. Given $X_s=x$, the increment $X_t-X_s$ is $B_t-B_s+t^2-s^2$, so the transition law is normal with mean $x+t^2-s^2$ and variance $t-s$.

Take $s=1,u=2,t=4$. From time $1$ to $2$, the mean shift is $2^2-1^2=3$ and the variance is $2-1=1$. From time $2$ to $4$, the mean shift is $4^2-2^2=12$ and the variance is $4-2=2$. Composing the normal transitions adds the shifts and variances: the resulting mean is $x+3+12=x+15$, and the variance is $1+2=3$. Directly from time $1$ to $4$, the shift is $4^2-1^2=15$ and the variance is $4-1=3$, the same parameters.

Check: the total drift shift is $t^2-s^2=(u^2-s^2)+(t^2-u^2)=3+12=15$, and the total variance is $t-s=(u-s)+(t-u)=1+2=3$. Thus the intermediate time changes neither the final transition law nor its normalization.

How to Use the Identity—and a Common Pitfall

Chapman–Kolmogorov is useful when a transition is easier to compute over short intervals than over a long one. It is also a consistency test: proposed transition laws must compose to the stated law over the combined interval. For an approximate finite-state model, matrix multiplication performs precisely this calculation, summing over every possible intermediate state.

A frequent error is to interpret the identity as an equality for every state just because it holds along one observed process. The conditional Markov property gives equality evaluated at $X_s$, almost surely. If the distribution of $X_s$ assigns probability zero to a state $x_0$, that conditional statement places no restriction on the kernel values at $x_0$. To assert the identity for all states, use a Markov family whose kernels are defined consistently for every initial state and apply the Markov property for each such start.

Another error is to multiply transition densities without integrating over the intermediate state. The composition of densities is an integral over that state; in a finite state space, it is a sum. The Brownian calculation and the matrix example show the two versions of the same rule. Nor should one confuse a transition kernel with a density: a density is only a representation relative to a chosen reference measure, when one exists. The kernel formulation covers continuous, discrete, and singular transition laws uniformly.

Check Your Understanding

For each question, state the assumptions needed for your conclusion, especially when an equality is only almost sure.

Use conditional expectation at an intermediate time $u$ to derive Chapman–Kolmogorov for a bounded Borel test function.
Explain why a conditional identity along one process need not determine the transition kernel at every state.
For Brownian motion, compose intervals of lengths $2$ and $5$. State the resulting variance parameter and normalization factor.
In a finite-state model, what does summing over intermediate states correspond to in matrix notation?
Can a deterministic process have a transition kernel even if its transition law has no Lebesgue density? Give its kernel explicitly for a deterministic flow $\phi_{s,t}(x)$.

Transition Kernels and Chapman-Kolmogorov

What You'll Learn