Matrix Conventions for Kolmogorov Equations

Why Matrix Convention Matters

A transition matrix can be written with probabilities in rows or in columns. Either arrangement is usable, but the generator and the order of matrix multiplication must match it. Mixing conventions can produce a plausible-looking equation that sends probability in the wrong direction or changes the meaning of a matrix entry.

This tutorial fixes the usual row-generator convention first, then translates it consistently to column form. The models considered are finite-state continuous-time Markov chains. Transition intensities are nonnegative, finite, and sufficiently regular for the displayed differential equations to hold. Intensities may depend on calendar time.

Definition: For a fixed ordering of the states, \(P_{ij}(s,t)\) is the probability that the process is in state \(j\) at time \(t\), given that it was in state \(i\) at time \(s\). The transition matrix \(P(s,t)\) has entry \(P_{ij}(s,t)\) in row \(i\), column \(j\). Thus each row represents a starting state, and each column represents an ending state.

Under this convention, an initial probability distribution is written as a row vector \(\boldsymbol{\pi}(s)\). Its \(i\)th entry is the probability of starting in state \(i\). The distribution at time \(t\) is then \(\boldsymbol{\pi}(s)P(s,t)\). The order is meaningful: the starting distribution is on the left of the transition matrix.

The Row Generator and Its Equations

The row generator \(Q(t)\) records the instantaneous transition intensities at time \(t\). For distinct states \(i\) and \(j\), the entry \(Q_{ij}(t)\) is the intensity of moving from \(i\) to \(j\). Its diagonal entry is chosen so that the entries in each row sum to zero.

Definition: The row generator satisfies \(Q_{ij}(t)\geq 0\) for \(i\ne j\) and \(Q_{ii}(t)=-\sum_{j\ne i}Q_{ij}(t)\). In small-time form, \(P_{ij}(t,t+h)=Q_{ij}(t)h+o(h)\) for \(i\ne j\), and \(P_{ii}(t,t+h)=1+Q_{ii}(t)h+o(h)\), as \(h\) decreases to zero.

The diagonal entry is not an intensity of a transition to a different state. It is the negative of the total intensity of leaving that state. The small-time expressions also show why the off-diagonal entries are nonnegative: they approximate probabilities divided by a positive interval length.

With this row convention, the forward equation differentiates with respect to the ending time \(t\), while the backward equation differentiates with respect to the starting time \(s\):

\frac{\partial}{\partial t}P(s,t)=P(s,t)Q(t), \qquad \frac{\partial}{\partial s}P(s,t)=-Q(s)P(s,t).

The generator is on the right in the forward equation and on the left in the backward equation. The time at which the generator is evaluated also differs: \(t\) for the forward equation and \(s\) for the backward equation. These are matrix equations, so the order of the factors cannot be exchanged unless there is a separate reason that the matrices commute.

Worked Example: Multiplying a Transition Matrix by a Generator

At an instant, suppose a two-state transition matrix and row generator are \(P=\begin{pmatrix}0.6&0.4\\0.25&0.75\end{pmatrix}\) and \(Q=\begin{pmatrix}-0.2&0.2\\0.1&-0.1\end{pmatrix}\). Calculate \(PQ\) and its transpose. Interpret the row-sum check.

Method. Multiply rows of \(P\) by columns of \(Q\). Under the row convention, this is the form used on the right side of a forward equation. The matrix \(P\) has nonnegative entries and row sums of one; \(Q\) has nonnegative off-diagonal entries and row sums of zero.

The first row of \(PQ\) is \((0.6(-0.2)+0.4(0.1),\;0.6(0.2)+0.4(-0.1))=(-0.08,\;0.08)\).

The second row is \((0.25(-0.2)+0.75(0.1),\;0.25(0.2)+0.75(-0.1))\). Its first entry is \(-0.05+0.075=0.025\), and its second entry is \(0.05-0.075=-0.025\). Therefore,

PQ= \begin{pmatrix} -0.08&0.08\\ 0.025&-0.025 \end{pmatrix}.

Transposing reverses the order of multiplication: \((PQ)^\mathsf{T}=Q^\mathsf{T}P^\mathsf{T}\). In this case,

(PQ)^\mathsf{T} = \begin{pmatrix} -0.08&0.025\\ 0.08&-0.025 \end{pmatrix} =Q^\mathsf{T}P^\mathsf{T}.

Each row of \(PQ\) sums to zero, as expected for an instantaneous change in a row of transition probabilities. Common mistake avoided: the second-row first entry is positive, not negative; the positive contribution \(0.75(0.1)\) exceeds the magnitude of \(0.25(-0.2)\).

Why the Matrix Order Appears

The forward equation follows from advancing the ending time by a short interval. For a Markov model, the transition from \(s\) to \(t+h\) can be split into a transition from \(s\) to \(t\), followed by one from \(t\) to \(t+h\). In matrix form, the first interval's matrix is on the left.

Split the interval.
Use \(P(s,t+h)=P(s,t)P(t,t+h)\), with the intermediate state at time \(t\) summed over by matrix multiplication.

Use the small-time expansion.
The row-generator convention gives \(P(t,t+h)=I+hQ(t)+o(h)\), where \(I\) is the identity matrix.

Subtract and divide by \(h\).
Then \(\{P(s,t+h)-P(s,t)\}/h=P(s,t)Q(t)+o(h)/h\). Taking the limit gives \(\partial P(s,t)/\partial t=P(s,t)Q(t)\).

The backward equation follows by splitting off a short interval at the starting end: \(P(s-h,t)=P(s-h,s)P(s,t)\). Since \(P(s-h,s)=I+hQ(s)+o(h)\), the left side is also \(P(s,t)-h\,\partial P(s,t)/\partial s+o(h)\). Equating the first-order terms gives \(-\partial P(s,t)/\partial s=Q(s)P(s,t)\). This explains both the negative sign and why \(Q(s)\) multiplies on the left.

Changing to a Column Convention

A probability distribution can instead be written as a column vector. Let \(\boldsymbol{p}(t)=\boldsymbol{\pi}(t)^\mathsf{T}\). Then its evolution is the transpose of the row-vector equation: \(\boldsymbol{p}(t)=P(s,t)^\mathsf{T}\boldsymbol{p}(s)\). Define \(C(s,t)=P(s,t)^\mathsf{T}\) and \(Q_{\mathrm{col}}(t)=Q(t)^\mathsf{T}\). The columns of \(C\) sum to one, and the columns of \(Q_{\mathrm{col}}\) sum to zero.

Taking transposes of the row equations reverses factor order. The resulting column-convention equations are:

\frac{\partial}{\partial t}C(s,t)=Q_{\mathrm{col}}(t)C(s,t), \qquad \frac{\partial}{\partial s}C(s,t)=-C(s,t)Q_{\mathrm{col}}(s), \qquad \frac{d}{dt}\boldsymbol{p}(t)=Q_{\mathrm{col}}(t)\boldsymbol{p}(t).

The state ordering is unchanged; only the arrangement of probabilities and the associated matrix convention have changed. A frequent error is to transpose a distribution but leave the original row generator acting on it. A column distribution requires \(Q^\mathsf{T}\), not \(Q\), unless the particular generator happens to be symmetric.

Worked Example: Evolving a Three-State Distribution

Consider the row generator \(Q=\begin{pmatrix}-0.4&0.3&0.1\\0.2&-0.5&0.3\\0.05&0.15&-0.2\end{pmatrix}\) and current row distribution \(\boldsymbol{\pi}=(0.4,0.4,0.2)\). Find the instantaneous derivative in row form and in column form.

Method. For a row distribution, calculate \(\boldsymbol{\pi}Q\). The generator rows sum to zero: \(-0.4+0.3+0.1=0\), \(0.2-0.5+0.3=0\), and \(0.05+0.15-0.2=0\).

The first component is \(0.4(-0.4)+0.4(0.2)+0.2(0.05)=-0.16+0.08+0.01=-0.07\). The second is \(0.4(0.3)+0.4(-0.5)+0.2(0.15)=0.12-0.20+0.03=-0.05\). The third is \(0.4(0.1)+0.4(0.3)+0.2(-0.2)=0.04+0.12-0.04=0.12\). Thus,

\frac{d\boldsymbol{\pi}}{dt}=\boldsymbol{\pi}Q=(-0.07,-0.05,0.12).

The components sum to \(-0.07-0.05+0.12=0\), so the instantaneous change preserves total probability. In column form, transpose the row result and use the transposed generator:

Q^\mathsf{T} \begin{pmatrix}0.4\\0.4\\0.2\end{pmatrix} = \begin{pmatrix}-0.07\\-0.05\\0.12\end{pmatrix}.

This is the same change expressed as a column. Common mistake avoided: the first component is \(-0.07\), not \(-0.08\), and the third is \(0.12\), not \(0.13\). Each component must be checked by multiplying the appropriate row of the distribution by the corresponding generator column.

Worked Example: Distinguishing Forward and Backward Products

At the relevant times, let \(P(s,t)=\begin{pmatrix}0.7&0.3\\0.2&0.8\end{pmatrix}\) and suppose the generator at each endpoint is \(Q=\begin{pmatrix}-0.4&0.4\\0.1&-0.1\end{pmatrix}\). Calculate the right side of the forward equation and the right side of the backward equation, using this same numerical generator at both endpoints. Then give the column-form forward derivative.

Forward calculation. Multiply \(P Q\). Its entries, row by row, are \((-0.28+0.03,\;0.28-0.03)=(-0.25,0.25)\) and \((-0.08+0.08,\;0.08-0.08)=(0,0)\). Hence the forward derivative is \(\begin{pmatrix}-0.25&0.25\\0&0\end{pmatrix}\).

Backward calculation. First, \(QP=\begin{pmatrix}-0.20&0.20\\0.05&-0.05\end{pmatrix}\). Therefore the backward derivative is \(-QP=\begin{pmatrix}0.20&-0.20\\-0.05&0.05\end{pmatrix}\). It is different from the forward derivative because multiplication order and differentiation endpoint differ.

For column form, transpose the forward derivative and reverse the product: \(Q^\mathsf{T}P^\mathsf{T}=(PQ)^\mathsf{T}\). Thus the column-form forward derivative is \(\begin{pmatrix}-0.25&0\\0.25&0\end{pmatrix}\). Both its columns sum to zero. Common mistake avoided: writing \(QP\) for the forward equation, or \(-PQ\) for the backward equation, would interchange the required order.

Checks and Common Exam Traps

The convention can be checked without solving any differential equation. With a row generator, every row of \(Q\) sums to zero, while every row of \(P(s,t)\) sums to one. Consequently, the row distribution \(\boldsymbol{\pi}P\) sums to one, and its derivative \(\boldsymbol{\pi}Q\) sums to zero. After transposing, the corresponding checks are column sums: columns of \(Q^\mathsf{T}\) sum to zero, and columns of \(P^\mathsf{T}\) sum to one.

Reading indices backward. \(P_{ij}(s,t)\) means start in \(i\), end in \(j\); it is not the probability of starting in \(j\) and ending in \(i\).
Using the wrong side. In row form, the forward equation is \(P Q\), while the backward equation is \(-Q P\).
Transposing without reversing order. \((AB)^\mathsf{T}=B^\mathsf{T}A^\mathsf{T}\), not \(A^\mathsf{T}B^\mathsf{T}\) in general.
Checking the wrong sums. Row sums are the natural check for the row convention; the transposed column convention uses column sums.

Key takeaway: Fix the state ordering, decide whether distributions are rows or columns, and then keep the transition matrix, generator, and multiplication order consistent. The column formulation is the transpose of the row formulation, including reversal of matrix-product order.

Check Your Understanding

For each problem, keep the stated convention fixed and check the relevant row or column sums.

Let \(P=\begin{pmatrix}0.8&0.2\\0.3&0.7\end{pmatrix}\) and \(Q=\begin{pmatrix}-0.5&0.5\\0.2&-0.2\end{pmatrix}\). Calculate \(PQ\). (Answer: \(\begin{pmatrix}-0.36&0.36\\-0.01&0.01\end{pmatrix}\).)
For \(Q=\begin{pmatrix}-0.4&0.3&0.1\\0.2&-0.5&0.3\\0.05&0.15&-0.2\end{pmatrix}\) and row distribution \((0.2,0.5,0.3)\), find the instantaneous row-distribution derivative. (Answer: \((0.035,-0.145,0.11)\).)
For the generator and distribution in Question 2, write the derivative in column form and identify the matrix acting on the column distribution. (Answer: \(Q^\mathsf{T}(0.2,0.5,0.3)^\mathsf{T}=(0.035,-0.145,0.11)^\mathsf{T}\); the acting matrix is \(Q^\mathsf{T}\).)
Using \(P=\begin{pmatrix}0.7&0.3\\0.2&0.8\end{pmatrix}\) and \(Q=\begin{pmatrix}-0.4&0.4\\0.1&-0.1\end{pmatrix}\), calculate the row-form forward product \(PQ\) and backward product \(-QP\). (Answer: \(PQ=\begin{pmatrix}-0.25&0.25\\0&0\end{pmatrix}\), and \(-QP=\begin{pmatrix}0.20&-0.20\\-0.05&0.05\end{pmatrix}\).)
If \(C=P^\mathsf{T}\) and \(Q_{\mathrm{col}}=Q^\mathsf{T}\), write the column-form forward equation and state the product-transpose identity that justifies its order. (Answer: \(\partial C/\partial t=Q_{\mathrm{col}}(t)C\), since \((P Q)^\mathsf{T}=Q^\mathsf{T}P^\mathsf{T}\).)

What You'll Learn