Diagnosing Sign, Index, and Time-Argument Errors

Why Small Notation Errors Matter

Kolmogorov equations describe how transition probabilities change as the start or end of an observation interval moves. Their calculations are compact, but a reversed pair of state indices, a misplaced minus sign, or a matrix product in the wrong order can produce an equation that looks plausible and is still wrong. A useful audit begins by identifying what each index and time argument means.

Throughout, use a finite-state continuous-time Markov model and the row-vector convention: rows of a transition matrix represent the state at the start of an interval, and columns represent the state at its end. The forward and backward equations below use this same convention.

Definition: Let \(p_{ij}(s,t)=P(X(t)=j\mid X(s)=i)\), for \(s\leq t\). The transition matrix \(P(s,t)\) has entry \(p_{ij}(s,t)\) in row \(i\), column \(j\). The intensity \(q_{ij}(t)\), for \(i\ne j\), is the instantaneous rate of transition from state \(i\) to state \(j\) at time \(t\). The generator \(Q(t)\) has off-diagonal entries \(q_{ij}(t)\) and diagonal entries \(q_{ii}(t)=-\sum_{j\ne i}q_{ij}(t)\).

Thus, the first index is the state being left and the second is the state being entered. Each off-diagonal generator entry is nonnegative, each diagonal entry is nonpositive, and every row of \(Q(t)\) sums to zero. In contrast, entries of \(P(s,t)\) are probabilities: they are nonnegative and each row sums to one.

Which Equation Uses Which Time

The forward equation differentiates with respect to the end of the interval. The backward equation differentiates with respect to its start. With the row convention, their matrix forms are:

\frac{\partial}{\partial t}P(s,t)=P(s,t)Q(t), \qquad \frac{\partial}{\partial s}P(s,t)=-Q(s)P(s,t).

In the forward equation, the current transition matrix is on the left and the generator at the terminal time is on the right. In the backward equation, the generator at the initial time is on the left, with a minus sign. The entrywise versions make the index pattern explicit:

\frac{\partial}{\partial t}p_{ij}(s,t) =\sum_k p_{ik}(s,t)q_{kj}(t), \qquad \frac{\partial}{\partial s}p_{ij}(s,t) =-\sum_k q_{ik}(s)p_{kj}(s,t).

These are not interchangeable rules. In particular, \(Q(t)P(s,t)\) is generally not the forward product, and \(P(s,t)Q(s)\) is generally not the backward product. If the intensities vary with time, evaluating the generator at the wrong endpoint is another distinct error. Even when the time argument is correct, swapping matrix order can change the result.

Derivation: The Chapman–Kolmogorov relation gives \(P(s,t+h)=P(s,t)P(t,t+h)\). Over a short interval, \(P(t,t+h)=I+hQ(t)+o(h)\). Subtract \(P(s,t)\), divide by \(h\), and let \(h\) tend to zero to obtain \(\partial P(s,t)/\partial t=P(s,t)Q(t)\). For the initial endpoint, \(P(s-h,t)=P(s-h,s)P(s,t)\), where \(P(s-h,s)=I+hQ(s)+o(h)\). Therefore \(P(s-h,t)=P(s,t)+hQ(s)P(s,t)+o(h)\). Rearranging the difference quotient with respect to \(s\) gives \(\partial P(s,t)/\partial s=-Q(s)P(s,t)\). The minus sign arises because moving the start time later shortens the interval.

Worked Matrix Audits

Worked Example: Checking a Forward Equation and Its Indices

Method. Multiply the supplied transition matrix by the generator on the right, then check row sums and interpret the signs. At a particular terminal time, suppose

P(s,t)= \begin{pmatrix} 0.6&0.3&0.1\\ 0.2&0.5&0.3\\ 0&0&1 \end{pmatrix}, \qquad Q(t)= \begin{pmatrix} -2&2&0\\ 0&-1&1\\ 0&0&0 \end{pmatrix}.

The generator describes transitions \(1\to2\) at rate 2 and \(2\to3\) at rate 1, with state 3 absorbing. Each generator row sums to zero. The forward equation requires \(P(s,t)Q(t)\):

\begin{aligned} \frac{\partial P(s,t)}{\partial t} &= \begin{pmatrix} 0.6&0.3&0.1\\ 0.2&0.5&0.3\\ 0&0&1 \end{pmatrix} \begin{pmatrix} -2&2&0\\ 0&-1&1\\ 0&0&0 \end{pmatrix}\\ &= \begin{pmatrix} -1.2&1.2-0.3&0.3\\ -0.4&0.4-0.5&0.5\\ 0&0&0 \end{pmatrix} = \begin{pmatrix} -1.2&0.9&0.3\\ -0.4&-0.1&0.5\\ 0&0&0 \end{pmatrix}. \end{aligned}

For example, the derivative in row 1, column 3 is \(0.3\): probability can flow into state 3 from state 2, and the current probability of being in state 2 is \(0.3\). Each derivative row sums to zero, as it must because the corresponding probability row continues to sum to one.

Common mistake and correction. Reversing the product gives \(Q(t)P(s,t)\), which in this case equals

Q(t)P(s,t)= \begin{pmatrix} -0.8&0.4&0.4\\ -0.2&-0.5&0.7\\ 0&0&0 \end{pmatrix}.

Although these rows also sum to zero, this is not the forward equation under the stated convention. A row-sum check can reveal some mistakes, but it cannot by itself establish that the matrix order is correct.

Worked Example: Separating Endpoint Times and the Backward Sign

Method. For the forward derivative use \(Q(t)\) on the right; for the backward derivative use \(-Q(s)\) on the left. Suppose an interval calculation supplies

P(s,t)= \begin{pmatrix} 0.8&0.2\\ 0.3&0.7 \end{pmatrix}, \quad Q(s)= \begin{pmatrix} -1&1\\ 2&-2 \end{pmatrix}, \quad Q(t)= \begin{pmatrix} -3&3\\ 4&-4 \end{pmatrix}.

First calculate the forward derivative, using the generator at the terminal time:

\begin{aligned} P(s,t)Q(t) &= \begin{pmatrix} 0.8&0.2\\ 0.3&0.7 \end{pmatrix} \begin{pmatrix} -3&3\\ 4&-4 \end{pmatrix}\\ &= \begin{pmatrix} -2.4+0.8&2.4-0.8\\ -0.9+2.8&0.9-2.8 \end{pmatrix} = \begin{pmatrix} -1.6&1.6\\ 1.9&-1.9 \end{pmatrix}. \end{aligned}

Next calculate the backward derivative, using the generator at the initial time and applying the minus sign:

\begin{aligned} -Q(s)P(s,t) &=- \begin{pmatrix} -1&1\\ 2&-2 \end{pmatrix} \begin{pmatrix} 0.8&0.2\\ 0.3&0.7 \end{pmatrix}\\ &=- \begin{pmatrix} -0.8+0.3&-0.2+0.7\\ 1.6-0.6&0.4-1.4 \end{pmatrix} = \begin{pmatrix} 0.5&-0.5\\ -1&1 \end{pmatrix}. \end{aligned}

Each derivative row sums to zero. The backward derivative can have negative off-diagonal entries: it describes change as the start time moves, not a probability distribution. The differing endpoint generators also show why replacing \(Q(t)\) by \(Q(s)\) in the forward equation, or vice versa, is not justified for time-varying intensities.

Common mistake and correction. Omitting the backward minus sign would give \(Q(s)P(s,t)\), the negative of the correct matrix. Moving the start time later removes some possible transition time, so the backward equation carries a minus sign.

Worked Example: Ordering Two Piecewise-Constant Transition Matrices

Method. Find each interval transition matrix, then multiply them from the first interval to the second. Consider three states, \(0,1,2\). During \(0\leq u<1\), the only transition is \(0\to1\) at rate \(\ln 2\). During \(1\leq u\leq2\), the only transition is \(1\to2\) at rate \(\ln 2\). State 2 is absorbing throughout.

For a one-unit interval with a single transition rate \(\ln 2\), the probability of no transition is \(e^{-\ln 2}=1/2\), and the probability of transition is \(1-1/2=1/2\). Thus the first interval matrix and second interval matrix are:

P(0,1)= \begin{pmatrix} 1/2&1/2&0\\ 0&1&0\\ 0&0&1 \end{pmatrix}, \qquad P(1,2)= \begin{pmatrix} 1&0&0\\ 0&1/2&1/2\\ 0&0&1 \end{pmatrix}.

By the Chapman–Kolmogorov relation, the chronological order is:

\begin{aligned} P(0,2) &=P(0,1)P(1,2)\\ &= \begin{pmatrix} 1/2&1/2&0\\ 0&1&0\\ 0&0&1 \end{pmatrix} \begin{pmatrix} 1&0&0\\ 0&1/2&1/2\\ 0&0&1 \end{pmatrix}\\ &= \begin{pmatrix} 1/2&1/4&1/4\\ 0&1/2&1/2\\ 0&0&1 \end{pmatrix}. \end{aligned}

Starting in state 0, the probability of ending in state 2 is \(1/4\): the process must first move to state 1 in the first interval, with probability \(1/2\), then move to state 2 in the second interval, also with probability \(1/2\). Their product is \(1/2\times1/2=1/4\). Each resulting row sums to one.

Common mistake and correction. The reversed product \(P(1,2)P(0,1)\) gives first row \((1/2,1/2,0)\), incorrectly assigning zero probability to ending in state 2 from state 0. Matrix factors must act in chronological order for row transition matrices: the earliest interval comes first on the left.

A Practical Error-Checking Sequence

When an equation or computed matrix is in doubt, check its ingredients in a fixed order. This keeps distinct errors from being hidden by a final numerical answer that happens to look reasonable.

Read the indices.
In \(p_{ij}(s,t)\) and \(q_{ij}(u)\), identify \(i\) as the origin and \(j\) as the destination. Check that the verbal transition agrees with the row and column.

Validate the generator.
Check that off-diagonal entries are nonnegative, diagonal entries are negative or zero, and every row sums to zero.

Mark the endpoint being varied.
For the forward equation, use \(Q(t)\) and \(P(s,t)Q(t)\). For the backward equation, use \(-Q(s)P(s,t)\).

Preserve the product order.
Use the equation’s prescribed order, and for successive time intervals place the earliest transition matrix first.

Check conservation and interpretation.
Derivative rows should sum to zero; transition-matrix rows should sum to one. Then check whether the signs and flows make sense for the stated states and time change.

A common exam trap is to judge a derivative matrix using the same sign expectations as a transition matrix. A transition matrix must have nonnegative entries, but a derivative matrix need not: probability can decrease in one state as it increases in another. The conservation check is that each derivative row sums to zero, not that each derivative entry is nonnegative.

Key takeaway: Under the row convention, the forward equation is \(P(s,t)Q(t)\), while the backward equation is \(-Q(s)P(s,t)\). State indices run from origin to destination, and interval matrices are multiplied in chronological order.

Check Your Understanding

For each problem, identify the relevant convention before calculating.

A three-state model has rates \(1\to2\) of \(0.4\), \(1\to3\) of \(0.1\), \(2\to1\) of \(0.2\), and \(2\to3\) of \(0.3\); state 3 is absorbing. Write its generator in state order \(1,2,3\), and check its row sums. (Answer: \(Q=\begin{pmatrix}-0.5&0.4&0.1\\0.2&-0.5&0.3\\0&0&0\end{pmatrix}\), with row sums \(0,0,0\).)
At terminal time \(t\), let \(P(s,t)=\begin{pmatrix}0.8&0.2\\0.3&0.7\end{pmatrix}\) and \(Q(t)=\begin{pmatrix}-2&2\\1&-1\end{pmatrix}\). Calculate the forward derivative and verify its row sums. (Answer: \(\begin{pmatrix}-1.4&1.4\\0.1&-0.1\end{pmatrix}\); both row sums are zero.)
At initial time \(s\), let \(Q(s)=\begin{pmatrix}-1&1\\3&-3\end{pmatrix}\) and \(P(s,t)=\begin{pmatrix}0.6&0.4\\0.2&0.8\end{pmatrix}\). Calculate the backward derivative, including its sign. (Answer: \(\begin{pmatrix}0.4&-0.4\\-1.2&1.2\end{pmatrix}\); both row sums are zero.)
During the first unit interval, a process can move from state 0 to state 1 with probability \(1/2\), with state 1 otherwise remaining in state 1. During the next unit interval, state 1 moves to state 2 with probability \(1/2\), and state 2 is absorbing. Starting from state 0, give the correct two-interval transition matrix and the probability of ending in state 2. (Answer: \(P(0,2)=\begin{pmatrix}1/2&1/4&1/4\\0&1/2&1/2\\0&0&1\end{pmatrix}\); the probability is \(1/4\).)
A model has different generators at the start \(s\) and end \(t\) of an interval. State the correct generator and product order for each derivative, and identify the minus sign. (Answer: forward: \(P(s,t)Q(t)\), with the terminal-time generator on the right and no leading minus sign; backward: \(-Q(s)P(s,t)\), with the initial-time generator on the left and a leading minus sign.)

What You'll Learn