Four Checks for a Transient Probability Matrix
For a continuous-time Markov chain, a computed matrix can look plausible while still representing the wrong probabilities. A sign error may produce negative transition probabilities; a transpose may give columns that sum to one when the convention requires rows; and a calculation can satisfy both checks at a particular time while using the wrong transition rates. A reliable diagnosis checks the probability constraints, the value at time zero, and the behavior immediately after time zero.
Let \((X_t)_{t\ge0}\) be a continuous-time Markov chain on a finite state space \(S=\{1,\ldots,m\}\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Its transition matrix at time \(t\) is \(P(t)=(p_{ij}(t))\), where \(p_{ij}(t)=\mathbb{P}(X_t=j\mid X_0=i)\). We use row vectors for distributions, so \(\mu(t)=\mu(0)P(t)\). The generator is \(Q=(q_{ij})\), with \(q_{ij}\ge0\) for \(i\ne j\) and each row summing to zero. In the finite-state setting, the transition matrix is \(P(t)=e^{tQ}\).
These are necessary checks, not interchangeable ones. Row sums test total probability, nonnegativity tests whether entries can be probabilities, and \(P(0)=I\) tests whether the chain starts in its initial state. The derivative test checks whether the proposed matrix uses the intended rates. A check at just one positive time cannot replace checks over the full time range.
Why the Generator Gives These Checks
Proof. First assume \(Q\) is a generator. Write \(q_i=-q_{ii}\ge0\), and set \(c=\max_i q_i\). The matrix \(B=Q+cI\) has nonnegative entries: its off-diagonal entries are those of \(Q\), and its \(i\)th diagonal entry is \(c-q_i\ge0\). Since \(Q=B-cI\) and \(B\) commutes with \(I\), the matrix exponential power series gives \(e^{tQ}=e^{-ct}e^{tB}=e^{-ct}\sum_{k=0}^{\infty}t^kB^k/k!\). Every \(B^k\) has nonnegative entries, so \(e^{tQ}\) does too. If \(c=0\), then \(Q=0\) and the same conclusion holds directly.
Let \(\mathbf{1}\) be the all-ones column vector. The zero row sums of \(Q\) give \(Q\mathbf{1}=0\), hence \(Q^k\mathbf{1}=0\) for every integer \(k\ge1\). Therefore the exponential series yields \(e^{tQ}\mathbf{1}=\mathbf{1}\), which says that every row sums to one. The series also gives \(e^{0Q}=I\) and derivative \(Q\) at zero.
Conversely, for a chain, \(P(0)=I\) because at elapsed time zero the state has not changed. For \(i\ne j\), \(p_{ij}(0)=0\) and \(p_{ij}(h)\ge0\) for \(h>0\), so if the right derivative exists, \(q_{ij}=\lim_{h\downarrow0}p_{ij}(h)/h\ge0\). Since \(\sum_jp_{ij}(h)=1\) for every \(h\), differentiating at zero gives \(\sum_jq_{ij}=0\). By definition, \(P'(0)=Q\). \(\square\)
For a complete check of a proposed formula, it is useful to test the forward Kolmogorov equation as well. With the row-stochastic convention, that equation is \(P'(t)=P(t)Q\). If a differentiable matrix \(C(t)\) satisfies \(C(0)=I\) and \(C'(t)=C(t)Q\) throughout an interval, it must equal \(e^{tQ}\) there.
Proof. Define \(D(t)=C(t)e^{-tQ}\). The product rule and the derivative of the matrix exponential give \(D'(t)=C'(t)e^{-tQ}-C(t)Qe^{-tQ}=C(t)Qe^{-tQ}-C(t)Qe^{-tQ}=0\). Thus \(D(t)\) is constant on the interval. Since \(D(0)=I\), we have \(C(t)e^{-tQ}=I\). Multiplying on the right by \(e^{tQ}\) gives \(C(t)=e^{tQ}\). \(\square\)
This result turns the checks into a diagnosis rather than a visual inspection: if the initial value and the forward equation both hold, the candidate is the intended matrix exponential. Checking nonnegativity and row sums remains important, particularly when a formula has been computed or transcribed incorrectly.
Worked Example: Checking a Two-State Transition Matrix
On \(S=\{1,2\}\), consider the generator \(Q=\begin{bmatrix}-2&2\\1&-1\end{bmatrix}\). Its rows sum to \(-2+2=0\) and \(1-1=0\), and its off-diagonal rates are nonnegative. Find and check \(P(t)=e^{tQ}\).
The stationary row vector is \((\tfrac13,\tfrac23)\), since the rate into state 1 balances when \(2\pi_1=\pi_2\), together with \(\pi_1+\pi_2=1\). The other eigenvalue of \(Q\) is \(-3\), so the decaying term is \(r=e^{-3t}\). The resulting matrix is
For \(t\ge0\), \(0<r\le1\). Every entry is nonnegative. The first row sums to \(\tfrac13+\tfrac23r+\tfrac23-\tfrac23r=1\), and the second sums to \(\tfrac13-\tfrac13r+\tfrac23+\tfrac13r=1\). At \(t=0\), \(r=1\), giving \(P(0)=I\).
Differentiate at zero. Since \(r'(t)=-3e^{-3t}\), we have \(r'(0)=-3\). The four entry derivatives at zero are respectively \(\tfrac23(-3)=-2\), \(-\tfrac23(-3)=2\), \(-\tfrac13(-3)=1\), and \(\tfrac13(-3)=-1\). Thus \(P'(0)=Q\), as required.
Check by the forward equation: the formula is also \(P(t)=\Pi+r(I-\Pi)\), where each row of \(\Pi\) is \((\tfrac13,\tfrac23)\). Direct multiplication gives \(P(t)Q=-3r(I-\Pi)\), while differentiation gives \(P'(t)=-3r(I-\Pi)\). Hence the forward equation holds, and the verification result confirms the formula.
Diagnosing Sign and Transpose Errors
A sign error often preserves the initial matrix and can even preserve row sums. The short-time derivative then reveals that the rates have the wrong signs. A transpose error has a different signature: the calculation may preserve column sums, but the rows do not represent probability distributions under the convention used here.
Worked Example: A Sign Error That Passes Two Checks
Keep the generator from the previous example, but suppose an incorrect sign in the exponential produces the candidate \(C(t)\) below, with \(s=e^{3t}\):
At \(t=0\), \(s=1\), so \(C(0)=I\). Its first row sums to \(\tfrac13+\tfrac23s+\tfrac23-\tfrac23s=1\); its second sums to \(\tfrac13-\tfrac13s+\tfrac23+\tfrac13s=1\). But for every \(t>0\), \(s>1\), so \(C_{12}(t)=\tfrac23(1-s)<0\) and \(C_{21}(t)=\tfrac13(1-s)<0\). It is not a probability matrix.
The derivative check identifies the same error at the initial time. Since \(s'(0)=3\), differentiating the four entries gives \(2,-2,-1,1\), respectively. Therefore \(C'(0)=\begin{bmatrix}2&-2\\-1&1\end{bmatrix}=-Q\), not \(Q\). The initial-value and row-sum checks alone would have missed this sign mistake.
Worked Example: A Transpose That Gives the Wrong Row Sums
Suppose the intended generator is again \(Q=\begin{bmatrix}-2&2\\1&-1\end{bmatrix}\), but a calculation returns \(C(t)=e^{tQ^{T}}=P(t)^T\). Using \(r=e^{-3t}\), its entries are
All entries are nonnegative for \(t\ge0\), and \(C(0)=I\). But the first row sums to \(\tfrac13+\tfrac23r+\tfrac13-\tfrac13r=\tfrac23+\tfrac13r\), and the second sums to \(\tfrac23-\tfrac23r+\tfrac23+\tfrac13r=\tfrac43-\tfrac13r\). These equal one at \(t=0\), but for \(t>0\), \(r<1\), so the first is less than one and the second is greater than one. Thus \(C(t)\) is not row-stochastic.
The derivative test gives \(C'(0)=Q^T=\begin{bmatrix}-2&1\\2&-1\end{bmatrix}\). Its row sums are \(-1\) and \(1\), whereas a generator must have zero row sums. The transpose has column sums zero, which explains why transposing can appear consistent when the opposite matrix convention is being used. Under the row-vector convention, the required forward equation is \(P'(t)=P(t)Q\), not \(P'(t)=Q P(t)\).
Why the Short-Time Derivative Matters
Passing the initial-value, nonnegativity, and row-sum checks does not establish that a candidate uses the intended rates. For example, a matrix exponential built from a different valid generator can pass all three. Comparing the derivative at zero with the specified \(Q\) detects that mismatch. For a calculation intended to give the entire transient matrix, the strongest direct check is to verify the initial condition and the forward equation over time.
Worked Example: A Valid Matrix for the Wrong Rates
Use the same two-state chain and let \(\widetilde C(t)=e^{2tQ}\), which incorrectly doubles every rate if the intended generator remains \(Q\). The formula from the first example, with \(r=e^{-6t}\), is
For \(t\ge0\), \(0<r\le1\), so all four entries are nonnegative. The first row sums to \(\tfrac13+\tfrac23r+\tfrac23-\tfrac23r=1\); the second sums to \(\tfrac13-\tfrac13r+\tfrac23+\tfrac13r=1\). At zero, \(r=1\) and \(\widetilde C(0)=I\). It passes these three checks.
Here \(r'(0)=-6\). Differentiating the entries at zero gives \(-4,4,2,-2\), so \(\widetilde C'(0)=\begin{bmatrix}-4&4\\2&-2\end{bmatrix}=2Q\ne Q\). The matrix is a valid transition matrix for a chain with generator \(2Q\), but it is not the answer for the specified generator \(Q\). This distinction is why the short-time derivative must be compared with the intended rates, not merely checked for valid generator signs.
A Practical Diagnostic Order
When a formula or computed matrix \(C(t)\) is in doubt, check it in this order:
- Check the generator first. Verify \(q_{ij}\ge0\) for \(i\ne j\) and \(\sum_jq_{ij}=0\) for every row.
- Check the initial value. Substitute \(t=0\) and confirm \(C(0)=I\), including every diagonal and off-diagonal entry.
- Check probability constraints. For each tested \(t\ge0\), verify every entry is nonnegative and each row sums to one. A negative entry or a row sum different from one rules the candidate out.
- Check the short-time rates. Differentiate at zero and compare every entry of \(C'(0)\) with \(Q\). In particular, off-diagonal derivatives must match the transition rates.
- Check the equation for the full formula. Verify \(C'(t)=C(t)Q\) and \(C(0)=I\). For a differentiable candidate and fixed finite generator, these conditions identify \(e^{tQ}\).
Common mistake. With row-vector distributions, it is wrong to accept a matrix because its columns sum to one. For the transpose example, the columns do sum to one: the first column is \(\tfrac13+\tfrac23r+\tfrac23-\tfrac23r=1\), and the second is \(\tfrac13-\tfrac13r+\tfrac23+\tfrac13r=1\). But its row sums are \(\tfrac23+\tfrac13r\) and \(\tfrac43-\tfrac13r\), not both one for \(t>0\). The correction is to check row sums, consistent with \(\mu(t)=\mu(0)P(t)\).
For numerical matrix exponentials, tiny deviations from zero or one can arise from rounding. In that setting, compare row sums, entries, and derivatives against a stated numerical tolerance; do not treat a substantial negative entry or a systematic row-sum error as harmless rounding. Exact calculations, when available, make the four structural checks especially transparent.
Check Your Understanding
For each question, use the row-stochastic convention and state which check detects the issue.
- For a proposed \(C(t)\), every row sums to one and \(C(0)=I\), but one off-diagonal entry is negative at a positive time. Can \(C(t)\) be a transition matrix?
- Why must the off-diagonal entries of \(C'(0)\) be nonnegative for a finite-state continuous-time Markov chain?
- A proposed matrix has columns summing to one, but its rows do not. What convention error is a natural possibility?
- If \(C(0)=I\) and \(C'(t)=C(t)Q\) for every \(t\) in an interval, which matrix function must \(C(t)\) equal?
- How can a candidate pass the initial-value, row-sum, and nonnegativity checks but still be wrong for a specified generator?