Deriving the Matrix Exponential Solution

Why Derive the Matrix Exponential Solution?

For a finite-state, time-homogeneous Markov model, the forward Kolmogorov equation describes how the transition probability matrix changes over time. The matrix exponential gives a compact solution, but using it safely requires knowing which equation it solves, how the initial condition enters, and when uniqueness is guaranteed. These details matter: a matrix can satisfy the initial condition, or even have plausible probabilities at one time, without tracing the correct evolution at all times.

Use the row convention: $P(t)=(p_{ij}(t))$, where $p_{ij}(t)$ is the probability of being in state $j$ after elapsed time $t$, given state $i$ at time zero. The generator $Q=(q_{ij})$ has nonnegative off-diagonal entries, and $q_{ii}=-\sum_{j\ne i}q_{ij}$. For a finite-state time-homogeneous model, $Q$ is constant and the forward equation is $P'(t)=P(t)Q$, with $P(0)=I$.

Definition: For a square matrix $Q$ and real $t$, the matrix exponential is defined by the convergent power series $$ e^{Qt}=\sum_{n=0}^{\infty}\frac{(Qt)^n}{n!} =I+Qt+\frac{Q^2t^2}{2!}+\frac{Q^3t^3}{3!}+\cdots. $$ Here $I$ is the identity matrix of the same dimension as $Q$.

Deriving and Verifying the Solution

The key fact is that $e^{Qt}$ can be differentiated term by term. For any finite matrix $Q$, the power series and its differentiated series converge uniformly on every bounded interval of $t$. Thus term-by-term differentiation is valid there. Since $Q$ is constant, each derivative of a power $(Qt)^n$ produces a factor $nQ$, and the resulting series can be re-indexed:

\begin{aligned} \frac{d}{dt}e^{Qt} &=\sum_{n=1}^{\infty}\frac{nQ^n t^{n-1}}{n!}\\ &=Q\sum_{n=1}^{\infty}\frac{Q^{n-1}t^{n-1}}{(n-1)!}\\ &=Qe^{Qt}. \end{aligned}

Because every term in the defining series is a power of the same matrix $Q$, $Q$ commutes with $e^{Qt}$. Therefore $Qe^{Qt}=e^{Qt}Q$. This establishes the forward equation with the row convention, not just a matrix differential equation with an unspecified multiplication order. At $t=0$, all positive powers vanish, so $e^{Q0}=I$. Consequently, $e^{Qt}$ satisfies both the differential equation and its required initial condition.

Theorem: For a constant finite-state generator $Q$, the unique solution of the initial-value problem $$ P'(t)=P(t)Q,\qquad P(0)=I $$ is $P(t)=e^{Qt}$. The same differentiation verifies that this candidate solves the equation; uniqueness follows from the argument below.

The derivative at zero also links the solution to the generator interpretation. Since $P'(0)=Q$, the off-diagonal entry $q_{ij}$ is the instantaneous rate of transition from state $i$ to state $j$. Differentiating once more gives $P''(t)=Q^2e^{Qt}$, so $P''(0)=Q^2$. These derivatives describe the local behavior of the transition probabilities; a truncated power series is only an approximation unless the omitted terms are zero.

Worked Example: Finding Initial Slopes and Curvature

A hypothetical three-state model has states 1, 2, and 3. The only possible transitions are from state 1 to state 2 at rate $2$ per year and from state 2 to state 3 at rate $3$ per year; state 3 is absorbing. Use the generator to find the initial slopes and second derivatives of the transition probabilities from state 1.

Method. Form $Q$, then use $P'(0)=Q$ and $P''(0)=Q^2$. Time is measured in years, so the rates in $Q$ are per year.

The diagonal entries are the negatives of the total rates out of their states:

Q= \begin{bmatrix} -2&2&0\\ 0&-3&3\\ 0&0&0 \end{bmatrix}.

The first row of $Q$ gives the initial slopes for transitions from state 1: $p'_{11}(0)=-2$, $p'_{12}(0)=2$, and $p'_{13}(0)=0$, all per year. For the second derivatives, multiply the matrices:

Q^2= \begin{bmatrix} (-2)(-2)+2(0)&(-2)(2)+2(-3)&(-2)(0)+2(3)\\ 0&(-3)(-3)&(-3)(3)\\ 0&0&0 \end{bmatrix} = \begin{bmatrix} 4&-10&6\\ 0&9&-9\\ 0&0&0 \end{bmatrix}.

Thus $p''_{11}(0)=4$, $p''_{12}(0)=-10$, and $p''_{13}(0)=6$, in units of inverse years squared. As a local check, the Taylor expansion for the first row begins $ (1,0,0)+t(-2,2,0)+\frac{t^2}{2}(4,-10,6)$. The constant terms give the initial state, and the linear terms agree with the stated instantaneous transition rates.

Check. The entries in each derivative row sum to zero: $-2+2+0=0$ and $4-10+6=0$. This is consistent with the row sums of $P(t)$ being one. The common mistake avoided here is treating the second derivative as $Q$ again; differentiating $P'=PQ$ gives $P''=P Q^2$, and hence $P''(0)=Q^2$.

Why the Solution Is Unique

Showing that $e^{Qt}$ is one solution does not by itself rule out other solutions. To prove uniqueness, suppose $R(t)$ and $S(t)$ both satisfy the same forward equation and initial condition. Their difference $D(t)=R(t)-S(t)$ satisfies $D'(t)=D(t)Q$ and $D(0)=0$. Multiply this difference on the right by $e^{-Qt}$, and define $E(t)=D(t)e^{-Qt}$. The product rule and the already established derivative formula give:

\begin{aligned} E'(t) &=D'(t)e^{-Qt}+D(t)\frac{d}{dt}e^{-Qt}\\ &=D(t)Qe^{-Qt}-D(t)Qe^{-Qt}\\ &=0. \end{aligned}

In the second line, $Qe^{-Qt}=e^{-Qt}Q$ allows the derivative term $-D(t)Qe^{-Qt}$ to be written in the displayed order. Since $E'(t)=0$, $E(t)$ is constant. Its initial value is $E(0)=D(0)I=0$, so $E(t)=0$ throughout the interval. The matrix $e^{-Qt}$ is invertible, with inverse $e^{Qt}$; multiplying by that inverse shows $D(t)=0$. Therefore $R(t)=S(t)$ at every time in the interval.

This argument uses a finite, constant matrix $Q$ and solutions differentiable on the time interval considered. More generally, a linear system with continuous time-dependent coefficients also has a unique solution for a specified initial condition, but its solution is not generally $e^{Qt}$ for a single constant matrix. The constant-generator assumption is essential to this particular exponential formula and proof.

Worked Example: Comparing Two Claimed Solutions

For a hypothetical model with three states, suppose transitions occur from state 1 to state 2 at rate $1$ per year and from state 2 to state 3 at rate $2$ per year, with state 3 absorbing. On the interval $0\le t\le 4$ years, two analysts claim to have found differentiable solutions $R(t)$ and $S(t)$ to the forward equation with $R(0)=S(0)=I$. Determine whether they can differ at $t=1.5$ years.

Method. Apply the uniqueness transformation to $D=R-S$. The generator is

Q= \begin{bmatrix} -1&1&0\\ 0&-2&2\\ 0&0&0 \end{bmatrix}.

Both claimed solutions satisfy $P'=PQ$, so subtracting their equations gives $D'=DQ$. Their identical initial conditions give $D(0)=I-I=0$. Set $E(t)=D(t)e^{-Qt}$. Then

E'(t)=D(t)Qe^{-Qt}-D(t)Qe^{-Qt}=0, \qquad E(0)=0.

Therefore $E(1.5)=0$. Since $e^{-1.5Q}$ is invertible, $D(1.5)=E(1.5)e^{1.5Q}=0$. Hence $R(1.5)=S(1.5)$: the analysts cannot have different solutions at that time, or at any other time in the interval.

Check. The conclusion depends on both analysts using the same constant $Q$, the same forward-equation convention, and the same initial condition. The argument would not establish equality if, for example, their initial matrices differed.

Common Mistakes and Limits of the Formula

A first-order approximation can be useful for small $t$: $e^{Qt}$ begins $I+tQ$. But this approximation is not the exact solution in general. The next term, $\frac{t^2}{2}Q^2$, is usually nonzero. A candidate matrix must satisfy the differential equation at every time, not merely match the initial value and initial slope.

Worked Example: Testing a First-Order Candidate

Consider a hypothetical three-state model with transitions from state 1 to state 2 at rate $2$ per year and from state 2 to state 3 at rate $1$ per year; state 3 is absorbing. An analyst proposes $R(t)=I+tQ$ as the exact transition matrix. Test the proposal at $t=0.25$ years.

Method. Substitute the candidate into the forward equation and calculate its residual $R'(t)-R(t)Q$. Here

Q= \begin{bmatrix} -2&2&0\\ 0&-1&1\\ 0&0&0 \end{bmatrix}, \qquad Q^2= \begin{bmatrix} 4&-6&2\\ 0&1&-1\\ 0&0&0 \end{bmatrix}.

The candidate has $R'(t)=Q$, whereas $R(t)Q=(I+tQ)Q=Q+tQ^2$. At $t=0.25$,

R'(0.25)-R(0.25)Q =-0.25Q^2 = \begin{bmatrix} -1&1.5&-0.5\\ 0&-0.25&0.25\\ 0&0&0 \end{bmatrix} \ne 0.

Thus $I+tQ$ does not satisfy the forward equation at $t=0.25$, and is not the exact solution. Indeed, $R(0.25)$ has rows $(0.5,0.5,0)$, $(0,0.75,0.25)$, and $(0,0,1)$, which look like valid probability vectors. That does not repair the nonzero residual: plausible entries at one time do not prove the proposed trajectory solves the equation.

Check. At $t=0$, the residual is zero, so the candidate has the correct initial value and initial slope. The failure at $t=0.25$ comes from omitting the nonzero $Q^2$ and higher-order terms. The common mistake is to confuse a first-order approximation with an exact matrix exponential.

There is a related caution for time-inhomogeneous models. If the generator is $Q(t)$, the forward equation is $P'(t)=P(t)Q(t)$. Replacing the constant-generator solution by $\exp\left(\int_0^t Q(s)\,ds\right)$ is not generally valid: matrices at different times need not commute, so the differentiation rule used above does not apply to that expression in general.

Key takeaway: For a finite-state, time-homogeneous model under the row convention, term-by-term differentiation proves that $e^{Qt}$ satisfies $P'=PQ$ and $P(0)=I$. A difference transformation proves it is the unique solution. The formula depends on a constant generator and should not be replaced by a truncated series or applied unchanged to a time-varying generator.

Check Your Understanding

For each problem, identify the equation or property being checked before calculating.

A hypothetical two-state model has transition rates $4$ per year from state 1 to state 2 and $1$ per year from state 2 to state 1. Write $Q$ and give $P'(0)$. (Answer: $Q=P'(0)=\begin{bmatrix}-4&4\\1&-1\end{bmatrix}$.)
For $Q=\begin{bmatrix}-1&1\\0&0\end{bmatrix}$, calculate $P''(0)$ for the constant-generator solution. (Answer: $Q^2=\begin{bmatrix}1&-1\\0&0\end{bmatrix}$.)
For $Q=\begin{bmatrix}-2&2\\0&0\end{bmatrix}$, a candidate is $R(t)=I+tQ$. What is $R'(0.1)-R(0.1)Q$, with time in years? (Answer: $-0.1Q^2=\begin{bmatrix}-0.4&0.4\\0&0\end{bmatrix}$, so the candidate is not an exact solution.)
Two differentiable matrices solve $P'=PQ$ for the same constant $Q$ and have the same initial matrix. State the transformation that proves they are identical. (Answer: Set $D=R-S$ and $E=De^{-Qt}$; then $E'=0$ and $E(0)=0$, so $D=0$.)
A time-varying generator is $Q(t)=A+tB$, where $A=\begin{bmatrix}-1&1\\0&0\end{bmatrix}$ and $B=\begin{bmatrix}0&0\\0&-1\end{bmatrix}$. Calculate $AB-BA$, and say whether the constant-matrix derivative argument automatically justifies $\exp(tA+t^2B/2)$ as the solution. (Answer: $AB-BA=\begin{bmatrix}0&-1\\0&0\end{bmatrix}\ne0$; no, the matrices do not commute, so that exponential cannot be justified by the constant-generator argument.)

What You'll Learn