Constant-Intensity Solutions by Matrix Exponentials

From Constant Intensities to Transition Probabilities

In a continuous-time multiple-state model, the entries of the generator describe instantaneous rates of movement between states. When those rates remain constant over an interval, one matrix exponential gives the transition probabilities for that interval. The calculation may be done by diagonalizing the generator or by using a convenient algebraic structure in the matrix.

The method is useful when several states interact and a single transition probability cannot be found by considering only one rate. It also makes the time interval explicit: a generator measured per year must be multiplied by an interval measured in years.

Definition: Let $Q$ be a constant generator matrix for a finite-state continuous-time Markov chain, using the row convention: $q_{ij}$ for $i\ne j$ is the intensity of a transition from state $i$ to state $j$, and $q_{ii}=-\sum_{j\ne i}q_{ij}$. For $s\le t$, the transition matrix is $$ P(s,t)=\exp\bigl(Q(t-s)\bigr) =\sum_{n=0}^{\infty}\frac{Q^n(t-s)^n}{n!}. $$ Its entry $P_{ij}(s,t)$ is the probability of being in state $j$ at time $t$, conditional on being in state $i$ at time $s$.

The formula assumes the generator is constant throughout the interval $[s,t]$, and that the process is described by the specified continuous-time Markov model. It does not apply to a generator that changes with time by simply substituting an average generator; time-varying intensities require different treatment. Under the row convention, each row of $Q$ sums to zero and each row of $P(s,t)$ sums to one.

Diagonalization: Turning a Matrix Exponential into Scalar Exponentials

If $Q$ is diagonalizable, write $Q=V\Lambda V^{-1}$, where the columns of $V$ are eigenvectors and the diagonal entries of $\Lambda$ are the corresponding eigenvalues. Then

\exp(Qh)=V\exp(\Lambda h)V^{-1}, \qquad h=t-s,

and $\exp(\Lambda h)$ is diagonal, with entries $e^{\lambda_i h}$. This replaces a matrix exponential with scalar exponentials and matrix multiplication. The eigenvalues of a finite-state generator have nonpositive real parts; the zero eigenvalue is consistent with the fact that probabilities are conserved.

Why the diagonalization formula holds: From $Q=V\Lambda V^{-1}$, multiplying repeatedly gives $Q^n=V\Lambda^nV^{-1}$ for every nonnegative integer $n$. Substitution into the power series for the matrix exponential gives $$ \exp(Qh)=\sum_{n=0}^{\infty}\frac{V\Lambda^nV^{-1}h^n}{n!} =V\left(\sum_{n=0}^{\infty}\frac{\Lambda^nh^n}{n!}\right)V^{-1} =V\exp(\Lambda h)V^{-1}. $$

Worked Example: Diagonalizing a Three-State Generator

A hypothetical model has three states, numbered 1, 2, and 3. From either end state, the process moves to the middle state at intensity 1 per year; from the middle state it moves to either end state at intensity 1 per year. Assume these intensities are constant. Find the transition matrix over $h=\ln 2$ years and the probability of moving from state 1 to state 3 over that interval.

Method. Form $Q$, find its eigenvalues and eigenvectors, and use the diagonalization formula. The generator is

Q= \begin{pmatrix} -1&1&0\\ 1&-2&1\\ 0&1&-1 \end{pmatrix} \quad\text{per year}.

The eigenvalues are $0,-1,-3$, with corresponding eigenvectors $(1,1,1)^\mathsf{T}$, $(1,0,-1)^\mathsf{T}$, and $(1,-2,1)^\mathsf{T}$. These eigenvectors are orthogonal. Their squared lengths are 3, 2, and 6, respectively, so the projection matrices are each the outer product of an eigenvector with itself, divided by its squared length. At $h=\ln 2$, the scalar exponential factors are

e^{0h}=1,\qquad e^{-h}=\frac12,\qquad e^{-3h}=\frac18.

Thus, the three weighted projections are

\frac13 \begin{pmatrix}1&1&1\\1&1&1\\1&1&1\end{pmatrix} +\frac14 \begin{pmatrix}1&0&-1\\0&0&0\\-1&0&1\end{pmatrix} +\frac1{48} \begin{pmatrix}1&-2&1\\-2&4&-2\\1&-2&1\end{pmatrix}.

Adding corresponding entries gives

P(s,s+\ln 2)= \begin{pmatrix} 29/48&7/24&5/48\\ 7/24&5/12&7/24\\ 5/48&7/24&29/48 \end{pmatrix} \approx \begin{pmatrix} 0.6042&0.2917&0.1042\\ 0.2917&0.4167&0.2917\\ 0.1042&0.2917&0.6042 \end{pmatrix}.

The requested probability is $P_{13}=5/48=0.1042$, rounded to four decimal places. As a check, the first row sums to $29/48+14/48+5/48=1$; the other rows also sum to 1. The matrix is symmetric here because the intensities in opposite directions along each connection are equal. That symmetry is specific to this example, not a general property of transition matrices.

Common mistake avoided: Do not exponentiate each entry separately. That would give $e^{q_{12}h}=e^{\ln 2}=2$ for the $(1,2)$ entry, which cannot be a transition probability. The matrix exponential includes matrix powers and gives $P_{12}=7/24$, not 2.

Direct Methods When the Generator Has a Useful Structure

Diagonalization is not always the most efficient route. Some generators can be written using a projection matrix, which makes their powers especially simple. If $\Pi^2=\Pi$ and $Q=\lambda(\Pi-I)$, then $Q$ acts as zero on the range of $\Pi$ and as multiplication by $-\lambda$ on the complementary component. This yields a compact expression for the exponential.

Useful direct result: If $\Pi^2=\Pi$ and $Q=\lambda(\Pi-I)$, then $$ \exp(Qh)=\Pi+e^{-\lambda h}(I-\Pi). $$ This result applies to a matrix with the stated projection structure; it is not a formula for an arbitrary generator.

To see the result, note that $\Pi(I-\Pi)=(I-\Pi)\Pi=0$. On the part represented by $\Pi$, the exponential leaves the vector unchanged; on the part represented by $I-\Pi$, it multiplies by $e^{-\lambda h}$. Equivalently, the power-series terms can be grouped using the fact that the two components do not interact.

Worked Example: A Generator with a Reset Distribution

In a hypothetical three-state model, events occur at rate $\lambda=0.6$ per year, after which the state is assigned according to the fixed distribution $\pi=(1/2,1/3,1/6)$. Use the generator $Q=\lambda(\Pi-I)$, where every row of $\Pi$ equals $\pi$. Find the transition matrix over 2 years and the probability of starting in state 2 and ending in state 1.

The matrix $\Pi$ and the resulting generator are

\Pi= \begin{pmatrix} 1/2&1/3&1/6\\ 1/2&1/3&1/6\\ 1/2&1/3&1/6 \end{pmatrix}, \qquad Q= \begin{pmatrix} -0.3&0.2&0.1\\ 0.3&-0.4&0.1\\ 0.3&0.2&-0.5 \end{pmatrix} \quad\text{per year}.

Each off-diagonal entry is nonnegative and each row sums to zero. Also, $\Pi^2=\Pi$, since applying the same distribution to any row of $\Pi$ leaves that row unchanged. Over $h=2$ years, $\lambda h=1.2$, so

P(s,s+2)=\Pi+e^{-1.2}(I-\Pi).

Using $e^{-1.2}=0.3011942$, the coefficient $1-e^{-1.2}$ is $0.6988058$. The diagonal entries are found by adding $e^{-1.2}$ to $1-e^{-1.2}$ times the corresponding $\pi_i$; each off-diagonal entry is $(1-e^{-1.2})\pi_j$. Therefore, to four decimal places,

P(s,s+2)\approx \begin{pmatrix} 0.6506&0.2330&0.1165\\ 0.3494&0.5341&0.1165\\ 0.3494&0.2330&0.4177 \end{pmatrix}.

The required probability is $P_{21}=0.3494029$, or $0.3494$ to four decimal places. Before rounding, every row sums to exactly 1 because each row of $\Pi$ sums to 1 and each row of $I-\Pi$ sums to zero. The displayed rounded values may differ from 1 by 0.0001 in a row.

Common mistake avoided: The destination distribution $\pi$ is not itself the transition matrix after a finite interval. The formula includes the probability of no event affecting the state, represented by $e^{-\lambda h}$, as well as the contribution from the reset distribution.

Applying the Matrix and Composing Intervals

With the row convention, a distribution over starting states is a row vector. If $\boldsymbol{\alpha}$ is the distribution at time $s$, then the distribution at time $t$ is $\boldsymbol{\alpha}P(s,t)$. For a constant generator, the elapsed time is what matters: $P(s,t)=\exp(Q(t-s))$. In particular, composition over two adjacent intervals agrees with the exponential over their combined duration.

P(s,u)P(u,t)=P(s,t), \qquad \exp(Q(u-s))\exp(Q(t-u))=\exp(Q(t-s)).

Worked Example: Combining Two Intervals and an Initial Distribution

Use the three-state generator from the first worked example, with constant rates per year. The process begins with distribution $\boldsymbol{\alpha}=(0.2,0.5,0.3)$. Find the probability that it is in state 2 after two successive intervals, each of length $\ln 2$ years.

For one such interval, the transition matrix $M$ is the matrix calculated in the first example. Composition gives $P=M^2$. Equivalently, the total elapsed time is $2\ln 2$, so the scalar factors for the three eigenvalues are $1$, $e^{-2\ln 2}=1/4$, and $e^{-6\ln 2}=1/64$. Using the same projection matrices as before gives

P= \begin{pmatrix} 59/128&21/64&27/128\\ 21/64&11/32&21/64\\ 27/128&21/64&59/128 \end{pmatrix}.

For example, the first-row entries are $1/3+1/8+1/384=59/128$, $1/3-1/96=21/64$, and $1/3-1/8+1/384=27/128$. The middle diagonal entry is $1/3+1/96=11/32$. The initial distribution assigns total probability $0.2+0.3=0.5$ to the two end states, each of which has probability $21/64$ of ending in state 2; the initial probability $0.5$ in state 2 contributes $11/32$. Thus,

P(\text{state 2 at the end}) =0.5\left(\frac{21}{64}\right)+0.5\left(\frac{11}{32}\right) =\frac{43}{128} =0.3359375.

The answer is $0.3359$ to four decimal places. The check is that each row of $P$ sums to 1, and the final state-2 probability lies between 0 and 1. This calculation uses the full initial distribution; using only the transition probability from one selected starting state would omit part of the population.

Checks and Common Exam Traps

A computed exponential should be checked as a transition matrix, not just accepted because the matrix multiplication is complete. In a finite-state model, all entries should be nonnegative and each row should sum to one, up to rounding. Also check the units before exponentiating: $Qh$ must be dimensionless. If $Q$ is specified per month, an 18-month interval uses $h=18$, not $h=1.5$; if it is per year, the same interval uses $h=1.5$ years.

When using eigenvectors, keep their order consistent. If $Q=V\Lambda V^{-1}$, the calculation is $V\exp(\Lambda h)V^{-1}$. Reversing the order of $V$ and $V^{-1}$ generally produces the wrong matrix. In a row-vector model, multiply the initial distribution on the left. A result with negative entries, row sums far from one, or an impossible entry larger than one signals an error in the generator, time units, matrix order, or arithmetic.

Key takeaway: For constant intensities, calculate $P(s,t)=\exp(Q(t-s))$. Use diagonalization when eigenvectors are convenient, exploit special matrix structure when available, and always check time units, row convention, nonnegativity, and row sums.

Check Your Understanding

For each problem, use the row convention and give probabilities to four decimal places where needed.

A three-state model has generator $Q=\begin{pmatrix}-1&1&0\\1&-2&1\\0&1&-1\end{pmatrix}$ per year. Over $\ln 2$ years, what is the probability of moving from state 1 to state 3? (Answer: 0.1042)
Let $Q=0.4(\Pi-I)$, where every row of $\Pi$ is $(1/4,1/2,1/4)$. For $h=\ln 2/0.4$ years, find the probability of starting in state 3 and ending in state 1. (Answer: 0.1250)
A three-state model has generator $Q=\begin{pmatrix}-4&2&2\\2&-4&2\\2&2&-4\end{pmatrix}$ per year. Find $P_{12}$ for an interval of $\ln 2/6$ years. (Answer: 0.1667)
For the generator in Question 1, the process starts with distribution $(0.2,0.5,0.3)$. Find the probability of being in state 2 after two adjacent intervals, each of length $\ln 2$ years. (Answer: 0.3359)
A constant generator is measured per month, and the requested transition matrix spans 18 months. State the correct exponential to compute. (Answer: $\exp(18Q)$)

What You'll Learn