A Three-State CTMC From Its Rate Table

From a Rate Table to a Continuous-Time Chain

A rate table gives a compact description of how a continuous-time Markov chain leaves each state: it specifies the rate of leaving and where the chain goes when it leaves. Turning that information into a generator makes the two parts of the dynamics explicit. The diagonal records how quickly time is spent waiting; the off-diagonal entries record the rates of particular jumps. From these entries we can recover both the distribution of the next jump and the distribution of the wait until it occurs.

Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), and let \((X_t)_{t\ge0}\) be a time-homogeneous continuous-time Markov chain with state space \(S=\{1,2,3\}\). Its natural filtration is \(\mathcal{F}_t=\sigma(X_s:0\le s\le t)\). A generator is a matrix \(Q=(q_{ij})\) whose off-diagonal entries are nonnegative and whose rows sum to zero. For \(i\ne j\), \(q_{ij}\) is the rate of a jump from \(i\) to \(j\). The exit rate from \(i\) is \(q_i=-q_{ii}=\sum_{j\ne i}q_{ij}\).

The distinction between a rate and a probability matters. The number \(a_i\) has units of inverse time and controls the waiting time. The numbers \(r_{ij}\) are dimensionless probabilities and describe the destination conditional on a jump. When \(a_i>0\), the embedded jump-chain transition probabilities are \(r_{ij}=q_{ij}/q_i\) for \(j\ne i\), with \(r_{ii}=0\).

Why the Conversion Gives the Holding and Jump Laws

Proof. Fix a state \(i\) with \(a_i>0\). Represent the possible outgoing destinations by independent exponential clocks, with clock \(j\) having rate \(a_i r_{ij}\) when \(r_{ij}>0\). Clocks with rate zero never ring and can be omitted. For \(t\ge0\), the probability that no clock has rung by time \(t\) is the product of their survival probabilities:

For a destination \(j\) with \(r_{ij}>0\), the density that clock \(j\) rings at time \(t\) while every other clock has not yet rung is its rate times the survival probabilities of all clocks at \(t\). Thus the joint density is \(a_i r_{ij}e^{-a_i t}\). Integrating over \(t\ge0\) gives \(\mathbb{P}(J=j)=a_i r_{ij}\int_0^\infty e^{-a_i t}\,dt=r_{ij}\), since the integral is \(1/a_i\). The joint density factors as \((a_i e^{-a_i t})r_{ij}\), the product of the holding-time density and the destination probability. This factorization proves independence. Destinations with \(r_{ij}=0\) have probability zero and satisfy the same formula. \(\square\)

This derivation also shows how to check a rate table before using it: exit rates must be nonnegative, destination probabilities must be nonnegative and sum to one wherever the exit rate is positive, and the resulting generator must have nonnegative off-diagonal entries and zero row sums. For a state with positive exit rate, the mean holding time is \(1/a_i\); its probability of exceeding \(t\) is \(e^{-a_i t}\). These holding times are random, not fixed durations.

Tracing a Path Through the Rate Table

For a sample path, record the state entered, the time spent there, and the next state. The first jump time from the initial state is the first holding time. Each later holding time and destination are determined by the rate table for the state just entered. A particular list of holding-time values is a realized path description; it is not a positive-probability event by itself, because exponential holding times have continuous distributions.

More generally, for a path starting in \(i_0\), making successive jumps to \(i_1,\ldots,i_m\), and spending durations \(t_0,\ldots,t_{m-1}\) in the states before those jumps, its joint density for those durations and destinations is

This follows by applying the holding-time and destination density at each visited state, conditional on the state entered. It describes the density for the specified finite sequence of jumps and holding times; it does not require assigning a density to the state after the last listed jump. If the question concerns only the destinations, sum or integrate out the holding times, leaving the product \(\prod_{\ell=0}^{m-1}r_{i_\ell i_{\ell+1}}\).

A Common Error and a Practical Checklist

Common mistake. It is incorrect to treat an off-diagonal rate as a jump probability. In the last example, writing \(r_{23}=q_{23}=1\) would assign probability \(1\) to a jump from state \(2\) to state \(3\), even though the rate of leaving state \(2\) is \(3\) and the competing rate to state \(1\) is \(2\). The correction is \(r_{23}=q_{23}/a_2=1/3\), while \(r_{21}=q_{21}/a_2=2/3\); these probabilities sum to \(1\). The rate \(q_{23}=1\) instead contributes to the total exit rate and to the generator.

When a rate table is given, proceed in this order: identify whether it lists total exit rates with destination probabilities or individual jump rates; multiply the former, or sum the latter, to find the off-diagonal rates and exit rates; set each diagonal entry to the negative exit rate; check that every generator row sums to zero; then divide outgoing rates by the exit rate to get the jump probabilities. Finally, use the exit rate—not a destination probability—to specify the exponential holding-time law. If the exit rate is zero, there is no next jump from that state, so the positive-exit-rate formulas for a destination distribution do not apply.