One Jump Chain, Different Clocks
The embedded jump chain records the successive states visited at jump times, but it does not record how much time passes between jumps. Two continuous-time Markov chains can therefore have the same probabilities for their next state and still evolve at different speeds. To distinguish them, one must specify both the jump probabilities and the exit rates.
Let \((X_t)_{t\ge0}\) be a continuous-time Markov chain on a finite state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with its natural filtration \((\mathcal{F}_t)_{t\ge0}\). For states \(i\ne j\), write \(q_{ij}\) for the rate of a jump from \(i\) to \(j\), and let \(Q=(q_{ij})\) be the generator. Its rows sum to zero, so \(q_{ii}=-\sum_{j\ne i}q_{ij}\). At a state with positive exit rate, define \(q_i=-q_{ii}\) and \(r_{ij}=q_{ij}/q_i\) for \(j\ne i\). The numbers \(r_{ij}\) give the transition probabilities of the embedded jump chain. The holding time at \(i\) is exponentially distributed with rate \(q_i\), and thus has mean \(1/q_i\).
For every state with \(q_i>0\), the rates and jump probabilities are related by
Thus, specifying all the \(r_{ij}\) does not specify \(Q\) unless the exit rates are also specified. This simple factorization is the central distinction in this tutorial.
What the Jump Probabilities Leave Unspecified
Consider two chains on the same finite state space with positive exit rates. Suppose their embedded jump chains have the same transition probabilities \(r_{ij}\), but their exit rates are \(q_i\) and \(\widetilde q_i\). Their generator entries are respectively \(q_i r_{ij}\) and \(\widetilde q_i r_{ij}\). If an exit rate changes, the holding-time distribution at that state changes, even though the next-state probabilities do not.
Proof. For each state \(i\) and each \(j\ne i\), the generator entry is \(q_{ij}=q_i r_{ij}\), and the diagonal entry is \(q_{ii}=-q_i\). These equations determine every entry of \(Q\) from the exit rates and embedded jump probabilities, proving the first assertion. If the exit rates differ at state \(i\), the diagonal entries at \(i\) differ, so the generators differ. The generator is the right derivative of the transition semigroup at zero: \(q_{ij}=\lim_{h\downarrow0}(p_{ij}(h)-\mathbf{1}_{\{i=j\}})/h\). If the two transition semigroups were identical for all \(t\ge0\), their right derivatives at zero would be identical entry by entry, contradicting the different generators. Therefore the semigroups cannot be identical. \(\square\)
The derivative identity in the proof also gives a direct diagnostic: a difference in a rate appears in the initial, per-unit-time change of a transition probability. It is not merely a difference in how the same sequence of states is described.
Worked Example: Doubling Every Exit Rate
Consider two chains on \(S=\{1,2\}\). In the first, the rates are \(q_{12}=1\) and \(q_{21}=2\); in the second, they are \(q_{12}=2\) and \(q_{21}=4\). Both chains jump from either state to the other with probability \(1\). Their generators are
Each generator row sums to zero: \(-1+1=0\), \(2-2=0\), \(-2+2=0\), and \(4-4=0\). The embedded jump probabilities are \(r_{12}=r_{21}=1\) for both chains. But the mean holding times are \(1\) and \(1/2\) in states 1 and 2 for the first chain, and \(1/2\) and \(1/4\) for the second.
For a two-state chain with rates \(q_{12}=a\) and \(q_{21}=b\), the transition probabilities are
To obtain these entries, let \(x(t)=p_{12}(t)\). Over a short interval of length \(h\), a chain in state 1 jumps to 2 with probability \(ah+o(h)\), while one in state 2 jumps to 1 with probability \(bh+o(h)\). Since the state space is finite and the rates are bounded, the probability of two or more jumps during this interval is \(o(h)\). Conditioning on the state at time \(t\) gives \(x'(t)=a(1-x(t))-bx(t)=a-(a+b)x(t)\), with \(x(0)=0\). Solving yields \(x(t)=\frac{a}{a+b}(1-e^{-(a+b)t})\). The same conditioning for \(y(t)=p_{21}(t)\) gives \(y'(t)=b(1-y(t))-ay(t)=b-(a+b)y(t)\), with \(y(0)=0\), and hence \(y(t)=\frac{b}{a+b}(1-e^{-(a+b)t})\). The diagonal entries are \(p_{11}(t)=1-x(t)\) and \(p_{22}(t)=1-y(t)\), giving the displayed matrix.
At \(t=\ln 2\), the first chain has \(a+b=3\) and \(e^{-3\ln2}=1/8\), so its transition matrix is
Its rows sum to \(17/24+7/24=1\) and \(7/12+5/12=1\). For the second chain, \(a+b=6\) and \(e^{-6\ln2}=1/64\), giving
Here the row sums are \(43/64+21/64=1\) and \(21/32+11/32=1\). The embedded jump chain is unchanged, but the transition matrices at the same elapsed time are different. In this example, multiplying all rates by 2 compresses the time scale: the second chain’s transition probabilities at time \(t\) equal the first chain’s at time \(2t\).
Changing One Exit Rate Also Changes the Semigroup
When every exit rate is multiplied by the same constant, the change can be understood as a common rescaling of time. If only some exit rates change, the timing changes differently in different states. The next comparison keeps the jump probabilities fixed but changes just one of the two exit rates.
Worked Example: Changing the Clock in Only One State
On \(S=\{1,2\}\), let the first chain have rates \(q_{12}=1\), \(q_{21}=1\). Let the second have rates \(\widetilde q_{12}=2\), \(\widetilde q_{21}=1\). Each chain alternates between states at its jumps, so both embedded jump chains have \(r_{12}=r_{21}=1\). The generators are
All four generator rows sum to zero: \(-1+1=0\), \(1-1=0\), \(-2+2=0\), and \(1-1=0\). At \(t=\ln2\), the first chain has \(a+b=2\) and \(e^{-2\ln2}=1/4\). Substitution into the two-state formula gives
The row sums are \(5/8+3/8=1\) and \(3/8+5/8=1\). For the second chain, \(a+b=3\) and \(e^{-3\ln2}=1/8\). Thus
These rows sum to \(5/12+7/12=1\) and \(7/24+17/24=1\). In state 1, the holding-time mean has changed from \(1\) to \(1/2\); in state 2, it remains \(1\). The transition matrices differ, and this difference cannot be captured by a single uniform change of time across both states.
Reconstructing Rates When the Jump Chain Is Shared
A jump chain on three states can have nontrivial choices at each jump. The following example displays the rate factorization row by row and shows how changing the clock at selected states changes the generator while retaining those choices.
Worked Example: The Same Three-State Jump Probabilities
Let the embedded jump probabilities be \(r_{12}=1/4,\ r_{13}=3/4\); \(r_{21}=r_{23}=1/2\); and \(r_{31}=1\). Each row sums to one: \(1/4+3/4=1\), \(1/2+1/2=1\), and \(1=1\). Choose exit rates \((2,3,4)\) for one chain and \((1,3,2)\) for another. Multiplying each jump-probability row by its exit rate gives
Check the generator row sums: \(-2+1/2+3/2=0\), \(3/2-3+3/2=0\), \(4+0-4=0\); and \(-1+1/4+3/4=0\), \(3/2-3+3/2=0\), \(2+0-2=0\). Dividing the off-diagonal entries in each row of \(Q\) by its exit rate gives the stated probabilities: in row 1, \((1/2)/2=1/4\) and \((3/2)/2=3/4\); in row 2, both \((3/2)/3=1/2\); in row 3, \(4/4=1\). For \(\widetilde Q\), the corresponding ratios are \((1/4)/1=1/4,\ (3/4)/1=3/4,\ (3/2)/3=1/2,\ (3/2)/3=1/2,\) and \(2/2=1\). The embedded jump probabilities therefore agree exactly.
Nevertheless, starting from state 1, the initial rate of transition to state 3 is \(q_{13}=3/2\) for the first chain and \(\widetilde q_{13}=3/4\) for the second. By the generator definition, \(p_{13}(h)=(3/2)h+o(h)\) for the first chain and \(\widetilde p_{13}(h)=(3/4)h+o(h)\) for the second as \(h\downarrow0\). Thus their transition semigroups differ, despite matching jump probabilities.
A Common Mistake and Its Correction
Common mistake: treating a jump probability as though it were a rate. In the three-state example, one might incorrectly put \(q_{12}=r_{12}=1/4\) in the first chain’s row 1. But its exit rate is \(q_1=2\), so the correct rate is \(q_{12}=q_1r_{12}=2(1/4)=1/2\). The incorrect entry would imply a different jump probability: if the row-1 exit rate remained 2, it would give \(q_{12}/q_1=(1/4)/2=1/8\), not \(1/4\). The correction is to multiply each jump probability by the exit rate of its starting state.
There is also a limiting case to keep in mind. At a state with \(q_i=0\), there is no next jump and hence no embedded jump probability defined by \(q_{ij}/q_i\). The examples here use positive exit rates throughout, so each visited state has a well-defined next-jump distribution. More generally, when interpreting an embedded chain, one must distinguish states with positive exit rates from absorbing states with no jump to follow.
The practical lesson is that a rate table contains two kinds of information: where the chain jumps, through \(r_{ij}\), and how quickly it leaves each state, through \(q_i\). To specify continuous-time behavior, both are needed. Matching only the embedded jump chain guarantees agreement about destinations at jump epochs; it does not guarantee agreement about holding times, transition probabilities at a fixed time, or the transition semigroup.
Check Your Understanding
For each question, distinguish jump probabilities from rates and check any proposed generator by its row sums.
- A state has exit rate \(q_i=5\) and jump probability \(r_{ij}=2/5\). What is the rate \(q_{ij}\), and what is the mean holding time at \(i\)?
- Two chains have the same embedded jump probabilities at every state, and their exit rates are \((1,2)\) and \((3,6)\). By what factor are all rates changed, and how are their time scales related?
- For a two-state chain with rates \(q_{12}=2\) and \(q_{21}=1\), write its generator and verify that both rows sum to zero.
- Why does a difference in one generator entry imply that two transition semigroups cannot agree for every \(t\ge0\)?
- What is missing if a model specifies only the embedded jump probabilities but not the exit rates?