Why Jump Frequencies Are Not Time Frequencies
Observing a continuous-time Markov chain at each jump gives a discrete-time chain, but the fraction of jumps made from a state is not generally the fraction of elapsed time spent there. A state with a small departure rate holds the process for longer on each visit. The stationary probability in continuous time must therefore account for both how frequently the jump chain visits a state and how long the process stays there.
Let \((X_t)_{t\ge0}\) be a continuous-time Markov chain on a finite or countable state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Assume it is conservative and nonexplosive, and that every state has a finite, strictly positive departure rate \(q_i\). Its generator \(Q=(q_{ij})\) has \(q_{ij}\ge0\) for \(j\ne i\), \(q_{ii}=-q_i\), and rows summing to zero. When the chain leaves \(i\), its next state has probability \(r_{ij}=q_{ij}/q_i\) of being \(j\ne i\). The matrix \(R=(r_{ij})\) is the transition matrix of the jump chain \((Y_n)_{n\ge0}\), whose state is recorded after each successive jump.
Throughout, a stationary distribution for \(R\) means a probability row vector \(\alpha\) satisfying \(\alpha R=\alpha\). A stationary distribution for the continuous-time chain means a probability row vector \(\pi\) such that \(\pi P(t)=\pi\) for every \(t\ge0\), where \(P(t)\) is the transition semigroup. These are different notions: \(\alpha\) describes proportions of visits at jump epochs, whereas \(\pi\) describes proportions of elapsed time.
The factor \(1/q_i\) is the mean holding time in state \(i\). Thus the formula has the form “frequency of visits multiplied by mean time per visit,” followed by normalization. The finiteness of \(Z\) is essential: if it is infinite, these weights cannot be turned into a probability distribution by dividing by their sum.
Proof. Since \(\alpha\) is a probability distribution and every \(q_i\) is finite and positive, at least one \(\alpha_i\) is positive, so \(Z>0\). Under the stated condition \(Z<\infty\), the numbers \(\pi_i=(\alpha_i/q_i)/Z\) are nonnegative and sum to one. For \(i\ne j\), \(q_{ij}=q_i r_{ij}\). Therefore, for each \(j\), the total off-diagonal rate into \(j\) under \(\pi\) is \(\sum_{i\ne j}\pi_iq_{ij}=Z^{-1}\sum_{i\ne j}\alpha_i r_{ij}\). Because \(r_{jj}=0\), stationarity of \(\alpha\) gives \(\sum_{i\ne j}\alpha_i r_{ij}=\alpha_j\). The diagonal term is \(\pi_jq_{jj}=-\pi_jq_j=-\alpha_j/Z\), so the incoming and diagonal terms cancel and \((\pi Q)_j=0\). These sums are well-defined: their off-diagonal terms are nonnegative and their sum is \(\alpha_j/Z<\infty\). If the rates are bounded, choose a finite \(\Lambda\ge\sup_iq_i\) with \(\Lambda>0\). Then \(A=I+Q/\Lambda\) is stochastic and \(\pi A=\pi\). Uniformization gives \(P(t)=e^{-\Lambda t}\sum_{n\ge0}(\Lambda t)^nA^n/n!\); since \(\pi A^n=\pi\) for every \(n\), this yields \(\pi P(t)=\pi\). For unbounded rates, this last argument does not apply, which is why generator balance must not be asserted to imply invariance without additional conditions. \(\square\)
Computing and Checking the Weights
Worked Example: A Three-State Directed Cycle
Consider a chain on \(S=\{1,2,3\}\) that jumps from \(1\) to \(2\) at rate \(2\), from \(2\) to \(3\) at rate \(1\), and from \(3\) to \(1\) at rate \(4\). Find the stationary distribution of the jump chain and the stationary distribution in continuous time.
The generator is \(Q=\begin{bmatrix}-2&2&0\\0&-1&1\\4&0&-4\end{bmatrix}\). Its row sums are \(-2+2+0=0\), \(0-1+1=0\), and \(4+0-4=0\). Dividing each off-diagonal row by its departure rate gives \(R=\begin{bmatrix}0&1&0\\0&0&1\\1&0&0\end{bmatrix}\); each row sums to \(1\). The equation \(\alpha R=\alpha\) makes all three coordinates equal, and their sum is one, so \(\alpha=(1/3,1/3,1/3)\).
The unnormalized time weights are \(\alpha_1/q_1=1/6\), \(\alpha_2/q_2=1/3\), and \(\alpha_3/q_3=1/12\). Their sum is \(Z=1/6+1/3+1/12=7/12\). Dividing each by \(7/12\) gives \(\pi=(2/7,4/7,1/7)\).
Check: The three coordinates sum to \(2/7+4/7+1/7=1\). Multiplication by the generator gives \((\pi Q)_1=-(2/7)2+(1/7)4=0\), \((\pi Q)_2=(2/7)2-(4/7)1=0\), and \((\pi Q)_3=(4/7)1-(1/7)4=0\). The rates are bounded, so the theorem’s uniformization argument also establishes invariance.
Worked Example: A Two-State Chain
Let \(S=\{1,2\}\), with a jump from \(1\) to \(2\) at rate \(2\), and a jump from \(2\) to \(1\) at rate \(1\). Find the continuous-time stationary distribution.
Here \(Q=\begin{bmatrix}-2&2\\1&-1\end{bmatrix}\), whose row sums are \(-2+2=0\) and \(1-1=0\). The jump-chain matrix is \(R=\begin{bmatrix}0&1\\1&0\end{bmatrix}\), and both rows sum to one. Solving \(\alpha R=\alpha\) with \(\alpha_1+\alpha_2=1\) gives \(\alpha=(1/2,1/2)\). The unnormalized weights are \(\alpha_1/q_1=1/4\) and \(\alpha_2/q_2=1/2\), with sum \(Z=3/4\). Thus \(\pi=(1/3,2/3)\).
Check: The coordinates sum to \(1/3+2/3=1\). Also, \((\pi Q)_1=-(1/3)2+(2/3)1=0\), and \((\pi Q)_2=(1/3)2-(2/3)1=0\). Each rate is finite and the state space is finite, so the resulting distribution is invariant.
When the Weights Can Be Normalized
For a countable state space, the existence of a stationary jump-chain distribution does not by itself ensure that the time weights have finite sum. The exact test for this formula is the series \(Z=\sum_i\alpha_i/q_i\). If \(Z<\infty\), normalization produces a probability distribution satisfying generator balance. If \(Z=\infty\), no constant multiple of the weights \(\alpha_i/q_i\) has total mass one. A useful interpretation is that stationary jump frequencies can coexist with so much expected holding time across states that the total time weight is infinite.
Worked Example: A Countable Jump Chain with Two Rate Choices
Take \(S=\{1,2,\ldots\}\). The jump chain moves from \(1\) to \(2\) with probability \(1\); from each \(i\ge2\), it moves to \(i-1\) with probability \(2/3\) and to \(i+1\) with probability \(1/3\). The row for state \(1\) sums to \(1\), and each other row sums to \(2/3+1/3=1\). Its stationary distribution is \(\alpha_1=1/4\) and \(\alpha_i=3/2^{i+1}\) for \(i\ge2\).
To verify stationarity at state \(1\), the incoming mass is \((2/3)\alpha_2=(2/3)(3/8)=1/4=\alpha_1\). At state \(2\), it is \(\alpha_1+(2/3)\alpha_3=1/4+(2/3)(3/16)=3/8=\alpha_2\). For \(j\ge3\), it is \((1/3)\alpha_{j-1}+(2/3)\alpha_{j+1}\). Substitution gives \((1/3)(3/2^j)+(2/3)(3/2^{j+2})=3/2^{j+1}=\alpha_j\). Finally, \(\sum_i\alpha_i=1/4+\sum_{i=2}^\infty3/2^{i+1}=1/4+3/4=1\).
First choose \(q_i=1\) for every state. Then all generator rows sum to zero by taking \(q_{ij}=r_{ij}\) off the diagonal and \(q_{ii}=-1\). The normalizing sum is \(Z=\sum_i\alpha_i=1\), so the formula gives \(\pi_i=\alpha_i\). This is a valid invariant distribution: the rates are bounded, and the theorem applies.
Now instead choose \(q_i=\alpha_i\). These are positive and bounded above by \(3/8\), so the rates are finite and bounded; define \(q_{ij}=q_i r_{ij}\) off the diagonal and \(q_{ii}=-q_i\), making every generator row sum to zero. For every \(i\), \(\alpha_i/q_i=1\), hence \(Z=\sum_{i=1}^\infty1=\infty\). The weights cannot be normalized. In particular, the jump-chain stationary probabilities alone do not determine a probability distribution in continuous time; the departure rates matter.
The Converse and a Common Mistake
There is also a converse under a finite-flux condition. Suppose \(\pi\) is an invariant probability distribution, its stationary generator balance equations hold, and \(c=\sum_i\pi_iq_i\) is finite and positive. Define \(\alpha_i=\pi_iq_i/c\). This is the distribution of states weighted by their departure rates, not their time proportions.
Proof. By the definition of \(c\), \(\sum_i\alpha_i=1\). For any state \(j\), the balance equation is \(\sum_{i\ne j}\pi_iq_{ij}=\pi_jq_j\). Since \(q_{ij}=q_ir_{ij}\) for \(i\ne j\), it follows that \(\sum_i\alpha_ir_{ij}=c^{-1}\sum_{i\ne j}\pi_iq_{ij}=\pi_jq_j/c=\alpha_j\), using \(r_{jj}=0\). Thus \(\alpha R=\alpha\). Also, \(\alpha_i/q_i=\pi_i/c\), so \(\sum_i\alpha_i/q_i=1/c\), and normalizing these weights recovers \(\pi_i\). \(\square\)
Common mistake: multiplying by the departure rate. In the two-state example, the incorrect weights \(\alpha_iq_i\) would be \(1\) and \(1/2\), producing the alleged distribution \((2/3,1/3)\). Substitution shows it is wrong: \(((2/3,1/3)Q)_1=-(2/3)2+(1/3)1=-1\), not zero. The correct weights divide by the rate: \((\alpha_1/q_1,\alpha_2/q_2)=(1/4,1/2)\), which normalize to \((1/3,2/3)\). Slow departure means longer holding and therefore more, not less, stationary time mass.
Finally, an invariant distribution is not automatically a limiting distribution from every initial state. This tutorial’s formula concerns invariant probabilities; convergence to equilibrium requires additional conditions. When using the formula, check that each departure rate is strictly positive and finite, that \(\alpha\) is genuinely a probability distribution for the jump chain, and that \(Z\) is finite. Then check separately whether the conditions linking generator balance to invariance are satisfied.
Check Your Understanding
Use the jump-chain stationary distribution and the departure rates to answer each question.
- For a two-state chain with jump rates \(3\) from \(1\) to \(2\) and \(1\) from \(2\) to \(1\), find \(\alpha\), the unnormalized time weights, and \(\pi\).
- Explain why a state with a smaller departure rate receives a larger time weight when its jump-chain frequency is held fixed.
- For countable \(S\), what does \(Z=\infty\) imply about the weights \(\alpha_i/q_i\)?
- In the converse argument, why is \(\alpha_i\) proportional to \(\pi_iq_i\), rather than to \(\pi_i/q_i\)?
- Why does \(\pi Q=0\) require additional care before it can be identified with invariance when the rates are unbounded?