Stationarity Through the Transition Semigroup
A continuous-time Markov chain can have a probability distribution that remains unchanged as time passes. The right way to express this is through the whole family of transition matrices, not through a single transition matrix at one selected time. This distinction matters: a distribution may be unchanged by every transition matrix and yet fail to be the long-time limit from a particular starting state. Conversely, a chain may converge to an invariant distribution from some initial states without converging to every invariant distribution.
Let \((X_t)_{t\ge0}\) be a time-homogeneous, nonexplosive continuous-time Markov chain on a finite state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Its transition semigroup is \(P(t)=(p_{ij}(t))\), where \(p_{ij}(t)=\mathbb{P}(X_t=j\mid X_0=i)\). Each \(P(t)\) is row-stochastic, \(P(0)=I\), and the Chapman–Kolmogorov property gives \(P(s+t)=P(s)P(t)\). If the initial distribution is the row vector \(\mu\), then the distribution at time \(t\) is \(\mu P(t)\).
This definition concerns invariance under the semigroup. It does not say that the chain, started from an arbitrary law, converges to \(\pi\). A limiting distribution from initial law \(\mu\) is a probability vector \(\nu\) such that \(\mu P(t)\to\nu\) as \(t\to\infty\), with convergence in each coordinate. The initial law must be specified: in a reducible chain, different initial laws can have different limits.
Proof. Suppose first that \(\pi P(t)=\pi\) for every \(t\ge0\). For states \(i_1,\ldots,i_k\), the Markov property gives the joint probability at times \(t_1,\ldots,t_k\) as \(\pi_{i_1}p_{i_1i_2}(t_2-t_1)\cdots p_{i_{k-1}i_k}(t_k-t_{k-1})\). At the shifted times, the distribution at \(t_1+h\) is \(\pi P(t_1+h)=\pi\), and all successive time gaps are unchanged. The shifted joint probability is therefore the same product. Equality for every tuple of states gives equality of the joint distributions. Conversely, if the process is stationary in this sense, its one-time distribution at time \(h\) equals its one-time distribution at time \(0\). The Markov property gives the former as \(\pi P(h)\), so \(\pi P(h)=\pi\) for every \(h\ge0\). \(\square\)
Finite-State Examples
Worked Example: A Symmetric Two-State Chain
Let \(S=\{0,1\}\). The chain switches from either state to the other at rate \(1\); equivalently, its jump times are those of a rate-\(1\) Poisson process, and every jump switches the state. This specifies a nonexplosive continuous-time chain on the stated probability space. If \(N_t\) is its number of jumps by time \(t\), then \(X_t=X_0\) when \(N_t\) is even and \(X_t\ne X_0\) when \(N_t\) is odd.
Since \(N_t\) has the Poisson distribution with mean \(t\), its even and odd probabilities are \(\mathbb{P}(N_t\text{ even})=\frac{1+e^{-2t}}2\) and \(\mathbb{P}(N_t\text{ odd})=\frac{1-e^{-2t}}2\). Thus, writing \(a(t)=\frac{1+e^{-2t}}2\) and \(b(t)=\frac{1-e^{-2t}}2\), the transition matrix is \(P(t)=\begin{bmatrix}a(t)&b(t)\\b(t)&a(t)\end{bmatrix}\). Each row sums to \(a(t)+b(t)=1\), and both entries are nonnegative for \(t\ge0\).
For \(\pi=(\tfrac12,\tfrac12)\), direct multiplication gives \(\pi P(t)=(\tfrac12(a(t)+b(t)),\tfrac12(a(t)+b(t)))=(\tfrac12,\tfrac12)\) for every \(t\ge0\). Hence \(\pi\) is stationary. From either initial state, the distribution converges to \((\tfrac12,\tfrac12)\), since \(e^{-2t}\to0\). In this example, the invariant distribution is also the common limiting distribution. As a check at \(t=0\), \(a(0)=1\) and \(b(0)=0\), so \(P(0)=I\), as required.
Worked Example: A One-Way Chain with an Absorbing State
Let \(S=\{0,1\}\), with a transition from \(0\) to \(1\) at rate \(2\), and with state \(1\) absorbing. From state \(0\), the holding time before the jump is exponential with rate \(2\). Therefore the probability of still being in \(0\) at time \(t\) is \(e^{-2t}\), and \(P(t)=\begin{bmatrix}e^{-2t}&1-e^{-2t}\\0&1\end{bmatrix}\). The first row sums to \(e^{-2t}+1-e^{-2t}=1\); the second row sums to \(1\). All entries are nonnegative for \(t\ge0\).
For \(\pi=(0,1)\), multiplication yields \(\pi P(t)=(0,1)\), so the point mass at the absorbing state is stationary. In fact, it is the only stationary distribution: if \(\eta=(x,1-x)\), where \(0\le x\le1\), then the first coordinate of \(\eta P(t)\) is \(xe^{-2t}\). Invariance for every \(t>0\) requires \(xe^{-2t}=x\), which implies \(x=0\).
Now start at state \(0\), so \(\mu=(1,0)\). Then \(\mu P(t)=(e^{-2t},1-e^{-2t})\to(0,1)=\pi\). This chain converges to its stationary distribution from either initial state. The calculation also illustrates why the condition is about every \(t\): a distribution that happened to return to itself at one chosen time would not, on that fact alone, be stationary.
Worked Example: Several Invariant Distributions and Initial-Dependent Limits
Let \(S=\{0,1,2\}\). States \(0\) and \(2\) are absorbing. From state \(1\), the chain jumps to \(0\) at rate \(1\) and to \(2\) at rate \(2\). The waiting time in state \(1\) is exponential with rate \(3\), and the destination probabilities are \(1/3\) and \(2/3\). Consequently, \(P(t)=\begin{bmatrix}1&0&0\\\frac13(1-e^{-3t})&e^{-3t}&\frac23(1-e^{-3t})\\0&0&1\end{bmatrix}\). The middle row sums to \(\frac13(1-e^{-3t})+e^{-3t}+\frac23(1-e^{-3t})=1\); the other rows each sum to \(1\). All entries are nonnegative.
For any \(0\le c\le1\), the distribution \(\pi_c=(c,0,1-c)\) is invariant: multiplying by \(P(t)\) leaves mass \(c\) at state \(0\), mass \(1-c\) at state \(2\), and zero mass at state \(1\). These are all the stationary distributions. Indeed, for any probability vector \((x,y,z)\), the middle coordinate after multiplication is \(ye^{-3t}\). If the vector is invariant for all \(t\), then \(y=ye^{-3t}\) for \(t>0\), so \(y=0\); any remaining vector \((x,0,z)\) is invariant.
Starting from state \(1\), the distribution is the middle row of \(P(t)\), and its limit is \((\tfrac13,0,\tfrac23)\). The weights agree with the probabilities of jumping to the two absorbing states. Starting instead from state \(0\), the distribution stays \((1,0,0)\). Thus both limits are stationary, but they are different. In particular, the stationary distribution \((1,0,0)\) is not the limit from initial state \(1\). This is why “stationary” and “the limiting distribution” are not interchangeable descriptions.
A Limit, When It Exists, Must Be Invariant
Proof. A coordinatewise limit of probability vectors on a finite state space is again a probability vector: each limiting coordinate is nonnegative, and the finite sum of the coordinates is the limit of sums equal to \(1\). Fix any \(s\ge0\). By the semigroup property, \((\mu P(t))P(s)=\mu P(t+s)\). As \(t\to\infty\), the left side tends to \(\nu P(s)\), because multiplication by the fixed finite matrix \(P(s)\) is a finite sum in each coordinate. The right side tends to \(\nu\), since \(t+s\to\infty\). Hence \(\nu P(s)=\nu\). This holds for every \(s\ge0\), proving invariance. \(\square\)
The theorem proves only one direction: a limiting distribution must be invariant. It does not say that every invariant distribution is reached, or that a limit exists from every initial law. The three-state example showed both that distinct initial states can have distinct limits and that one invariant distribution need not be the limit from a given initial state. Additional structure is needed for stronger convergence claims; invariance alone is not a convergence theorem.
A Common Mistake: Multiplying in the Wrong Orientation
Distributions here are row vectors, and transition matrices are row-stochastic. For the one-way chain with rate \(2\), the stationary row vector is \(\pi=(0,1)\). The correct calculation is \(\pi P(t)=(0,1)\begin{bmatrix}e^{-2t}&1-e^{-2t}\\0&1\end{bmatrix}=(0,1)\).
A common incorrect calculation treats \(\pi\) as a column and multiplies \(P(t)\begin{bmatrix}0\\1\end{bmatrix}\). This gives \(\begin{bmatrix}1-e^{-2t}\\1\end{bmatrix}\), which is not even a probability vector: its coordinates sum to \(2-e^{-2t}\), not \(1\) for \(t>0\). The correction is to keep the convention consistent: use row distributions and compute \(\pi P(t)\). Also check that every row of \(P(t)\) sums to \(1\) before interpreting it as a transition matrix.
When evaluating a proposed stationary distribution, check its normalization and nonnegativity, then verify invariance for every time parameter in the semigroup. When discussing a limit, state the initial law and compute or justify the limit from that law. These separate checks prevent an invariant probability from being mistaken for a universal long-time answer.
Check Your Understanding
Use the semigroup definition and row-vector convention to answer the following.
- What conditions must a probability row vector satisfy to be a stationary distribution for a continuous-time chain?
- For the symmetric two-state chain, verify directly that \((\tfrac12,\tfrac12)P(t)=(\tfrac12,\tfrac12)\).
- In the one-way chain, why is \((1,0)\) not stationary for any \(t>0\)?
- For the three-state chain, what is the limiting distribution from state \(1\), and what are the two absorbing-state probabilities?
- Why must any limiting distribution, if it exists, be invariant under every \(P(s)\)?