Communicating Classes in Continuous Time

Access by Positive-Time Transitions

In continuous time, a chain may spend a random duration in each state before jumping. Communication is nevertheless governed by which jumps have positive rates and which finite paths those jumps allow. This viewpoint separates the structure of a chain from the speed at which it moves: changing positive holding rates can change waiting times without changing the communicating classes.

Let \((X_t)_{t\ge0}\) be a nonexplosive continuous-time Markov chain on a finite or countable state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Nonexplosive means that the chain makes only finitely many jumps in every bounded time interval, almost surely. Write \(P_{ij}(t)=\mathbb{P}_i(X_t=j)\), and let \(Q=(q_{ij})\) be its generator. Assume the rates are finite, \(q_{ij}\ge0\) for \(i\ne j\), and \(q_{ii}=-q_i\), where \(q_i=\sum_{j\ne i}q_{ij}\). Thus each generator row sums to zero. For distinct states, say that \(j\) is accessible from \(i\) if \(P_{ij}(t)>0\) for some \(t>0\). State \(i\) is accessible from itself by taking time \(0\). States \(i\) and \(j\) communicate, written \(i\leftrightarrow j\), if each is accessible from the other.

The self-loop convention at a zero-rate state makes the jump-chain transition matrix stochastic at every state. It does not claim that the CTMC actually makes a jump there: the CTMC stays in that state forever. For a positive-rate state, the diagonal generator entry \(q_{ii}<0\) is not a jump rate to itself. The jump chain records changes of state, so its diagonal entry is zero there.

Proof. First suppose there is a finite jump-chain path \(i=i_0,i_1,\ldots,i_n=j\) with positive probability. If \(i=j\), accessibility holds at time zero. Otherwise, every edge on the path has positive rate, and the path probability is the product \(\prod_{\ell=0}^{n-1}r_{i_\ell i_{\ell+1}}>0\). At each positive-rate state, the holding time is exponential with finite positive parameter, so it has positive probability to lie in any nonempty interval inside \((0,\infty)\). Choose intervals for the first \(n\) holding times whose sums are less than some fixed \(t>0\). The probability of following the specified path with holding times in those intervals is positive. If \(q_j>0\), the holding time at \(j\) has positive probability to exceed the remaining time until \(t\); if \(q_j=0\), the chain stays at \(j\). Thus \(P_{ij}(t)>0\).

Conversely, suppose \(i\ne j\) and \(P_{ij}(t)>0\) for some \(t>0\). On the event \(X_t=j\), nonexplosion implies that only finitely many jumps occurred by time \(t\). The successive states therefore form one of countably many finite paths from \(i\) to \(j\). A jump from \(a\) to \(b\ne a\) can occur only when \(q_{ab}>0\), which gives \(r_{ab}>0\). Since the event \(X_t=j\) has positive probability and is contained in the countable union of events corresponding to these paths, at least one such path has positive probability. Its jump-chain path probability is a finite product of positive transition probabilities, hence is positive. This proves both directions; applying the equivalence to \(i\) reaching \(j\) and \(j\) reaching \(i\) proves the assertion about communication classes. \(\square\)

Classes, Closure, and Absorption

Communication is an equivalence relation. Reflexivity follows from accessibility at time zero; symmetry is part of the definition of communication. For transitivity, if \(i\) communicates with \(j\) and \(j\) communicates with \(k\), the jump-chain paths from \(i\) to \(j\) and from \(j\) to \(k\) concatenate to give a positive-probability path from \(i\) to \(k\). Reversing the roles gives a path from \(k\) to \(i\). The classes are the resulting disjoint groups of states. A class need not be closed: the chain may leave it and never return.

Proposition: A communicating class is closed if and only if all its rates to states outside the class are zero. A state is absorbing if and only if \(q_i=0\); in that case its singleton is a closed communicating class.

Proof. If \(q_{ij}>0\) for some \(i\in C\) and \(j\notin C\), then \(r_{ij}=q_{ij}/q_i>0\), so the jump chain can leave \(C\) in one step. The accessibility equivalence just proved shows that \(C\) is not closed. Conversely, if every such rate is zero, no jump-chain transition from \(C\) goes outside \(C\). Every finite jump-chain path starting in \(C\) therefore remains in \(C\), so no state outside \(C\) is accessible from it; \(C\) is closed. Now suppose \(q_i=0\). All off-diagonal rates from \(i\) are nonnegative and sum to zero, so each is zero. The chain cannot leave \(i\), making it absorbing. If \(q_i>0\), its first holding time is exponential with parameter \(q_i\), and the probability that this time is at most \(t\) is \(1-e^{-q_i t}>0\) for every \(t>0\). The chain therefore has positive probability to leave \(i\) by time \(t\), so \(i\) is not absorbing. Finally, a state communicates with no distinct state if and only if it has no positive-rate path to another state. For an absorbing state this holds, and its singleton is closed by the rate characterization. \(\square\)

Closure is about whether a class can be left, not whether every state in it is absorbing. A class with several states can be closed while the chain continues to jump within it. Conversely, a singleton communicating class with positive exit rate is not closed: it is a singleton because the chain cannot return to its state after leaving, not because it is trapped there.

Worked Classifications

What the Classification Does—and Does Not—Say

Once the positive-rate paths are known, classifying the chain is a graph problem: draw an arrow \(i\to j\) when \(q_{ij}>0\), find which states are mutually reachable by finite directed paths, and then check whether any arrow leaves each resulting class. Self-loops in the jump-chain row at zero-rate states are a bookkeeping convention; they do not create an exit or make those states move in continuous time.

These structural facts do not by themselves identify a stationary distribution or determine long-run behavior. In particular, being in a closed class is not, by itself, a calculation of limiting probabilities. The classification tells which states can be reached from one another and which groups cannot be left; additional hypotheses and results are needed for conclusions about recurrence, invariant probabilities, or convergence.

Common pitfall: Do not treat the negative diagonal entry \(q_{ii}\) as a transition to \(i\), or treat a singleton class as automatically absorbing. For example, in the first worked example, state \(3\) is a singleton class but \(q_{34}=5>0\); the class is not closed, and the state is not absorbing. The relevant checks are the off-diagonal rates out of the class and, for absorption, whether \(q_i=0\).