Closure Is a Property of a Set
Communicating classes divide a Markov chain’s state space according to which states can reach one another. Closure asks a different question: once the chain is in a particular set, can it move outside that set in one step? The distinction matters because a closed set need not be a single communicating class, and a communicating class need not be closed.
Let \((X_n)_{n\ge0}\) be a discrete-time Markov chain on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with countable state space \(S\) and row-stochastic transition matrix \(P=(p_{ij})\). Thus \(p_{ij}=\mathbb{P}(X_{n+1}=j\mid X_n=i)\), and each row sums to \(1\). Write \(P^{(n)}=P^n\) for the \(n\)-step transition matrix, where \(P^{(0)}\) is the identity matrix. For states \(i,j\), communication \(i\leftrightarrow j\) means that \(p_{ij}^{(n)}>0\) and \(p_{ji}^{(m)}>0\) for some integers \(n,m\ge0\). The communication class of \(i\) is the set of states that communicate with \(i\).
The condition is imposed on every state in \(A\), not just on a typical state or on a starting distribution. In particular, closure of a set concerns where the chain can move from each of its states. It does not require the states in the set to communicate with one another.
What Closedness Implies
Proof. We use induction on \(n\). At \(n=0\), \(X_0=i\in A\), so the probability is \(1\). Suppose that \(\mathbb{P}_i(X_n\in A)=1\). By the Markov property and the definition of a closed set, conditional on \(X_n=k\in A\), the probability that \(X_{n+1}\in A\) is \(\sum_{j\in A}p_{kj}=1\). Therefore \[ \mathbb{P}_i(X_{n+1}\in A) =\sum_{k\in A}\mathbb{P}_i(X_n=k)\sum_{j\in A}p_{kj} =\sum_{k\in A}\mathbb{P}_i(X_n=k) =1. \] This proves the claim for every \(n\). \(\square\)
So the one-step definition also prevents the chain from leaving \(A\) after any finite number of steps. The next result explains how a closed set relates to the communication-class partition. It is important not to read more into it than it says: a closed set contains whole communication classes, but those classes do not have to be closed individually.
Proof. Let \(i\in A\), and suppose \(p_{ij}^{(n)}>0\) for some \(n\ge0\). By the proposition, \(\mathbb{P}_i(X_n\in A)=1\). But \(p_{ij}^{(n)}=\mathbb{P}_i(X_n=j)>0\), so \(j\) cannot be outside \(A\): otherwise the event \(X_n=j\) would give positive probability to \(X_n\notin A\). Thus every state reachable from \(i\) in a finite number of steps belongs to \(A\). If \(j\) is in the communication class of \(i\), then \(i\) reaches \(j\), so \(j\in A\). This proves that every class intersecting \(A\) lies wholly inside \(A\), and hence that \(A\) is a union of classes. If \(A\) is itself a communication class, it is a communication class and a closed set, which is exactly a closed communicating class. \(\square\)
This theorem rules out a subtle possibility: a closed set cannot contain only part of a communication class. However, it does not say that each class contained in a closed set is itself closed. A class can have transitions to another class while the larger set containing both remains closed.
Worked Examples
Worked Example: A Closed Set That Is Not a Class
Consider a chain on \(S=\{1,2,3,4\}\) with transition matrix \[ P=\begin{bmatrix} \tfrac12&\tfrac12&0&0\\ \tfrac13&\tfrac23&0&0\\ 0&0&1&0\\ \tfrac12&0&\tfrac12&0 \end{bmatrix}. \] First check the row sums: they are \(\tfrac12+\tfrac12=1\), \(\tfrac13+\tfrac23=1\), \(1=1\), and \(\tfrac12+\tfrac12=1\). Thus the rows are probability distributions.
States \(1\) and \(2\) communicate: \(p_{12}=\tfrac12>0\) and \(p_{21}=\tfrac13>0\). Neither can reach \(3\) or \(4\), so their communication class is \(\{1,2\}\). State \(3\) is absorbing, so its class is \(\{3\}\). State \(4\) communicates with itself by a zero-step path, but it cannot be reached from any other state; its class is \(\{4\}\). Hence the classes are \(\{1,2\}\), \(\{3\}\), and \(\{4\}\).
The whole set \(S\) is closed because for each state the sum over \(S\) is its row sum, which is \(1\). But \(S\) is not a communicating class: for example, state \(3\) cannot reach state \(1\). Also, the class \(\{4\}\) is not closed. From state \(4\), the probability of remaining in \(\{4\}\) after one step is \(p_{44}=0\), not \(1\); the probability of moving outside it is \(p_{41}+p_{43}=\tfrac12+\tfrac12=1\).
Check: \(\{4\}\) is a class contained in the closed set \(S\), but it is not a closed class. This confirms that being a class within a closed set does not by itself imply that the class is closed.
Worked Example: A Closed Class and a Nonclosed Proper Subset
Let \(S=\{1,2,3\}\) and \[ P=\begin{bmatrix} \tfrac12&\tfrac12&0\\ \tfrac13&\tfrac23&0\\ 0&0&1 \end{bmatrix}. \] The row sums are \(\tfrac12+\tfrac12=1\), \(\tfrac13+\tfrac23=1\), and \(1=1\), so the matrix is row-stochastic.
States \(1\) and \(2\) communicate because \(p_{12}=\tfrac12>0\) and \(p_{21}=\tfrac13>0\). Neither state can reach \(3\), so \(\{1,2\}\) is their full communication class. For state \(1\), the one-step probability of staying in the class is \(p_{11}+p_{12}=\tfrac12+\tfrac12=1\). For state \(2\), it is \(p_{21}+p_{22}=\tfrac13+\tfrac23=1\). Thus \(\{1,2\}\) is a closed communicating class.
Now consider the proper subset \(\{1\}\). Its one-step probability of staying inside is \(p_{11}=\tfrac12\), which is less than \(1\). The remaining probability is \(p_{12}=\tfrac12\), a transition outside \(\{1\}\); therefore \(\{1\}\) is not closed. It is not a communication class either, because state \(1\) communicates with state \(2\).
Check: The class \(\{1,2\}\) is closed when tested as a whole, but its proper subset \(\{1\}\) is not. The class-level test cannot be replaced by checking only one state in the class.
Worked Example: A Union of Classes That Is Not Closed
Take \(S=\{1,2,3\}\) and \[ P=\begin{bmatrix} 1&0&0\\ 0&1&0\\ \tfrac12&\tfrac12&0 \end{bmatrix}. \] The row sums are \(1=1\), \(1=1\), and \(\tfrac12+\tfrac12=1\). States \(1\) and \(2\) are absorbing. State \(3\) can move to either of them, but neither can reach \(3\). Thus the communication classes are the three singletons \(\{1\}\), \(\{2\}\), and \(\{3\}\).
Let \(A=\{1,3\}\), which is a union of two communication classes. For state \(1\), the one-step probability of remaining in \(A\) is \(p_{11}+p_{13}=1+0=1\). For state \(3\), it is \(p_{31}+p_{33}=\tfrac12+0=\tfrac12\). Since \(\tfrac12\ne1\), \(A\) is not closed: from state \(3\), the chain moves outside \(A\) to state \(2\) with probability \(p_{32}=\tfrac12\).
Check: Being a union of whole communication classes is necessary for a set to be closed, by the theorem, but it is not sufficient. The transition probabilities from every state in the set still have to be checked.
Common Mistake and Practical Use
A common mistake is to check only whether a set contains a destination of a transition, rather than whether it contains all destinations with positive probability. In the last example, one might incorrectly call \(\{1,3\}\) closed because state \(3\) moves into it with probability \(p_{31}=\tfrac12\). The correction is to sum over all states outside the set: here the outside probability from state \(3\) is \(p_{32}=\tfrac12\), so the probability of staying in the set is only \(p_{31}+p_{33}=\tfrac12\). The required value is \(1\), not merely a positive value.
For a proposed closed set \(A\), use this check state by state:
- For each \(i\in A\), add the probabilities \(p_{ij}\) over \(j\in A\).
- Confirm that each sum is exactly \(1\), equivalently that the probability of moving outside \(A\) is \(0\).
- If the set is also claimed to be a communication class, verify separately that its states communicate and that it is the full class, rather than just a communicating subset.
These distinctions help organize a chain’s long-run structure. Closed sets identify regions that cannot be left once entered, while communicating classes describe mutual reachability. A closed communicating class has both properties; an arbitrary closed set may combine several classes, including classes that are not individually closed. Conversely, a communicating class may have positive probability of moving outside itself and therefore fail to be closed.
Check Your Understanding
For each question, use the one-step definition of closedness and state which states or classes are involved.
- If \(A\) is closed and \(i\in A\), what does the definition imply about \(\mathbb{P}_i(X_1\in A)\)? What does the proposition imply for \(\mathbb{P}_i(X_n\in A)\)?
- Can a closed set contain only one state from a communication class with two or more states? Explain why or why not.
- In the first worked example, why is \(\{4\}\) a communication class but not a closed communicating class?
- In the third worked example, compute the probability of leaving \(A=\{1,3\}\) in one step when the chain starts at \(3\).
- Is a union of communication classes necessarily closed? Give the reason using the third worked example.