Why Recurrence Is a Class Property

Why Communication Matters for Recurrence

Recurrence is defined using returns to a particular state, so it may initially seem to depend on that state alone. The Markov property reveals a stronger fact: if two states communicate, then recurrence of either one forces recurrence of the other. The key is to use a positive-probability path repeatedly, while making sure the attempts to follow it do not overlap.

Let \((X_n)_{n\ge0}\) be a discrete-time Markov chain on a finite or countable state space \(S\), defined on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with transition probabilities \(p_{ij}\). Use the filtration \(\mathcal{F}_n=\sigma(X_0,\ldots,X_n)\). Write \(\mathbb{P}_i\) for probability when \(X_0=i\), and define \(T_i^+=\inf\{n\ge1:X_n=i\}\). A state \(i\) is recurrent if \(\mathbb{P}_i(T_i^+<\infty)=1\). States \(i\) and \(j\) communicate, written \(i\leftrightarrow j\), if there is a finite positive-probability path from \(i\) to \(j\) and another from \(j\) to \(i\).

A Recurrent State Pulls Reachable States Back

We first establish a useful consequence of recurrence. For \(x\ne i\), write \(T_i=\inf\{n\ge0:X_n=i\}\), the first time the chain hits \(i\). If \(x\) can be reached from recurrent state \(i\), then the chain started at \(x\) must hit \(i\) almost surely. Otherwise, there would be a positive chance to leave \(i\) along a path to \(x\) and then never return.

Proof. Choose a path \(i=x_0,x_1,\ldots,x_r=x\) of positive probability \(c=\prod_{\ell=0}^{r-1}p_{x_\ell x_{\ell+1}}>0\), with no occurrence of \(i\) among \(x_1,\ldots,x_r\). Such a path can be chosen by removing any portion between repeated visits to \(i\). Suppose instead that \(q=\mathbb{P}_x(T_i=\infty)>0\). By the Markov property at the deterministic time \(r\), following the chosen path and then never hitting \(i\) has probability \(cq>0\). On that event, the chain started at \(i\) never returns to \(i\), contradicting \(\mathbb{P}_i(T_i^+<\infty)=1\). Hence \(q=0\), as required. \(\square\)

In particular, after any finite positive-probability path starting from \(i\), the chain returns to \(i\) almost surely if the path ends away from \(i\). This is what allows repeated attempts at a route to another state.

Proving the Class Property

Assume \(i\ne j\), \(i\) is recurrent, and \(i\) can reach \(j\). Choose a positive-probability path \(i=x_0,x_1,\ldots,x_m=j\) with no interior visit to \(i\). Its probability is \(a=\prod_{\ell=0}^{m-1}p_{x_\ell x_{\ell+1}}>0\). We show that a chain started at \(i\) eventually hits \(j\) with probability one.

An attempt begins whenever the chain is at \(i\). During the attempt, compare the chain with the chosen path, transition by transition. The attempt ends at the first deviation from the path, or at the end of the path if every transition matches. Thus each attempt is a success with conditional probability \(a\), given the history at its starting time. If it fails, it ends at a state reached by a positive-probability path from \(i\). If that endpoint is not \(i\), the lemma says the chain returns to \(i\) almost surely, so another attempt can begin. If the endpoint is \(i\), another attempt can begin immediately.

Proof of the theorem. For completeness, the conditional probability assertion at each attempt follows from the Markov property: at a starting time \(\sigma\), conditioned on \(\mathcal{F}_\sigma\) and \(X_\sigma=i\), the probability of the specified next \(m\) transitions is \(a\). For a stopping time \(\sigma\), this follows by splitting over the events \(\{\sigma=n\}\) and applying the ordinary Markov property at each deterministic \(n\). Each failed attempt ends before any later attempt begins, so the attempts do not overlap. The lemma ensures that each next attempt starts in finite time almost surely. Inductively, the probability that the first \(k\) attempts all fail is at most \((1-a)^k\): conditional on the history at each attempt's start, its failure probability is \(1-a\). Since \(0<a\le1\), \((1-a)^k\to0\). Therefore the chain started at \(i\) hits \(j\) almost surely.

Now use communication in the other direction: \(j\) can reach \(i\). The lemma gives \(\mathbb{P}_j(T_i<\infty)=1\). After that hit of \(i\), the chain hits \(j\) almost surely by the result just proved. Applying the Markov property at \(T_i\), the chain started at \(j\) therefore returns to \(j\) in finite positive time with probability one. Thus \(\mathbb{P}_j(T_j^+<\infty)=1\), so \(j\) is recurrent. If \(i=j\), the conclusion is the assumption itself. This proves the theorem in all cases. \(\square\)

Applying the theorem with the roles of \(i\) and \(j\) reversed shows that if \(j\) is recurrent, then \(i\) is recurrent as well. Hence within a communicating class either every state is recurrent or none is. This conclusion does not say that different communicating classes have the same status.

Worked Examples

What the Result Does—and Does Not—Say

The theorem is useful because it lets us classify a communicating class by studying just one convenient state. Its proof uses only positive-probability paths, recurrence, and the Markov property; it does not require a finite state space or a finite expected return time. Recurrence concerns whether a return happens eventually with probability one. It does not assert that the expected return time is finite.

The attempts in the proof must end at the first deviation, not after a fixed number of steps regardless of what happened. Fixed-length windows started at successive returns to \(i\) can overlap when the chain returns to \(i\) quickly. Treating those overlapping windows as independent trials would not justify multiplying failure probabilities. Ending an attempt at its first deviation or at path completion ensures the next attempt cannot begin before the current one has ended; the Markov property then supplies the conditional success probability at each start.

Finally, communication is essential, not a technical convenience. A path from \(i\) to \(j\) alone does not prevent the chain from leaving \(j\)'s region and never returning. The theorem applies precisely when each state can reach the other, so that the recurrent state's repeated-return behavior can be used in both directions.