Why State Roles Need Careful Definitions
A transition diagram can show that a person may move from one state to another, but a state’s clinical label does not by itself tell us whether that state can be left, revisited, or entered only once. Those distinctions matter when interpreting a multi-state history. For example, “death” is often represented as a state that cannot be left, while “hospitalized” may be left and later entered again. But a state that can be revisited is not necessarily one that will be revisited with probability one.
The classifications below have a structural part and a probabilistic part. The diagram identifies allowed transitions. To classify a state as transient or recurrent in the formal probabilistic sense, we also need a stochastic model that assigns probabilities to paths. We use finite-state Markov-chain terminology for the precise definitions and results; an analogous interpretation applies to a finite-state continuous-time model by considering the sequence of states visited at jumps.
Accessibility is a path property: there must be at least one possible route with positive probability. It does not mean that the route is common or that it occurs in a particular observed sample. A state diagram generally records structural possibilities; numerical probabilities or transition intensities are needed to quantify how likely different routes are.
Transient, Recurrent, and Absorbing
To define return, start in state \(i\) and ask whether the process visits \(i\) again after time zero. Let \(T_i^+=\min\{n\geq 1:X_n=i\}\), with \(T_i^+=\infty\) if there is no such visit. The probability \(P_i(T_i^+<\infty)\) is the probability of ever returning to \(i\).
The distinction between “can be revisited” and “is recurrent” is important. A path from \(i\) back to \(i\) may exist, making return possible, while another possible path leads away permanently. If that permanent departure has positive probability, the state is transient even though the diagram contains a cycle. Conversely, recurrence means return eventually occurs with probability one under the specified model; it does not mean that return happens quickly.
For a finite-state Markov chain, a useful classification rule is that every state in a closed communicating class is recurrent. States outside the closed communicating classes are transient. A singleton class is closed only if it has no positive-probability transitions out of the state; in a discrete-time chain, that means the state is absorbing. A self-transition alone does not make a state absorbing if other transitions out are possible.
A Finite-Class Result and Its Justification
The closed-class rule follows from finiteness and communication. Suppose \(C\) is a finite, closed communicating class and \(i\) is any state in \(C\). From every state \(x\in C\), there is a path to \(i\). Since \(C\) is finite, choose one such path for each \(x\), and let \(m\) be at least as large as all the chosen path lengths. Each chosen path has positive probability. The minimum of these finitely many positive path probabilities is therefore some number \(\varepsilon>0\).
Starting anywhere in \(C\), the probability of hitting \(i\) within the next \(m\) steps is at least \(\varepsilon\). If the chain has not hit \(i\) by then, the same bound applies to the next block of \(m\) steps, because the chain remains in \(C\) and the class communicates. Thus the probability of avoiding \(i\) for \(km\) steps is at most \((1-\varepsilon)^k\), which tends to zero as \(k\) grows. The probability of never hitting \(i\) is therefore zero. Starting at \(i\), after the first step the chain is still in \(C\); it either has already returned or will hit \(i\) eventually by the same argument. Hence \(i\) is recurrent.
For the converse in a finite chain, consider a state outside all closed communicating classes. There is a path from it to some closed class. A path can be chosen that does not revisit the starting state before reaching that class. That path has positive probability, and once the chain enters a closed class it cannot return to the starting state. So the probability of never returning is positive, which makes the starting state transient. This argument depends on the finite-state Markov-chain setting; a diagram without transition probabilities does not establish a numerical return probability.
Worked Example: A Cycle Does Not Guarantee Recurrence
Consider a hypothetical chain with states \(U,V,Z\). From \(U\), the next state is \(V\) with probability \(0.5\) or \(Z\) with probability \(0.5\). From \(V\), the next state is \(U\) with probability \(0.4\) or \(Z\) with probability \(0.6\). State \(Z\) is absorbing. The probabilities in each row sum to one, and the diagram includes the cycle \(U\to V\to U\).
Starting in \(U\), a return after time zero requires the first move to \(V\), followed by a move from \(V\) to \(U\). If the first move is to \(Z\), return is impossible. Therefore,
So \(U\) is transient. Starting in \(V\), a return requires \(V\to U\to V\), giving
Thus \(V\) is also transient, despite the cycle between \(U\) and \(V\). State \(Z\) is absorbing and recurrent. The positive chance of exiting to \(Z\) before returning is what prevents recurrence in \(U\) and \(V\).
Closed Classes Can Be Recurrent Without Being Absorbing
A recurrent state may have transitions out to other states, provided those states belong to the same closed communicating class and return occurs eventually with probability one. “Closed” means no transition leaves the class; it does not mean that each state in the class is fixed.
Worked Example: Recurrent States That Are Not Absorbing
Suppose two states \(A\) and \(B\) form a hypothetical closed class. From \(A\), the chain stays at \(A\) with probability \(0.3\) and moves to \(B\) with probability \(0.7\). From \(B\), it moves to \(A\) with probability \(0.4\) and stays at \(B\) with probability \(0.6\). There are no transitions to other states.
From \(B\), the probability of staying at \(B\) for \(k\) consecutive transitions is \(0.6^k\), which tends to zero. Therefore the chain eventually reaches \(A\) with probability one. Starting at \(A\), it either returns immediately through the self-transition or moves to \(B\) and eventually returns to \(A\). Hence,
For \(B\), the probability of eventually reaching \(B\) from \(A\) is also one: the chance of staying at \(A\) for \(k\) consecutive transitions is \(0.3^k\to 0\). Thus,
Both states are recurrent, but neither is absorbing: \(A\) can move to \(B\), and \(B\) can move to \(A\). This example shows why “recurrent” should not be used as a synonym for “cannot be left.”
Interpreting Roles in Clinical Diagrams
In a simple hypothetical illness-death diagram, suppose the permitted transitions are from a disease-free state to an illness state or a death state, and from illness to death; death has no outgoing transition. The death state is absorbing in this model. The other states have no route back to themselves, so their return probabilities are zero and they are transient. This classification describes the specified structure; it does not claim that every patient follows each route.
Care-process diagrams can contain cycles, such as movement between community care and hospital care. A cycle means return is possible, not certain. If a patient can also enter an absorbing endpoint, the states in the cycle may be transient. If the cycle forms a closed communicating class with no exit, its states are recurrent under a finite-state Markov model. Whether that structure is scientifically appropriate is a separate modeling question.
Worked Example: Repeated Care Settings With an Absorbing Endpoint
Consider a hypothetical care-process model with outpatient care \(O\), hospital care \(H\), post-discharge care \(R\), and death \(D\). The transition probabilities are \(O\to H:0.3\), \(O\to D:0.7\); \(H\to R:0.6\), \(H\to D:0.4\); \(R\to H:0.5\), \(R\to D:0.5\); and \(D\to D:1\). Each state’s outgoing probabilities sum to one.
There is no route back to \(O\), so \(P_O(T_O^+<\infty)=0\) and \(O\) is transient. From \(H\), the only first move is to \(R\). A return to \(H\) then occurs if the next move is \(R\to H\), so
Similarly, starting at \(R\), the chain must first move to \(H\), then move to \(R\) rather than \(D\). Therefore,
Both \(H\) and \(R\) are transient, even though they can be revisited along the cycle \(H\to R\to H\). State \(D\) is absorbing. The chance of eventual entry to \(D\) is reflected in the possibility of leaving the cycle permanently; it is not necessary to call \(H\) or \(R\) absorbing merely because a patient eventually stops being observed there.
Diagnostics, Alternatives, and Common Pitfalls
To classify states reliably, first check the diagram’s directed edges, then check whether the probabilities or rates support the conclusion. In a finite-state model, identify communicating classes and ask whether each class is closed. When a class is not closed, find a route from its states to an outside class or endpoint. If explicit return probabilities are needed, enumerate the possible routes or calculate the probability of eventually returning under the specified model.
- Cycle mistaken for recurrence. A route back proves return is possible, not that its probability is one. Check for exits that can occur before return.
- Self-transition mistaken for absorption. A state with a self-transition can still be left if it has another outgoing transition. Absorption requires that the probability of leaving is zero.
- Clinical finality mistaken for a model property. A label such as “discharged” or “end of treatment” does not guarantee absorption. The model must specify that no transitions out are allowed.
- End of observation mistaken for an absorbing state. Loss to follow-up or administrative study closure usually means the future state is unobserved, not that the person entered a permanent clinical state. Treating observation ending as absorbing changes the modeled process.
- No observed return mistaken for transience. A finite sample or limited follow-up may contain no returns even when the model’s return probability is one. Classification is about the process, not a single observed history.
The principal alternative to a probabilistic classification is a purely structural description: a state has a path back, belongs to a closed set, or has no outgoing edges. These statements can be useful when transition probabilities are not available, but they are not interchangeable with “recurrent” and “transient” as formal probability terms. A diagram can support structural conclusions; probabilistic conclusions require the transition model and its assumptions.
These categories also have limits. They summarize long-run return and exit possibilities, not the time spent in a state, the expected number of visits, or the clinical importance of a transition. A recurrent state can take a long time to revisit, while a transient state can be visited many times before permanent departure. The classifications depend on which states and transitions the model includes, so changing the diagram can change the classification.
Check Your Understanding
Answer each question using the definitions above. Solutions follow the questions.
- A state has a self-transition with probability \(0.8\) and a transition to another state with probability \(0.2\). Is it absorbing? Explain.
- In a finite chain, two states communicate and there are no transitions from either state to anywhere outside the pair. What can be concluded about their recurrence?
- A care state can be left for a terminal state, and there is also a route that eventually returns to the care state. Does this establish recurrence? Why or why not?
- Patients are no longer observed after a fixed study end date. Does that fact alone make their last recorded clinical state absorbing?
Solutions. (1) No. The probability \(0.2\) of leaving is positive, so the state is not absorbing, even though it has a self-transition. (2) The pair is a finite closed communicating class, so both states are recurrent under the finite-state Markov-chain model. Neither state must be absorbing. (3) No. A route back establishes that return is possible. Recurrence requires return probability one; an exit to the terminal state may give a positive probability of never returning. (4) No. Study end means the future is unobserved. It does not, by itself, define a clinical state with no outgoing transitions.