Transient States and Possible Exit Routes

Introduction

A transition intensity matrix records which state changes can occur directly, and how quickly they may occur at the current model conditions. Its positive off-diagonal entries can be read as arrows in a directed graph. Following several arrows in sequence identifies possible routes through the model, but a route through intermediate states is not itself a direct transition.

The word transient also needs care. A state with an outgoing arrow is not automatically transient in the formal Markov-chain sense: it might belong to a closed group of states that can move among themselves but never leave the group. This tutorial uses the graph and matrix together to distinguish direct exits, indirect routes, and exits from a state's communicating class.

Definition: Let \(Q=(q_{ij})\) be a valid intensity matrix, with rows and columns indexed by the states. For distinct states \(i\) and \(j\), a direct permitted transition from \(i\) to \(j\) is represented by \(q_{ij}>0\). The directed transition graph has an arrow \(i\to j\) for each such entry. An indirect route from \(i\) to \(j\) is a sequence of one or more arrows connecting them; it may pass through other states.

Reading Exit Routes from a Matrix

The row for state \(i\) shows all its direct outgoing transitions. Every off-diagonal entry in that row is nonnegative. A positive entry gives a permitted direct transition; a zero entry means there is no direct transition represented from \(i\) to that destination under the model's stated route structure. The diagonal entry is not an arrow to another state. It is the negative of the total exit intensity from the row's state.

q_{ii}=-\sum_{j\ne i}q_{ij}, \qquad \text{total exit intensity from }i=\sum_{j\ne i}q_{ij}=-q_{ii}.

A direct transition is different from a possible route. If \(q_{ij}=0\), the model does not permit a direct \(i\to j\) jump, even if there is a route from \(i\) to \(j\) through other states. Conversely, a positive \(q_{ij}\) says the direct transition is permitted; it is an intensity, not a probability over a specified period.

For a fixed route structure, a state is absorbing if it has no outgoing arrows, so its entire row is zero. The classification of transient states is more subtle. Two states communicate if each is reachable from the other by directed routes, allowing a state to communicate with itself. A communicating class is closed if there is no arrow from any state in the class to a state outside it.

Definition: In a finite-state continuous-time Markov chain with finite transition intensities, states in a communicating class that is not closed are transient. A state in a closed communicating class is recurrent. Thus a state can have an outgoing transition and still be recurrent if all its outgoing routes stay within its closed class. An absorbing state is a particular one-state closed class.

In many actuarial models with an explicitly terminal state, the intended structure has all nonterminal states eventually leading to that terminal state; those nonterminal states are transient. The graph should support that conclusion. Do not infer it merely from the fact that a state's row has a positive off-diagonal entry.

Why a Route Is Not a Direct Intensity

The intensity \(q_{ij}\) concerns a direct transition from \(i\) to \(j\) over a very short interval. In particular, it is the limit of the probability of being in \(j\) after a short interval, divided by the interval length, when starting in \(i\). A route requiring two jumps has a different small-interval order.

For a short justification, suppose the only route from \(A\) to \(C\) is through \(B\), with constant intensities \(q_{AB}=a>0\) and \(q_{BC}=b>0\) over a short interval of length \(h\), and no direct \(A\to C\) transition. If the first jump occurs near time \(s\), its contribution to the chance of then jumping from \(B\) to \(C\) before \(h\) is approximately \(a\,ds\cdot b(h-s)\). Integrating over possible first-jump times gives the leading two-jump contribution:

\int_0^h a b(h-s)\,ds =\frac{ab h^2}{2}.

Factors for remaining in the states before the jumps tend to 1 as \(h\) tends to 0, so they do not change this leading term. The route can therefore have a positive chance of completion during the interval, but that chance is of order \(h^2\), not order \(h\). Dividing by \(h\) and taking the limit gives \(q_{AC}=0\). The matrix entry records a direct jump, not every destination reachable by a sequence of jumps.

Worked Example: Finding Transient States in a Disability Model

Consider a hypothetical model with states \(A\) (active), \(D\) (disabled), and \(X\) (dead). All intensities are per year, and the model permits the routes active-to-disabled, active-to-dead, disabled-to-active, and disabled-to-dead. The death state has no exits. Construct the intensity matrix, identify the direct exits, and classify the states.

Method. List each state's direct outgoing routes, enter their intensities off the diagonal, and set each diagonal entry to the negative sum of its row's off-diagonal entries.

From \(A\), the total exit intensity is \(0.12+0.03=0.15\) per year. From \(D\), it is \(0.08+0.04=0.12\) per year. State \(X\) has no outgoing transitions, so its total exit intensity is zero. In the order \(A,D,X\), the matrix is

Q= \begin{pmatrix} -0.15 & 0.12 & 0.03\\ 0.08 & -0.12 & 0.04\\ 0 & 0 & 0 \end{pmatrix}.

The first row gives direct arrows \(A\to D\) and \(A\to X\); there is no direct \(A\to A\) jump. The second row gives \(D\to A\) and \(D\to X\). The zero third row means \(X\) is absorbing. The class \(\{A,D\}\) communicates because \(A\to D\) and \(D\to A\). It is not closed: both \(A\) and \(D\) have a permitted route to \(X\). Therefore \(A\) and \(D\) are transient, while \(X\) belongs to a closed absorbing class.

Check. The row sums are \(-0.15+0.12+0.03=0\), \(0.08-0.12+0.04=0\), and \(0+0+0=0\). The total exit intensities are \(0.15\), \(0.12\), and \(0\) per year, as required. This method avoids the mistake of treating the negative diagonal as an outgoing arrow.

Worked Example: Separating a Direct Transition from an Indirect Route

A hypothetical care model has states \(W\) (waiting), \(C\) (receiving care), \(R\) (recovered), and \(X\) (dead). Its only direct routes are \(W\to C\) at \(0.20\) per year, \(W\to X\) at \(0.01\), \(C\to R\) at \(0.10\), \(C\to X\) at \(0.03\), and \(R\to X\) at \(0.02\). State \(X\) is absorbing. Write the matrix and determine whether \(W\) has a route to \(R\).

Method. Enter only the stated direct routes, then calculate the diagonal entries from the row totals. In row \(W\), the exit total is \(0.20+0.01=0.21\). In row \(C\), it is \(0.10+0.03=0.13\). In row \(R\), it is \(0.02\). Thus, in the order \(W,C,R,X\),

Q= \begin{pmatrix} -0.21 & 0.20 & 0 & 0.01\\ 0 & -0.13 & 0.10 & 0.03\\ 0 & 0 & -0.02 & 0.02\\ 0 & 0 & 0 & 0 \end{pmatrix}.

There is no direct \(W\to R\) transition: \(q_{WR}=0\). There is, however, an indirect route \(W\to C\to R\), since both \(q_{WC}=0.20\) and \(q_{CR}=0.10\) are positive. The zero \(q_{WR}\) must remain in the matrix; replacing it with a positive entry would add a direct transition not specified by the model.

Every nonterminal state has a route to \(X\): \(R\to X\), \(C\to X\), and \(W\to X\) directly (as well as through \(C\)). The communicating classes \(\{W\}\), \(\{C\}\), and \(\{R\}\) are not closed because each has an arrow leading out; they are transient. The class \(\{X\}\) is closed and absorbing. The row sums are \(-0.21+0.20+0+0.01=0\), \(0-0.13+0.10+0.03=0\), \(0+0-0.02+0.02=0\), and \(0\), which checks the construction.

This method avoids a common trap: adding an \(W\to R\) matrix entry just because recovery is reachable from waiting. Reachability through an intermediate state belongs to the graph's routes, not to the direct-transition entry.

Worked Example: An Outgoing Arrow Does Not Guarantee Transience

A hypothetical model has states \(P,Q,T\) and intensities \(q_{PQ}=0.20\), \(q_{QP}=0.30\), and \(q_{TP}=0.10\) per year, with no other direct transitions. Construct the matrix and decide which states are transient in the formal sense.

Method. Calculate each row's total exit intensity. State \(P\) has total \(0.20\), \(Q\) has total \(0.30\), and \(T\) has total \(0.10\). Therefore, in the order \(P,Q,T\),

Q= \begin{pmatrix} -0.20 & 0.20 & 0\\ 0.30 & -0.30 & 0\\ 0.10 & 0 & -0.10 \end{pmatrix}.

States \(P\) and \(Q\) communicate: \(P\to Q\) and \(Q\to P\). Neither has an arrow to \(T\), so \(\{P,Q\}\) is a closed communicating class. In a finite-state chain, states in this closed class are recurrent. Each state has an outgoing arrow, but those arrows only move within the class.

State \(T\) has an arrow to \(P\) and no route back from \(P\) or \(Q\) to \(T\). Its communicating class \(\{T\}\) is not closed, so \(T\) is transient. The row sums are \(-0.20+0.20=0\), \(0.30-0.30=0\), and \(0.10-0.10=0\). This example avoids the mistake of labelling every nonabsorbing state transient just because its row contains a positive off-diagonal entry.

A Reliable Graph-and-Matrix Procedure

When a problem asks for permitted exit routes or transient states, use the matrix and graph in sequence. This keeps local transition information separate from eventual reachability.

Read each row as its origin state.
For row \(i\), inspect entries in columns \(j\ne i\). Do not interpret the diagonal as a transition to a state.

Draw direct arrows.
Add \(i\to j\) exactly when \(q_{ij}>0\). Keep a zero entry zero even if a multi-step route connects the states.

Find communicating classes.
Check which states are mutually reachable by following arrows. Then determine whether any arrow leaves each class.

Classify using closure.
A class with an arrow to a state outside it is not closed; in a finite-state model its states are transient. A closed class is recurrent, with an absorbing state as the special case of a one-state closed class.

This classification presumes a finite-state Markov model with finite intensities. If the problem uses a different state-space setting or defines “transient” operationally as “nonterminal,” follow its stated convention and make that convention explicit. In all cases, the matrix itself records direct transitions, not every possible sequence of future transitions.

Common Mistakes and Exam Traps

Reading the diagonal as an exit route. The diagonal is negative and balances the off-diagonal entries; it is not an arrow to another state. The exit intensity is the sum of the positive off-diagonal entries in the row.
Confusing reachability with a direct transition. A path \(A\to B\to C\) does not make \(q_{AC}\) positive. Only a permitted direct \(A\to C\) jump belongs in that matrix entry.
Calling every state with an exit transient. In a closed communicating class, states can have outgoing transitions and still be recurrent. Check whether the class, not just the individual state, has an exit.
Treating a positive intensity as a finite-period probability. An intensity has units of inverse time. It identifies a permitted direct route and its instantaneous rate; it is not itself the probability of that transition over a year.

Key takeaway: Positive off-diagonal entries identify direct arrows; sequences of arrows identify indirect routes. For a finite-state model, determine transience by checking whether a state's communicating class has a permitted route out, not merely whether the state has an outgoing arrow.

Check Your Understanding

For each problem, distinguish direct transitions from routes through intermediate states and use closed communicating classes when classifying transience.

A model has states \(M,N,Z\), with direct intensities \(q_{MN}=0.07\), \(q_{MZ}=0.03\), and \(q_{NZ}=0.02\); \(Z\) is absorbing. What is the total exit intensity from \(M\), and is \(M\to N\to Z\) a direct \(M\to Z\) route? (Answer: \(0.10\) per year; no, \(M\to Z\) is a separate direct transition, while \(M\to N\to Z\) is an indirect route.)
In the order \(U,V,W\), the only routes are \(U\to V\) at \(0.06\), \(V\to W\) at \(0.04\), and \(W\to Z\) at \(0.01\), where \(Z\) is an additional absorbing state. What is \(q_{UW}\), and is \(U\) transient? (Answer: \(q_{UW}=0\); yes, \(U\) is transient because its communicating class is not closed.)
Two states \(B,C\) have routes \(B\to C\) at \(0.12\) and \(C\to B\) at \(0.09\), with no arrows out of the pair. Are \(B\) and \(C\) transient or recurrent? (Answer: recurrent; they form a closed communicating class.)
A state \(H\) has routes to \(J\) at \(0.05\) and to absorbing state \(K\) at \(0.01\). What is its diagonal intensity entry, and what is its total exit intensity? (Answer: \(q_{HH}=-0.06\) per year; total exit intensity \(0.06\) per year.)
A finite model has states \(R,S,T\), with \(R\to S\), \(S\to R\), and \(T\to R\), all at positive finite intensities, and no other routes. Which state is transient? Explain using classes. (Answer: \(T\) is transient; \(\{R,S\}\) is a closed communicating class, so \(R\) and \(S\) are recurrent.)

What You'll Learn