Two-State Models With Transitions in Both Directions

Why Two-State Models Can Return to Their Starting State

A model with two states and transitions in both directions can represent movement between, for example, active and disabled status, or between two levels of care. These are hypothetical illustrations: the model’s rates must come from the assumptions and data appropriate to the situation being studied. Unlike a one-way model, the process may leave a state and later return to it. A finite-time transition probability must therefore account for all possible paths, not only the chance of making no transition.

This tutorial considers a continuous-time Markov chain with two states, labeled 0 and 1, and a constant transition intensity in each direction. The goal is to calculate the probability of occupying either state after a specified duration and to understand the state proportions toward which the model tends over time. The results are aligned with the SOA Exam ALTAM (Advanced Long-Term Actuarial Mathematics) syllabus.

The positivity assumption means both transitions are possible. The model is time-homogeneous because the intensities do not vary with calendar time or elapsed time. Use one consistent time unit for the rates and the duration: if the rates are per year, \(t\) must be measured in years.

Use the row convention: row \(i\), column \(j\) of the transition matrix gives the probability of being in state \(j\) after time \(t\), conditional on starting in state \(i\). Write these probabilities as \(p_{ij}(t)\). Thus, each row sums to 1. If the initial distribution is the row vector \((\pi_0(0),\pi_1(0))\), the distribution at time \(t\) is obtained by multiplying it on the left by the transition matrix.

Finite-Time Transition Probabilities

Set \(k=\alpha+\beta\). The sum \(k\) determines the rate at which the model loses the effect of its starting state. The finite-time probabilities can be expressed using \(e^{-kt}\), which decreases toward zero as \(t\) increases.

One useful derivation starts with \(p_{01}(t)\), the probability of being in state 1 at time \(t\) when starting in state 0. Over a short additional interval of length \(h\), a process in state 0 moves to state 1 with probability \(\alpha h+o(h)\); a process in state 1 moves to state 0 with probability \(\beta h+o(h)\). Consequently, the forward equation is

The starting condition is \(p_{01}(0)=0\), since at duration zero a process starting in state 0 is still in state 0. Solving this linear differential equation gives

The other entries follow from the same reasoning and the row sums. Starting in state 0 gives \(p_{00}(t)=1-p_{01}(t)\). Starting in state 1, the probability \(p_{10}(t)\) solves \(p'_{10}(t)=\beta-kp_{10}(t)\) with \(p_{10}(0)=0\), so \(p_{10}(t)=\frac{\beta}{k}(1-e^{-kt})\), and \(p_{11}(t)=1-p_{10}(t)\). This derivation also explains why the answer includes a return probability: state 1 at time \(t\) does not require exactly one transition.

At \(t=0\), \(e^{-kt}=1\), so the formula gives the identity matrix. For \(t>0\), each entry is nonnegative, and the row sums are 1. For example, the first row sums to \(\frac{\beta}{k}+\frac{\alpha}{k}e^{-kt}+\frac{\alpha}{k}(1-e^{-kt})=1\). These are useful checks before applying the matrix to an initial distribution.

Applying the Matrix to an Initial Distribution

A transition matrix gives conditional probabilities for each possible starting state. To find the unconditional probability of occupying a state, weight each row by the probability of starting in that row’s state. With initial row distribution \((\pi_0(0),\pi_1(0))\), the state-1 probability at time \(t\) is

This is the law of total probability: the process must have started in either state 0 or state 1. The order of the initial probabilities must match the row order of the matrix.

Stationary Probabilities and Long-Run Interpretation

As \(t\) increases, \(e^{-kt}\) tends to zero. The transition probabilities therefore approach a matrix whose rows are identical. Its common row is the stationary distribution: if the process starts in that distribution, its distribution remains unchanged at every later time.

The formula can be justified by balancing the long-run flow between the states. In a stationary distribution, the rate of flow from state 0 to state 1, \(\pi_0\alpha\), must equal the rate of flow from state 1 to state 0, \(\pi_1\beta\). Together with \(\pi_0+\pi_1=1\), these equations give

The stationary probability of state 1 is larger when the rate into state 1 is larger relative to the total rate. It is not generally the proportion of time spent in state 1 for every finite observation period or every starting distribution. It describes the limiting distribution under this homogeneous model. Since both rates are positive, either state can be reached from the other, and the limiting distribution does not depend on the starting state.

Common Exam Trap: Reversing the State Order

The numerical values in an initial distribution are meaningful only together with their stated order. If the distribution is \((0.40,0.60)\) in the order (state 0, state 1), the probability of starting in state 1 is 0.60, not 0.40. A related error is to use \(p_{01}(t)\) for a person known to start in state 1; that conditional probability belongs to the state-0 starting row.

For the preceding mixed-start example, an incorrect calculation would be to reverse the weights:

This is wrong because it treats the initial distribution as \((0.60,0.40)\), contrary to the stated ordering. The correction is to use the state-0 starting probability 0.40 with \(p_{01}(2)\), and the state-1 starting probability 0.60 with \(p_{11}(2)\), giving \(0.7264\). Always label the states and match each initial probability to the corresponding row before multiplying.

A second check is dimensional: the exponent must be \(-(\alpha+\beta)t\), which is dimensionless. Using rates per month with time in years without conversion produces an invalid result even if the matrix rows happen to sum to 1.