Three-State Sequential Model Worked Example

Introduction

In a sequential model, movement through the states can occur only in one direction: first from state 1 to state 2, then from state 2 to state 3. This structure appears in hypothetical settings such as a contract moving from active to disabled to terminated, provided the model assumes no recovery and no direct transition from active to terminated. The transition probabilities depend on how long the process spends exposed to each transition intensity.

The key calculation is not to treat the two transitions as independent events over the whole time interval. A process must first enter state 2 before it can leave for state 3. The forward equations account for that sequence by tracking the probability flowing into and out of each state.

Model assumptions: The process is a time-homogeneous continuous-time Markov chain with states \(1,2,3\). Its only possible transitions are \(1\to2\), with constant intensity \(\lambda>0\), and \(2\to3\), with constant intensity \(\mu>0\). State 3 is absorbing. Time and both intensities must use compatible units.

Generator and Forward Equations

Use the row convention: \(p_{ij}(t)=P(X(t)=j\mid X(0)=i)\), so row \(i\) of the transition matrix describes probabilities given the initial state \(i\). The generator has an off-diagonal entry for each possible transition and a diagonal entry equal to the negative sum of the other entries in that row.

Q= \begin{pmatrix} -\lambda & \lambda & 0\\ 0 & -\mu & \mu\\ 0 & 0 & 0 \end{pmatrix}, \qquad P(0)=I, \qquad P'(t)=P(t)Q.

The forward equation \(P'(t)=P(t)Q\) gives differential equations for each row of \(P(t)\). For a process starting in state 1, write \(p_{1j}(t)\) for the probability of being in state \(j\) at time \(t\). The initial conditions are \(p_{11}(0)=1\), \(p_{12}(0)=0\), and \(p_{13}(0)=0\). Multiplying the first row of \(P(t)\) by \(Q\) gives

\begin{aligned} p_{11}'(t)&=-\lambda p_{11}(t),\\ p_{12}'(t)&=\lambda p_{11}(t)-\mu p_{12}(t),\\ p_{13}'(t)&=\mu p_{12}(t). \end{aligned}

The negative term in the second equation is the probability flow out of state 2; the positive term is flow into state 2 from state 1. State 3 has no exit flow, so its probability increases only as probability arrives from state 2.

Transition probability formulas: For \(\lambda\ne\mu\), starting in state 1, \(p_{11}(t)=e^{-\lambda t}\), \(p_{12}(t)=\dfrac{\lambda}{\mu-\lambda}(e^{-\lambda t}-e^{-\mu t})\), and \(p_{13}(t)=1-p_{11}(t)-p_{12}(t)\). For \(\lambda=\mu=r\), the corresponding formulas are \(p_{11}(t)=e^{-rt}\), \(p_{12}(t)=rt e^{-rt}\), and \(p_{13}(t)=1-(1+rt)e^{-rt}\).

Deriving the Probabilities

First solve the equation for state 1. It is a first-order equation with initial value 1, so \(p_{11}(t)=e^{-\lambda t}\). To solve for state 2 when \(\lambda\ne\mu\), substitute this result into its differential equation:

p_{12}'(t)+\mu p_{12}(t)=\lambda e^{-\lambda t}.

Multiply both sides by the integrating factor \(e^{\mu t}\). The left side becomes the derivative of \(e^{\mu t}p_{12}(t)\). Integrating from 0 to \(t\), and using \(p_{12}(0)=0\), gives

\begin{aligned} \frac{d}{dt}\left(e^{\mu t}p_{12}(t)\right)&=\lambda e^{(\mu-\lambda)t},\\ e^{\mu t}p_{12}(t)&=\int_0^t \lambda e^{(\mu-\lambda)s}\,ds =\frac{\lambda}{\mu-\lambda}\left(e^{(\mu-\lambda)t}-1\right),\\ p_{12}(t)&=\frac{\lambda}{\mu-\lambda}\left(e^{-\lambda t}-e^{-\mu t}\right). \end{aligned}

This derivation explains the sequential timing: the probability of being in state 2 at time \(t\) accumulates contributions from entry into state 2 at different earlier times, with each contribution weighted by the chance of remaining there until \(t\). The same result follows from the integral \(p_{12}(t)=\int_0^t \lambda e^{-\lambda s}e^{-\mu(t-s)}\,ds\).

The equation for state 3 can be integrated directly, but conservation of probability is simpler. Each row of \(P(t)\) sums to 1, so \(p_{13}(t)=1-p_{11}(t)-p_{12}(t)\). For an initial state of 2, the equations are \(p_{22}'=-\mu p_{22}\) and \(p_{23}'=\mu p_{22}\), with \(p_{22}(0)=1\) and \(p_{23}(0)=0\). Thus \(p_{22}(t)=e^{-\mu t}\) and \(p_{23}(t)=1-e^{-\mu t}\). Starting in state 3 leaves the process there with probability 1.

Equal Intensities

The distinct-intensity expression for \(p_{12}(t)\) has \(\mu-\lambda\) in its denominator, so it cannot be substituted into directly when \(\lambda=\mu\). Instead, solve the equation in that case. If both intensities equal \(r\), the state 2 equation is \(p_{12}'(t)+r p_{12}(t)=r e^{-rt}\). Multiplying by \(e^{rt}\) gives

\frac{d}{dt}\left(e^{rt}p_{12}(t)\right)=r.

Integrating from 0 to \(t\) yields \(e^{rt}p_{12}(t)=rt\), hence \(p_{12}(t)=rt e^{-rt}\). Conservation then gives \(p_{13}(t)=1-(1+rt)e^{-rt}\). The term \(rt e^{-rt}\) is the probability of completing the first transition and still occupying state 2 at time \(t\), not the probability of completing both transitions.

Worked Example: Unequal Intensities and a Full Transition Matrix

A hypothetical process has states 1, 2, and 3, with transitions only \(1\to2\) and \(2\to3\). The intensities are \(\lambda=1\) per year and \(\mu=2\) per year. Find the transition matrix after \(t=\ln 2\) years. Use the unequal-intensity forward-equation solution.

First evaluate the exponentials: \(e^{-\lambda t}=e^{-\ln 2}=1/2\), and \(e^{-\mu t}=e^{-2\ln 2}=1/4\). Starting in state 1, the probability of still being in state 1 is \(p_{11}=1/2\). The probability of being in state 2 is

p_{12}=\frac{1}{2-1}\left(\frac12-\frac14\right)=\frac14.

The remaining probability is in state 3: \(p_{13}=1-\frac12-\frac14=\frac14\). Starting in state 2, \(p_{22}=e^{-2\ln 2}=1/4\), and \(p_{23}=1-1/4=3/4\). Starting in state 3 gives \(p_{33}=1\). Therefore,

P(\ln 2)= \begin{pmatrix} 1/2&1/4&1/4\\ 0&1/4&3/4\\ 0&0&1 \end{pmatrix}.

Each row sums to 1, as required. The method is to solve each starting-state row using the forward equations and then use conservation for the absorbing-state probability. A common mistake is to use the second row's probability \(p_{22}\) as the probability of being in state 2 when starting in state 1; that ignores the inflow from state 1.

Worked Example: Equal Intensities

For another hypothetical sequential process, both transition intensities are \(r=0.5\) per year. Find the probabilities at \(t=2\) years, conditional on starting in state 1. Give probabilities to four decimal places.

Here \(rt=(0.5)(2)=1\), so \(e^{-rt}=e^{-1}=0.367879\ldots\). The probability of remaining in state 1 is \(p_{11}=e^{-1}=0.367879\ldots\). The equal-intensity solution gives \(p_{12}=rt e^{-rt}=1(e^{-1})=0.367879\ldots\). Then \(p_{13}=1-(1+rt)e^{-rt}=1-2e^{-1}=0.264241\ldots\).

Rounded only at the final step, the probabilities are \(p_{11}=0.3679\), \(p_{12}=0.3679\), and \(p_{13}=0.2642\). Their rounded sum is \(1.0000\), and their unrounded sum is exactly 1. The method uses the equal-intensity differential equation rather than substituting equal values into a formula with a zero denominator. The common mistake is to treat the singular-looking unequal-rate expression as an undefined model; the model remains well-defined, and the differential equation has the solution shown above.

Worked Example: Applying an Initial State Distribution

Suppose a hypothetical portfolio is initially distributed across the three states with probabilities \((0.6,0.4,0)\). The transition intensities are \(\lambda=1\) per year and \(\mu=2\) per year. Find the state probabilities after \(t=\ln 2\) years.

From the first worked example, the transition matrix at that time is \(\begin{pmatrix}1/2&1/4&1/4\\0&1/4&3/4\\0&0&1\end{pmatrix}\). Multiply the initial row distribution by this matrix. The probability of state 1 is \(0.6(1/2)+0.4(0)+0(0)=0.30\). The probability of state 2 is \(0.6(1/4)+0.4(1/4)+0(0)=0.15+0.10=0.25\). The probability of state 3 is \(0.6(1/4)+0.4(3/4)+0(1)=0.15+0.30=0.45\).

The resulting distribution is \((0.30,0.25,0.45)\), and its entries sum to 1. This method applies the transition matrix separately to each possible initial state, weighted by its initial probability. A common mistake is to apply only the first row because some members start in state 1; the initial mass in state 2 must also be propagated.

Using the Result Carefully

The transition matrix for this model, for \(\lambda\ne\mu\), is

P(t)= \begin{pmatrix} e^{-\lambda t} & \dfrac{\lambda}{\mu-\lambda}(e^{-\lambda t}-e^{-\mu t}) & 1-e^{-\lambda t}-\dfrac{\lambda}{\mu-\lambda}(e^{-\lambda t}-e^{-\mu t}) \\ 0&e^{-\mu t}&1-e^{-\mu t}\\ 0&0&1 \end{pmatrix}.

The formulas require constant intensities, the stated state sequence, and the Markov assumption. If transitions can occur in additional directions, if there is a direct transition from state 1 to state 3, or if intensities vary with time, these equations and closed forms must be changed. In particular, they should not be used as though the two transitions were competing exits from state 1.

A useful check is to verify nonnegative probabilities and row sums of 1. For the state-1 row, \(p_{11}\) and \(p_{12}\) are probabilities derived from their differential equations, while \(p_{13}=1-p_{11}-p_{12}\). The initial conditions also provide a check: at \(t=0\), the transition matrix must be the identity matrix.

Key takeaway: In a sequential three-state model, solve the state-1 equation first, use its solution as the inflow to the state-2 equation, and obtain the state-3 probability from its inflow or from conservation. When the intensities are equal, solve that case directly rather than dividing by their difference.

Check Your Understanding

For each question, assume the only transitions are \(1\to2\) and \(2\to3\), with state 3 absorbing. State intensities are constant in time.

Starting in state 1, let \(\lambda=2\) per year and \(\mu=1\) per year. Find the three state probabilities at \(t=\ln 2\) years. (Answer: \(p_{11}=0.2500,\ p_{12}=0.5000,\ p_{13}=0.2500\))
Starting in state 1, let both intensities equal 1 per year. Find the probabilities at \(t=1\) year, to four decimals. (Answer: \(p_{11}=0.3679,\ p_{12}=0.3679,\ p_{13}=0.2642\))
Starting in state 1, let \(\lambda=1\) per year, \(\mu=3\) per year, and \(t=\ln 2\) years. Find \(p_{12}(t)\) and \(p_{13}(t)\). (Answer: \(p_{12}=0.1875,\ p_{13}=0.3125\))
The initial state distribution is \((0.2,0.5,0.3)\), with \(\lambda=1\) per year, \(\mu=2\) per year, and \(t=\ln 2\) years. Find the state distribution at time \(t\). (Answer: \((0.1000,0.1750,0.7250)\))
For \(\lambda=1\) per year and \(\mu=2\) per year, starting in state 1, find the probability of being in state 2 at \(t=\ln 2\), conditional on not being in state 3 at that time. (Answer: \(1/3=0.3333\))

What You'll Learn