Sequential Transitions in a Three-State Chain

A Chain That Moves Only Forward

In a sequential continuous-time Markov chain, the process can move from state 1 to state 2 and then to state 3, but never backward. If state 3 is absorbing, the transient probabilities describe how probability mass moves through the two nonabsorbing states before it accumulates at state 3. This simple structure lets us solve the forward equations directly, including when the two jump rates are different or equal.

Work on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). Let \((X_t)_{t\ge0}\) be a continuous-time Markov chain with state space \(S=\{1,2,3\}\). From state 1 it jumps to state 2 at rate \(\alpha\), and from state 2 it jumps to state 3 at rate \(\beta\), where \(\alpha>0\) and \(\beta>0\). State 3 is absorbing. The generator, in the row-stochastic convention for transition probabilities, is

Each generator row sums to zero: \(-\alpha+\alpha+0=0\), \(0-\beta+\beta=0\), and \(0+0+0=0\). Write \(P(t)=(p_{ij}(t))\), where \(p_{ij}(t)=\mathbb{P}(X_t=j\mid X_0=i)\). The forward equation is \(P'(t)=P(t)Q\), with \(P(0)=I\). In particular, if the chain starts in state 1, its row of transition probabilities satisfies

The initial conditions for this row are \(p_{11}(0)=1\), \(p_{12}(0)=0\), and \(p_{13}(0)=0\). We first solve these equations, then assemble the full transition matrix.

Transition Probabilities

When \(\alpha=\beta\), the transition matrix is

Proof. Fix a starting state \(i\); its row \(r(t)=(p_{i1}(t),p_{i2}(t),p_{i3}(t))\) solves \(r'(t)=r(t)Q\). Starting from state 1, the first equation and initial condition give \(p_{11}(t)=e^{-\alpha t}\). For the second equation, multiply \(p_{12}'(t)+\beta p_{12}(t)=\alpha e^{-\alpha t}\) by \(e^{\beta t}\). The product rule gives \(\frac{d}{dt}(e^{\beta t}p_{12}(t))=\alpha e^{(\beta-\alpha)t}\). Integrating from \(0\) to \(t\), using \(p_{12}(0)=0\), yields \(e^{\beta t}p_{12}(t)=\alpha(e^{(\beta-\alpha)t}-1)/(\beta-\alpha)\) if \(\alpha\ne\beta\). Thus \(p_{12}(t)=\frac{\alpha}{\beta-\alpha}(e^{-\alpha t}-e^{-\beta t})\). If \(\alpha=\beta\), the integrand is the constant \(\alpha\), so \(e^{\alpha t}p_{12}(t)=\alpha t\), giving \(p_{12}(t)=\alpha t e^{-\alpha t}\). In either case \(p_{13}(t)=1-p_{11}(t)-p_{12}(t)\): the row sum is one because the forward equations imply \((p_{11}+p_{12}+p_{13})'=0\), and its initial value is one. This expression for \(p_{13}\) also satisfies \(p_{13}'=\beta p_{12}\) and \(p_{13}(0)=0\). Starting from state 2, the first coordinate remains zero, \(p_{22}'=-\beta p_{22}\) with \(p_{22}(0)=1\), and \(p_{23}=1-p_{22}\). Starting from state 3, absorption gives the constant row \((0,0,1)\). These scalar equations determine their solutions uniquely: each equation of the form \(y'+cy=f(t)\) becomes an equation for \((e^{ct}y)'\), which determines \(y\) from its initial value. The displayed rows therefore solve the forward equations with \(P(0)=I\). \(\square\)

For unequal rates, the formula for \(p_{12}\) can also be written as an integral: \(\int_0^t \alpha e^{-\alpha s}e^{-\beta(t-s)}\,ds\). Evaluating this integral gives the stated expression. The integrand is nonnegative, so this form also confirms \(p_{12}(t)\ge0\). In the equal-rate case its value is \(\alpha t e^{-\alpha t}\ge0\). The other entries are nonnegative as probabilities from their formulas or their row-sum complements; the examples below check them directly. Each transition row sums to one by construction.

The Equal-Rate Case

When \(\alpha=\beta\), it is important to solve the differential equation directly rather than use a formula with denominator \(\beta-\alpha\). The term \(\alpha t e^{-\alpha t}\) is the probability of being in state 2 at time \(t\) when starting in state 1. It increases initially as probability enters state 2, then declines as probability leaves state 2 for the absorbing state.

Arbitrary Starting Distributions

The transition matrix gives the distribution at time \(t\) from any initial distribution. If \(\mu=(\mu_1,\mu_2,\mu_3)\) is a row vector with nonnegative entries summing to one, then the distribution at time \(t\) is \(\mu P(t)\). This follows by conditioning on the starting state: the probability of being in state \(j\) at time \(t\) is \(\sum_i\mu_i p_{ij}(t)\).

Proof. For each \(j\in S\), partition the event \(\{X_t=j\}\) according to the initial state. The events \(\{X_0=i,X_t=j\}\) are disjoint, and \(\mathbb{P}(X_0=i,X_t=j)=\mu_i p_{ij}(t)\). Summing over \(i\) gives \(\mathbb{P}(X_t=j)=\sum_i\mu_i p_{ij}(t)\), which is the \(j\)th entry of \(\mu P(t)\). For the limit, \(e^{-\alpha t}\to0\) and \(e^{-\beta t}\to0\). When the rates differ, the formula for \(p_{12}(t)\) therefore tends to zero; when they agree, \(t e^{-\alpha t}\to0\) because \(\alpha>0\). The state-2 row also has \(p_{22}(t)=e^{-\beta t}\to0\). Every transition row consequently converges to \((0,0,1)\); multiplying by \(\mu\), whose entries sum to one, gives the same limit. \(\square\)

Common Pitfall: The Sign in the Rate Difference

A frequent error is to reverse the denominator \(\beta-\alpha\) without also reversing the exponential difference. For example, with \(\alpha=1\), \(\beta=2\), and \(t=1\), the incorrect expression \(\frac{\alpha}{\alpha-\beta}(e^{-\alpha t}-e^{-\beta t})\) gives \(-\,(e^{-1}-e^{-2})<0\), which cannot be a probability. The correct expression is \(\frac{\alpha}{\beta-\alpha}(e^{-\alpha t}-e^{-\beta t})=e^{-1}-e^{-2}>0\). A useful check is to verify both that the state-2 probability is nonnegative and that the transition row sums to one.