Sequential Three-State Transition Probabilities

Introduction

A sequential three-state model describes movement through ordered stages: a person starts in state 0, may move to state 1, and may then move to state 2. Movement is only forward. For example, a hypothetical contract model might represent an insured as active, then disabled, then dead, with no return to an earlier state. The main calculation is not just the probability of leaving state 0: it is the probability of being in each state after a specified duration, including the possibility that both transitions have occurred.

This tutorial treats a time-homogeneous continuous-time Markov chain with constant transition intensities. The earlier material on transition intensities and holding-time survival gives the basic interpretation: an intensity is an instantaneous rate, not itself a probability over a finite period. Here, the sequential structure lets us calculate finite-duration transition probabilities directly.

Definition: Consider states \(0,1,2\), with transitions permitted only from \(0\) to \(1\) and from \(1\) to \(2\). Let the constant intensity of \(0\to1\) be \(\alpha\geq0\), and the constant intensity of \(1\to2\) be \(\beta\geq0\). State 2 has no outgoing transition. Define \(p_{ij}(t)=P(X(t)=j\mid X(0)=i)\), the probability of being in state \(j\) after duration \(t\), given a start in state \(i\).

The model assumes that, while in state 0, the waiting time to enter state 1 has an exponential distribution with rate \(\alpha\). While in state 1, the waiting time to enter state 2 has an exponential distribution with rate \(\beta\). The Markov assumption means that future transitions depend on the current state, not on the time already spent there. The intensities are constant over the period considered. In particular, this model does not represent recovery, a direct transition from state 0 to state 2, or duration-dependent rates.

Transition Probabilities Over a Fixed Duration

Starting in state 0, the probability of remaining there for all of duration \(t\) is the no-jump probability \(e^{-\alpha t}\). Starting in state 1, the corresponding probability is \(e^{-\beta t}\). State 2 is absorbing, so once entered, it is occupied thereafter.

p_{00}(t)=e^{-\alpha t},\qquad p_{11}(t)=e^{-\beta t},\qquad p_{22}(t)=1.

To be in state 1 at time \(t\), a person who starts in state 0 must make exactly one transition, from 0 to 1, at some time \(s\) between 0 and \(t\), and then remain in state 1 until time \(t\). The probability contribution from transitions in a small interval around \(s\) is approximately the probability of staying in state 0 until \(s\), multiplied by the transition intensity \(\alpha\), and by the probability of remaining in state 1 for the remaining \(t-s\) time units. Integrating over all possible transition times gives the intermediate-state probability.

p_{01}(t)=\int_0^t e^{-\alpha s}\alpha e^{-\beta(t-s)}\,ds.

For \(\alpha\ne\beta\), evaluate the integral by factoring out \(e^{-\beta t}\). For \(\alpha=\beta\), the integrand simplifies directly.

p_{01}(t)= \begin{cases} \displaystyle \frac{\alpha}{\beta-\alpha} \left(e^{-\alpha t}-e^{-\beta t}\right),&\alpha\ne\beta,\\[8pt] \alpha t e^{-\alpha t},&\alpha=\beta. \end{cases}

The probability of being in state 2 after starting in state 0 is the remaining probability after accounting for states 0 and 1. This is valid because the three states are mutually exclusive and exhaustive at time \(t\). Starting in state 1, the only possible destinations are states 1 and 2.

p_{02}(t)=1-p_{00}(t)-p_{01}(t),\qquad p_{12}(t)=1-p_{11}(t)=1-e^{-\beta t}.

These expressions give the full transition matrix. Rows correspond to starting states and columns to states occupied at time \(t\). The zero entries encode the transitions the model forbids.

\mathbf P(t)= \begin{pmatrix} e^{-\alpha t} & p_{01}(t) & 1-e^{-\alpha t}-p_{01}(t)\\ 0 & e^{-\beta t} & 1-e^{-\beta t}\\ 0 & 0 & 1 \end{pmatrix}.

Each row sums to 1. In particular, the third entry in the first row should be calculated as a complement, not by treating the two transition times as if they were independent events. The intermediate state probability already accounts for the required order of transitions.

Worked Calculations

Worked Example: Unequal Intensities and the Probability of Two Transitions

A hypothetical model has states 0, 1, and 2, with constant annual intensities \(\alpha=0.20\) for \(0\to1\) and \(\beta=0.30\) for \(1\to2\). There are no other transitions, and state 2 is absorbing. Calculate the transition probabilities after 2 years for a person starting in state 0.

Method. Use the unequal-intensity formula for \(p_{01}(t)\), then find \(p_{02}(t)\) by subtraction. The duration is 2 years, so \(\alpha t=0.40\) and \(\beta t=0.60\).

First, the probability of remaining in state 0 is \(p_{00}(2)=e^{-0.40}=0.670320046\). Next, since \(\beta-\alpha=0.10\),

\begin{aligned} p_{01}(2) &=\frac{0.20}{0.30-0.20} \left(e^{-0.40}-e^{-0.60}\right)\\ &=2(0.670320046-0.548811636)\\ &=2(0.121508410)\\ &=0.243016820. \end{aligned}

The probability of being in state 2 is therefore \(p_{02}(2)=1-0.670320046-0.243016820=0.086663134\). The complete row, rounded to four decimal places, is \((0.6703,\ 0.2430,\ 0.0867)\). As a check, its entries sum to \(0.6703+0.2430+0.0867=1.0000\). The common mistake avoided here is using an incorrect coefficient when simplifying the complement: the factor \(0.20/0.10\) is \(2\), not \(0.40\).

Worked Example: Equal Intensities

For a hypothetical sequential model, let the two constant annual intensities both be \(0.25\), with transitions only \(0\to1\to2\) and state 2 absorbing. Find the row of transition probabilities after 2 years for a person starting in state 0.

Method. Because \(\alpha=\beta\), use the equal-intensity expression rather than dividing by \(\beta-\alpha=0\). Here \(\alpha t=0.25(2)=0.50\), and \(e^{-0.50}=0.606530660\).

\begin{aligned} p_{00}(2)&=e^{-0.50}=0.606530660,\\ p_{01}(2)&=(0.25)(2)e^{-0.50}\\ &=0.50(0.606530660)=0.303265330,\\ p_{02}(2)&=1-p_{00}(2)-p_{01}(2)\\ &=1-0.606530660-0.303265330\\ &=0.090204010. \end{aligned}

Thus the row, rounded to four decimal places, is \((0.6065,\ 0.3033,\ 0.0902)\), and the rounded entries sum to 1.0000. The equal-rate formula is also consistent with the complement form \(p_{02}(t)=1-(1+\alpha t)e^{-\alpha t}\). The key trap is substituting equal rates into the unequal-rate formula, which produces a zero denominator instead of a probability.

Worked Example: Starting From a Distribution of States

A hypothetical group has starting-state probabilities \((0.60,\ 0.30,\ 0.10)\) for states 0, 1, and 2. The constant annual intensities are \(\alpha=0.40\) for \(0\to1\) and \(\beta=0.20\) for \(1\to2\); state 2 is absorbing and there are no other transitions. Find the state distribution after 1.5 years.

Method. Calculate the transition probabilities from each possible starting state, then weight each transition-matrix column by the starting probabilities. The relevant exponents are \(\alpha t=0.60\) and \(\beta t=0.30\), with \(e^{-0.60}=0.548811636\) and \(e^{-0.30}=0.740818221\).

For a start in state 0,

\begin{aligned} p_{00}(1.5)&=0.548811636,\\ p_{01}(1.5) &=\frac{0.40}{0.20-0.40} (0.548811636-0.740818221)\\ &=(-2)(-0.192006585)=0.384013170,\\ p_{02}(1.5)&=1-0.548811636-0.384013170\\ &=0.067175194. \end{aligned}

For a start in state 1, \(p_{11}(1.5)=0.740818221\) and \(p_{12}(1.5)=1-0.740818221=0.259181779\). A start in state 2 remains in state 2. The resulting probabilities are:

\begin{aligned} P(X(1.5)=0)&=0.60(0.548811636)=0.329286982,\\ P(X(1.5)=1)&=0.60(0.384013170)+0.30(0.740818221)\\ &=0.230407902+0.222245466=0.452653368,\\ P(X(1.5)=2)&=0.60(0.067175194)+0.30(0.259181779)+0.10\\ &=0.040305116+0.077754534+0.100000000\\ &=0.218059650. \end{aligned}

Rounded to four decimal places, the distribution is \((0.3293,\ 0.4527,\ 0.2181)\). The unrounded probabilities sum to \(0.329286982+0.452653368+0.218059650=1.000000000\). This check confirms that the initially existing 10% in state 2 has been retained in the final state-2 probability.

Why the Sequential Formula Works

The integral for \(p_{01}(t)\) has a direct event-based interpretation. If the transition from state 0 occurs at time \(s\), the probability of no earlier transition is \(e^{-\alpha s}\). Over a small interval of length \(ds\), the chance of the transition is approximately \(\alpha\,ds\). After entering state 1, the probability of no transition to state 2 during the remaining \(t-s\) is \(e^{-\beta(t-s)}\). Multiplying these factors and summing over all possible \(s\) yields the integral.

For unequal intensities, evaluating it gives

\begin{aligned} p_{01}(t) &=\alpha e^{-\beta t}\int_0^t e^{(\beta-\alpha)s}\,ds\\ &=\alpha e^{-\beta t} \left[\frac{e^{(\beta-\alpha)s}}{\beta-\alpha}\right]_0^t\\ &=\frac{\alpha}{\beta-\alpha} \left(e^{-\alpha t}-e^{-\beta t}\right). \end{aligned}

If \(\beta=\alpha\), the factor \(e^{(\beta-\alpha)s}\) equals 1 throughout the integral, so \(p_{01}(t)=\alpha e^{-\alpha t}\int_0^t 1\,ds=\alpha t e^{-\alpha t}\). This derivation explains why the equal-intensity case is finite even though the unequal-rate expression appears to have a zero denominator there.

Common Errors and Model Checks

A frequent error is reversing the rate difference in the denominator without also reversing the difference of exponentials. The formula may be written with either sign convention, but the two signs must match. The version used here has denominator \(\beta-\alpha\) and numerator \(e^{-\alpha t}-e^{-\beta t}\). For example, with \(\alpha=0.20,\ \beta=0.30,\ t=2\), the denominator is \(0.10\), the exponential difference is \(0.670320046-0.548811636=0.121508410\), and the result is positive. A negative transition probability signals a sign error.

Another trap is treating a transition intensity as a finite-period transition probability. In general, the probability of remaining in state 0 for duration \(t\) is \(e^{-\alpha t}\), not \(1-\alpha t\); the latter is only a short-duration approximation. Also check that the model actually permits the path being calculated. Under this sequential structure, \(p_{02}(t)\) can be positive because the path \(0\to1\to2\) is allowed, while \(p_{20}(t)\) and \(p_{10}(t)\) are zero because backward transitions are not allowed.

Key takeaway: In a one-way three-state model, calculate the probability of remaining in each transient state from its holding-time survival probability. The probability of occupying the intermediate state is obtained by integrating over the possible time of the first transition; the final-state probability follows from row-sum conservation.

Check Your Understanding

For each problem, assume constant intensities over the stated duration, transitions only \(0\to1\to2\), and an absorbing state 2. Round probabilities to four decimal places where needed.

A model has \(\alpha=0.10\) and \(\beta=0.20\) per year. For a person starting in state 0, calculate \(p_{00}(3)\), \(p_{01}(3)\), and \(p_{02}(3)\). (Answer: \(0.7408,\ 0.1920,\ 0.0672\).)
Both intensities are \(0.30\) per year. Find the transition probabilities after 2 years from state 0, and state which formula is needed for \(p_{01}(2)\). (Answer: Use the equal-intensity formula; the row is \(0.5488,\ 0.3293,\ 0.1219\).)
Let \(\alpha=0.20\) and \(\beta=0.50\) per year. Calculate the probability of being in state 2 after 2 years, given a start in state 0. (Answer: \(0.1281\).)
A group starts with probabilities \((0.25,\ 0.50,\ 0.25)\). With \(\alpha=0.40,\ \beta=0.20\) per year, find its state distribution after 1 year. (Answer: \(0.1676,\ 0.4836,\ 0.3488\).)
For \(\alpha=0.25,\ \beta=0.50\) per year and a 2-year duration, calculate \(p_{01}(2)\) and \(p_{02}(2)\) from state 0. Verify that the two probabilities, together with \(p_{00}(2)\), sum to 1. (Answer: \(p_{01}(2)=0.2387,\ p_{02}(2)=0.1548,\ p_{00}(2)=0.6065\); the sum is \(1.0000\).)

What You'll Learn