Return Probabilities Through Multiple Transition Paths

Why a Same-State Endpoint Can Hide a Journey

In a continuous-time multiple-state model, an individual can start in state \(A\), leave it, and return before the time of observation. Thus, the probability of being in \(A\) at time \(t\) is generally not the probability of making no transition during \([0,t]\). That distinction matters whenever a contract benefit or state classification depends on status at a specified endpoint.

This tutorial focuses on constant transition intensities and a finite-state, time-homogeneous Markov model. The Markov assumption means that, conditional on the current state, future transition behavior does not depend on the earlier path. Intensities are fixed over the interval; simultaneous transitions have probability zero. Define \(p_{ij}(t)=P(X(t)=j\mid X(0)=i)\). The first subscript is the starting state, and the second is the endpoint state.

Definition: A return probability \(p_{ii}(t)\) is the probability that the process is in state \(i\) at time \(t\), given that it started in state \(i\). It includes paths with no transitions and paths that leave \(i\) and return. By contrast, the probability of no transition from \(i\) during the entire interval is the exponential of minus the total intensity out of \(i\), multiplied by \(t\).

For a state \(A\) with possible destinations \(B\) and \(C\), let the transition intensities from \(A\) be \(\lambda_{AB}\) and \(\lambda_{AC}\). The total intensity out of \(A\) is \(\alpha=\lambda_{AB}+\lambda_{AC}\). The probability of no transition by time \(t\) is \(e^{-\alpha t}\). If the process does leave \(A\), the first transition can follow either route, and the subsequent chance of returning depends on which state it enters.

Conditioning on the First Departure

The first departure provides a direct way to organize all paths that end in \(A\). If the first jump occurs at time \(s\) and goes to state \(j\), its density is \(\lambda_{Aj}e^{-\alpha s}\). After that jump, the probability of being back in \(A\) at time \(t\) is \(p_{jA}(t-s)\). Adding the no-departure path and the possible first-departure routes gives:

p_{AA}(t)=e^{-\alpha t} +\sum_{j\in\{B,C\}}\int_0^t \lambda_{Aj}e^{-\alpha s}p_{jA}(t-s)\,ds.

This is a conditioning identity, not a claim that the process makes only one jump after leaving \(A\). Each \(p_{jA}(t-s)\) includes every possible continuation from \(j\), including further departures and returns. The formula assumes the stated constant intensities and Markov property; if intensities vary with time, the first-jump density and subsequent transition probabilities must reflect that time dependence.

Worked Example: Return Through Either of Two Branches

Consider a hypothetical long-term care model with states \(A\), \(B\), and \(C\). From \(A\), transitions to \(B\) and \(C\) each have intensity \(1\) per year. From either \(B\) or \(C\), the only transition is back to \(A\), with intensity \(1\) per year. There are no other transitions. Find the probability of being in \(A\) after one year, starting in \(A\).

Method. Use first-departure conditioning. The total intensity out of \(A\) is \(\alpha=1+1=2\), so the probability of no departure in one year is \(e^{-2}\). To evaluate the return terms, first find \(p_{BA}(u)\), the probability of being in \(A\) after duration \(u\) when starting in \(B\).

By symmetry, \(p_{BA}(u)=p_{CA}(u)\). Write \(q(u)=p_{AA}(u)\) and \(r(u)=p_{BA}(u)\). The rate of change of \(q\) is the inflow from \(B,C\) minus the outflow from \(A\), while the rate of change of \(r\) is the inflow from \(A\) minus the outflow from \(B\):

q'(u)=-2q(u)+2r(u),\qquad r'(u)=q(u)-r(u),\qquad q(0)=1,\quad r(0)=0.

Differentiating the first equation gives \(q''=-2q'+2r'\). Since \(r'=q-r\) and \(q'=-2q+2r\), we have \(q-r=-q'/2\). Therefore \(q''=-3q'\). The initial derivative is \(q'(0)=-2\), so solving this differential equation with \(q(0)=1\) gives \(q(u)=\frac13+\frac23e^{-3u}\). Substitution into \(r'=q-r\), with \(r(0)=0\), gives \(r(u)=\frac13(1-e^{-3u})\).

Now condition on the first departure from \(A\). The two routes have equal intensity, and the return probability from either branch is \(r\):

\begin{aligned} p_{AA}(1) &=e^{-2}+\int_0^1 2e^{-2s}\,\frac{1-e^{-3(1-s)}}{3}\,ds\\ &=\frac13+\frac23e^{-3}\\ &=0.3665247122\ldots \end{aligned}

The endpoint probability is \(0.3665\) to four decimal places. As a check, the no-transition probability is only \(e^{-2}=0.1353\) to four decimal places; the larger endpoint probability includes paths that return to \(A\). The common mistake this method avoids is treating “in \(A\) at the end” as “never left \(A\).”

A Useful Exact Result for Multiple Branches

The preceding example illustrates a reusable special case. Suppose \(A\) has any finite number of possible destination states \(j\), with intensities \(\lambda_{Aj}\) and total intensity \(\alpha=\sum_j\lambda_{Aj}\). Suppose that from every such destination state the only transition is back to \(A\), at the same intensity \(\beta\). The destination labels can differ, and the intensities from \(A\) need not be equal. Because every branch has identical return behavior, the endpoint probability from any branch is the same function \(r(t)\).

Result: Under these assumptions, \(p_{AA}(t)=\frac{\beta}{\alpha+\beta}+\frac{\alpha}{\alpha+\beta}e^{-(\alpha+\beta)t}\). This result includes the possibility of multiple departures and returns. It does not apply unchanged if branch states have different return intensities or can transition to other states.

For justification, let \(q(t)=p_{AA}(t)\) and let \(r(t)\) be the probability of being in \(A\) at time \(t\) starting from any one branch. The transition rates give \(q'=-\alpha q+\alpha r\) and \(r'=\beta q-\beta r\), with \(q(0)=1\) and \(r(0)=0\). Thus \(q''=-(\alpha+\beta)q'\). The initial derivative is \(q'(0)=-\alpha\). Solving yields the stated expression, which satisfies both the initial value \(q(0)=1\) and the differential equations. The stationary level in this expression is \(\beta/(\alpha+\beta)\); it is not the probability of making no transition.

Worked Example: Separating the Contributions of Two First Routes

A hypothetical disability model has starting state \(A\) and two temporary states, \(B\) and \(C\). The intensities from \(A\) are \(\lambda_{AB}=0.25\) and \(\lambda_{AC}=0.75\) per year. From either temporary state, the only transition is to \(A\), at intensity \(\beta=2\) per year. Find the endpoint probability \(p_{AA}(2)\), and split its first-departure contributions by route.

Method. Here \(\alpha=0.25+0.75=1\). The common-branch result applies, so \(p_{AA}(2)=\frac{2}{3}+\frac{1}{3}e^{-6}\). Using \(e^{-6}=0.0024787522\ldots\), this is \(0.6674929174\ldots\), or \(0.6675\) to four decimal places.

The no-departure contribution is \(e^{-2}=0.1353352832\ldots\). Therefore the combined contribution from paths whose first departure occurs before time \(2\) is \(0.6674929174-0.1353352832=0.5321576342\), or \(0.5322\) to four decimal places. Given a departure from \(A\), the first route is \(B\) with probability \(0.25/1=0.25\), and \(C\) with probability \(0.75/1=0.75\). The respective first-route contributions are:

\begin{aligned} \text{First route }B &: 0.25(0.5321576342)=0.1330394086,\\ \text{First route }C &: 0.75(0.5321576342)=0.3991182257. \end{aligned}

Adding the route contributions and the no-departure term gives \(0.1330394086+0.3991182257+0.1353352832=0.6674929175\), with the final-digit difference due to rounding; using the unrounded values gives \(0.6674929174\ldots\). The route split is valid because the branch states have the same return behavior. A common mistake is to assign the entire return contribution to whichever branch has the larger intensity without first separating the first-departure probabilities.

Endpoint Return Versus Exactly One Excursion

A same-state endpoint probability counts paths with any number of transitions, provided the process ends in its starting state. In contrast, a probability for exactly one completed excursion from \(A\) counts only paths with two transitions: one out of \(A\), then one back to \(A\), with no further transition before the endpoint. These are different events.

Worked Example: Probability of Exactly One Completed Excursion

Use a hypothetical model with \(A\to B\) and \(A\to C\) intensities each \(1\) per year, and \(B\to A\) and \(C\to A\) intensities each \(1\) per year. Starting in \(A\), find the probability that exactly two transitions occur by time \(1\), with the second transition returning the process to \(A\).

Method. Let \(x\) be the time spent in \(A\) before the first transition, \(y\) the time in the branch state, and \(z\) the time back in \(A\). These durations satisfy \(x+y+z=1\), with all three nonnegative. The first-transition intensity summed over the two branches is \(2\); the return intensity is \(1\). The probability density contribution for durations \(x,y,z\) is \(2e^{-2x}e^{-y}e^{-2z}\). The last factor requires no further transition from \(A\) before time \(1\).

\begin{aligned} P(\text{exactly two transitions and endpoint }A) &=2e^{-2}\int_0^1(1-y)e^y\,dy\\ &=2e^{-2}(e-2)\\ &=0.1944177495\ldots \end{aligned}

Thus the requested probability is \(0.1944\) to four decimal places. To check its scale, the full endpoint probability from the first worked example is \(0.3665247122\ldots\), and the no-transition probability is \(e^{-2}=0.1353352832\ldots\). The probability of ending in \(A\) after four or more transitions is their difference after subtracting the exactly-two-transition probability: \(0.3665247122-0.1353352832-0.1944177495=0.0367716795\), or \(0.0368\) to four decimal places. The common trap is to report the exactly-two-transition probability as the total return probability, omitting paths with additional excursions.

Common Checks and Exam Traps

Do not confuse an endpoint with a path restriction. \(p_{AA}(t)\) allows any number of transitions, whereas \(e^{-\alpha t}\) describes no departure from \(A\).
Use the correct route intensity. The first-departure density for route \(A\to j\) is \(\lambda_{Aj}e^{-\alpha s}\), where \(\alpha\) is the total intensity out of \(A\), not just the intensity of that route.
Keep the remaining time in the return probability. A first jump at time \(s\) leaves duration \(t-s\), so its continuation term is \(p_{jA}(t-s)\).
Check limiting values. At \(t=0\), \(p_{AA}(0)=1\). For \(t>0\), a return probability can exceed the no-transition probability, but it must remain between zero and one.

Key takeaway: To calculate a same-state endpoint probability, separate the no-departure path from paths grouped by their first departure. Each route’s contribution combines its first-jump density with the probability of returning over the remaining interval.

Check Your Understanding

For each problem, identify whether the requested probability includes all possible later transitions or imposes a restriction on the path.

A two-state model has \(A\to B\) intensity \(0.4\) and \(B\to A\) intensity \(0.6\) per year. Find \(p_{AA}(3)\). (Answer: \(0.6199\) to four decimal places.)
From \(A\), a model has two branches with intensities \(0.5\) and \(1.0\) per year; from either branch, the only transition is back to \(A\) at intensity \(0.5\) per year. Find \(p_{AA}(2)\). (Answer: \(0.2637\) to four decimal places.)
In a model with total intensity out of \(A\) equal to \(1\) per year, the first-departure route to \(C\) has intensity \(0.6\). Both branches return to \(A\) at intensity \(2\) per year. Find the contribution to \(p_{AA}(1)\) from paths whose first departure is to \(C\). (Answer: \(0.1892\) to four decimal places.)
A model has total intensity out of \(A\) equal to \(1\) per year and return intensity \(1\) per year from either branch. Starting in \(A\), find the probability of exactly two transitions and ending in \(A\) by time \(1\). (Answer: \(e^{-1}/2=0.1839\) to four decimal places.)
From \(A\), two branch intensities are \(2\) and \(1\) per year. From either branch the only transition is back to \(A\), at intensity \(1\) per year. Find \(p_{AA}(1)\). (Answer: \(0.2637\) to four decimal places.)

What You'll Learn