Tutorials › Exam ALTAM - Advanced Long-Term Actuarial Mathematics › Endpoint Probabilities Versus Ever-Visited Events

Home  /  Tutorials  /  Exam ALTAM / Transition Probabilities

Exam ALTAM · Transition Probabilities · Tutorial 65 of 999

Endpoint Probabilities Versus Ever-Visited Events

Learn to calculate endpoint and ever-visited probabilities in a continuous-time model that allows movement back and forth between states.

Beginner 11 min read

What You'll Learn

  • Define endpoint occupancy and ever-visited events over a specified time interval.
  • Calculate both probabilities in a two-state, time-homogeneous continuous-time Markov model.
  • Explain how repeated transitions make endpoint occupancy differ from the chance of any visit.
  • Handle cases where the process starts in the state of interest or in a mixture of states.
  • Identify common errors involving return transitions and the time interval for a visit.

Why Endpoint and Ever-Visited Probabilities Differ

A probability about a state must specify when occupancy is being measured. “In state B at the end of two years” is an endpoint event. “In state B at some time during the two years” is an ever-visited event. These events are not interchangeable when the process can leave B and later return to it.

For example, a hypothetical participant might move between an active state A and a temporarily impaired state B. The probability of being impaired at the end of a period counts only the participant’s state at that endpoint. The probability of ever being impaired also counts paths that enter B and then return to A before the endpoint. The calculation must therefore match the event being asked about.

Definition: Let \(X(s)\) denote the state occupied at time \(s\), and let the observation interval be \([0,t]\). The endpoint probability of B, given starting state i, is \(P(X(t)=B\mid X(0)=i)\). The ever-visited probability of B is \(P(\text{there exists }s\in[0,t]\text{ such that }X(s)=B\mid X(0)=i)\). The ever-visited event includes occupancy at time 0 and at time t.

A Two-State Model With Possible Returns

Consider a time-homogeneous continuous-time Markov model with states A and B. While in A, the process moves to B at constant intensity \(\alpha>0\) per year. While in B, it moves to A at constant intensity \(\beta>0\) per year. There are no other states or transitions, and the process cannot make two jumps at the same instant. The parameters \(\alpha\) and \(\beta\) are transition intensities, not probabilities.

Write \(p_{ij}(t)=P(X(t)=j\mid X(0)=i)\), and let \(c=\alpha+\beta\). The transition probabilities are

$$ \begin{aligned} p_{AA}(t)&=\frac{\beta}{c}+\frac{\alpha}{c}e^{-ct}, & p_{AB}(t)&=\frac{\alpha}{c}\left(1-e^{-ct}\right),\\ p_{BA}(t)&=\frac{\beta}{c}\left(1-e^{-ct}\right), & p_{BB}(t)&=\frac{\alpha}{c}+\frac{\beta}{c}e^{-ct}. \end{aligned} $$

The row indicates the initial state and the column the endpoint state. For instance, \(p_{BA}(t)\) is the probability of being in A at time \(t\) when starting in B. Each row sums to 1, as required. The endpoint probability of B is \(p_{AB}(t)\) when starting in A and \(p_{BB}(t)\) when starting in B.

Deriving the Ever-Visited Probability From A

If the process starts in A, it visits B by time \(t\) exactly when its first transition from A to B occurs by time \(t\). Before that first transition, it remains in A. The holding time in A is exponential with rate \(\alpha\), so its probability of exceeding \(t\) is \(e^{-\alpha t}\).

1
Describe the complement.
The complement of visiting B during \([0,t]\) is making no A-to-B transition by time \(t\).
2
Use the holding-time distribution.
Under the constant-intensity assumption, the first A-to-B holding time is exponential, so the probability of no such transition by \(t\) is \(e^{-\alpha t}\).
3
Take the complement.
Thus the probability of at least one visit to B is one minus the probability of no visit.
$$ P(\text{visit B by }t\mid X(0)=A)=1-e^{-\alpha t}. $$

This formula concerns whether the first entry into B has happened, not whether the process is still in B at time \(t\). Once the process enters B, it can leave at intensity \(\beta\), and it may subsequently return. By contrast, \(p_{AB}(t)\) counts only paths that end in B.

The endpoint formula can also be justified by conditioning on a short additional interval. Let \(q(t)=p_{AB}(t)\). Over a small interval of length \(h\), a process in A enters B with probability \(\alpha h+o(h)\); a process in B leaves for A with probability \(\beta h+o(h)\). Therefore

$$ q(t+h)=q(t)(1-\beta h)+(1-q(t))\alpha h+o(h). $$

Subtract \(q(t)\), divide by \(h\), and let \(h\) decrease to zero. This gives \(q'(t)=\alpha-(\alpha+\beta)q(t)\), with \(q(0)=0\). Multiplying by \(e^{ct}\) and integrating gives

$$ \frac{d}{dt}\left(e^{ct}q(t)\right)=\alpha e^{ct}, \qquad e^{ct}q(t)=\frac{\alpha}{c}(e^{ct}-1), \qquad q(t)=\frac{\alpha}{c}(1-e^{-ct}). $$

The departure intensity \(\beta\) appears in the endpoint result because leaving B can affect the state at time \(t\). It does not appear in the ever-visited formula from A: after the first entry into B, a later departure does not undo the fact that B was visited.

Worked Comparisons

Worked Example: Starting in A With Equal Transition Intensities

In a hypothetical two-state model, the process starts in A. The A-to-B intensity is \(\alpha=0.5\) per year, the B-to-A intensity is \(\beta=0.5\) per year, and the observation period is \(t=2\) years. Find (a) the probability of being in B at the endpoint, (b) the probability of having visited B at least once, and (c) the probability of having visited B but ending in A.

Method. Use the endpoint transition probability for part (a), the first-entry complement for part (b), and subtraction for part (c). The model is time-homogeneous with only the two specified transitions.

(a) Here \(c=\alpha+\beta=1\). Starting in A, the endpoint probability is

$$ p_{AB}(2)=\frac{0.5}{1}\left(1-e^{-1(2)}\right) =0.5(1-e^{-2}) =0.5(1-0.1353353) =0.4323324. $$

(b) The probability of no A-to-B transition in two years is \(e^{-\alpha t}=e^{-0.5(2)}=e^{-1}=0.3678794\). Therefore,

$$ P(\text{visit B by 2}\mid X(0)=A)=1-e^{-1}=0.6321206. $$

(c) Ending in A and having visited B is the part of the ever-visited event not included in the endpoint-B event. Since the process starts in A, ending in B implies that B was visited. Thus

$$ P(\text{visit B and end in A}\mid X(0)=A) =0.6321206-0.4323324 =0.1997882. $$

To four decimal places, the answers are 0.4323, 0.6321, and 0.1998. The check is \(0.4323324+0.1997882=0.6321206\): the two endpoint states partition the paths that visited B. The mistake avoided is treating every path that enters B as if it must still be there at the endpoint.

Worked Example: Starting in B Does Not Make Endpoint Occupancy Certain

A hypothetical participant starts in state B. The B-to-A intensity is \(\beta=0.6\) per year, and the A-to-B intensity is \(\alpha=0.4\) per year. Calculate the probability of being in B after two years and the probability of having occupied B at some time in \([0,2]\).

Method. Starting in B makes the ever-visited event certain under the stated interval, because time 0 is included. For endpoint occupancy, use \(p_{BB}(t)\), which allows the participant to leave B and return.

The total intensity is \(c=0.4+0.6=1\). Hence

$$ p_{BB}(2) =\frac{\alpha}{c}+\frac{\beta}{c}e^{-c(2)} =0.4+0.6e^{-2} =0.4+0.6(0.1353353) =0.4812012. $$

Because \(X(0)=B\), the process has already occupied B within the interval, so the ever-visited probability is 1.0000. The endpoint probability is 0.4812, not 1.0000. As a check, the probability of being in A at the endpoint is

$$ p_{BA}(2)=\frac{0.6}{1}(1-e^{-2})=0.5187988, \qquad p_{BA}(2)+p_{BB}(2)=1.0000000. $$

The common mistake avoided here is to confuse “has been in B” with “is in B at the end.” The participant starts in B, but may transition to A and remain there through the endpoint.

Worked Example: A Visit Followed by a Return to A

A hypothetical process starts in A, with \(\alpha=0.25\) per year for A-to-B transitions and \(\beta=0.75\) per year for B-to-A transitions. Over \(t=4\) years, find the probabilities of ever visiting B, ending in B, and visiting B but ending in A. Also find the probability of ending in B conditional on having visited B.

Method. Apply the first-entry formula and the endpoint transition formula, then use the fact that ending in B from A requires a visit to B. The total intensity is \(c=1\).

The probability of ever visiting B is

$$ 1-e^{-\alpha t}=1-e^{-0.25(4)} =1-e^{-1} =0.6321206. $$

The endpoint probability is

$$ p_{AB}(4) =\frac{0.25}{1}(1-e^{-4}) =0.25(1-0.0183156) =0.2454211. $$

The probability of having visited B but ending in A is the difference:

$$ 0.6321206-0.2454211=0.3866995. $$

Since ending in B implies a visit to B for a process starting in A, the conditional probability is

$$ P(X(4)=B\mid \text{visit B by 4},X(0)=A) =\frac{0.2454211}{0.6321206} \approx0.3883. $$

To four decimal places, the probabilities are 0.6321, 0.2454, 0.3867, and 0.3883, respectively. The check is that the first-visit paths split into those ending in A and those ending in B: \(0.3866995+0.2454211=0.6321206\). The conditional probability is less than 1 because a visit does not guarantee continued occupancy.

Common Mistakes and Scope of the Formulas

The ever-visited formula \(1-e^{-\alpha t}\) has specific assumptions. It applies when the process starts in A, the only way to reach B is the direct A-to-B transition, and that transition has a constant intensity \(\alpha\). It is not a general formula for a multi-state model in which B can be reached by several routes, or for a model with time-varying intensities. In those settings, the probability of no visit must account for the full path structure and applicable transition probabilities.

  • Do not ignore possible returns when finding an endpoint probability. The transition intensity out of B affects whether the process is still there at time \(t\).
  • Do not use the endpoint formula for “ever visited.” A path can enter B and leave before the observation ends.
  • Check the starting state and interval. With \([0,t]\) as the interval, starting in B makes the ever-visited probability 1. If a question excludes time 0, it is asking a different event.
  • Keep the transition direction clear. Here \(\alpha\) is A-to-B and \(\beta\) is B-to-A; reversing these labels changes the formulas.
Key takeaway: Endpoint occupancy asks where the process is at one specified time. Ever-visited probability asks whether the path entered the state at any time in the interval. With possible returns, the latter can be substantially larger.

Check Your Understanding

For each question, assume a time-homogeneous, two-state continuous-time Markov model with only the stated A-to-B and B-to-A transitions. Give probabilities to four decimal places.

  1. A process starts in A with \(\alpha=0.3\) per year and \(\beta=0.7\) per year. Find the probability of being in B after one year and the probability of having visited B by then. (Answer: endpoint 0.1896; ever visited 0.2592)
  2. A process starts in B with \(\alpha=0.2\) per year and \(\beta=0.8\) per year. Find the probability of being in B after three years and the probability of having occupied B at least once in \([0,3]\). (Answer: endpoint 0.2398; ever visited 1.0000)
  3. A process starts in A with \(\alpha=0.4\) per year and \(\beta=0.6\) per year. At two years, find the probability of having visited B but ending in A. (Answer: 0.2048)
  4. A process starts in A with \(\alpha=0.4\) per year and \(\beta=0.6\) per year. At one year, find the probability of being in B conditional on having visited B. (Answer: 0.7670)
  5. A mixed starting population is in A with probability 0.7 and in B with probability 0.3. With \(\alpha=0.25\) per year and \(\beta=0.75\) per year, find the probability of being in B after two years and the probability of having visited B during \([0,2]\). (Answer: endpoint 0.2568; ever visited 0.5754)

Previous

Next

Back to Tutorials
Clinical Biostats