Hitting Probabilities From First-Step Recurrences

From Jump Rates to Boundary-Hitting Probabilities

In a birth-death process, every jump changes the state by one: from state $i$, the process moves up to $i+1$ or down to $i-1$. To find the chance of reaching an upper boundary before a lower one, it is the direction of the next jump—not the amount of time spent waiting for it—that determines the first-step equation. The jump rates nevertheless matter, because they determine the relative chances of the two directions.

Let $(X_t)_{t\ge0}$ be a continuous-time Markov chain on $S=\{0,1,\ldots,N\}$, defined on a probability space $(\Omega,\mathcal F,\mathbb P)$, with natural filtration $(\mathcal F_t)_{t\ge0}$. Fix $N\ge2$. At each interior state $i\in\{1,\ldots,N-1\}$, suppose the birth rate $\lambda_i$ and death rate $\mu_i$ are both strictly positive and finite. Thus the only possible jumps from $i$ are to $i+1$ and $i-1$, at rates $\lambda_i$ and $\mu_i$. For the hitting question, stop the process when it first reaches either endpoint; what it would do after that time does not affect which endpoint was reached first.

Definition: For $i\in S$, let $h_i=\mathbb P_i(T_N<T_0)$, where $T_a=\inf\{t\ge0:X_t=a\}$. This is the probability of hitting $N$ before $0$, starting from $i$. If neither endpoint is ever reached, the event $T_N<T_0$ does not occur. By the definition of the hitting times, $h_0=0$ and $h_N=1$.

These endpoint values are boundary conditions, not equations to be solved for unknowns. A process started at $0$ has already reached the lower boundary, so it cannot reach $N$ first; a process started at $N$ has reached the upper boundary at time zero. For interior states, condition on the first jump. The holding time at $i$ is exponential with rate $\lambda_i+\mu_i$, and the probability that the jump is upward is $\lambda_i/(\lambda_i+\mu_i)$; the downward probability is $\mu_i/(\lambda_i+\mu_i)$. For example, these jump probabilities follow by integrating the density that the upward clock rings first: $\int_0^\infty \lambda_i e^{-\lambda_i t}e^{-\mu_i t}\,dt=\lambda_i/(\lambda_i+\mu_i)$.

Theorem (First-jump recurrence and rate ratios): Under the stated assumptions, the boundary-hitting probabilities satisfy $h_0=0$, $h_N=1$, and, for $1\le i\le N-1$, $$ h_i=\frac{\lambda_i}{\lambda_i+\mu_i}h_{i+1} +\frac{\mu_i}{\lambda_i+\mu_i}h_{i-1}. $$ Equivalently, $$ \lambda_i(h_{i+1}-h_i)=\mu_i(h_i-h_{i-1}). $$ If $\Delta_i=h_i-h_{i-1}$ for $1\le i\le N$, then $$ \Delta_{i+1}=\frac{\mu_i}{\lambda_i}\Delta_i,\qquad 1\le i\le N-1. $$

Proof. Fix an interior state $i$. The first holding time is finite with probability one because it is exponential with positive rate $\lambda_i+\mu_i$. The first jump is upward with probability $\lambda_i/(\lambda_i+\mu_i)$ and downward with probability $\mu_i/(\lambda_i+\mu_i)$. Conditional on an upward first jump, the process is at $i+1$; by the Markov property, the probability of subsequently hitting $N$ before $0$ is $h_{i+1}$. Conditional on a downward first jump, that probability is $h_{i-1}$. The two first-jump possibilities are disjoint and have total probability one, so conditioning gives the displayed recurrence. Multiplying it by $\lambda_i+\mu_i$, then subtracting $(\lambda_i+\mu_i)h_i$ from both sides, gives $\lambda_i(h_{i+1}-h_i)=\mu_i(h_i-h_{i-1})$. By the definition of the increments, this is $\lambda_i\Delta_{i+1}=\mu_i\Delta_i$. Since $\lambda_i>0$, division by $\lambda_i$ gives the rate-ratio form. The boundary values follow directly from the definition of the hitting event. $\square$

The recurrence says that the probability at a state is a weighted average of the probabilities at its two neighbors. The increment version makes a different feature visible: the change in hitting probability across a step depends on the local ratio of death to birth rates. If death is more likely than birth at $i$, then $\Delta_{i+1}$ exceeds $\Delta_i$; if birth is more likely, then it is smaller. The formula is local: it relates neighboring increments without, by itself, giving a normalized solution across the entire interval.

The finite interval and positive interior rates also ensure that one of the two boundaries is eventually reached with probability one. To see why, set $r_i=\lambda_i/(\lambda_i+\mu_i)>0$ and $r=\min_{1\le i\le N-1}r_i>0$. From any interior state, the event of taking only upward jumps until reaching $N$ has probability at least $r^{N-1}>0$, since it uses at most $N-1$ jumps. Thus, in each block of $N-1$ jumps before absorption, there is a uniform positive chance to reach a boundary. The probability of avoiding both boundaries over $k$ such blocks is at most $(1-r^{N-1})^k$, which tends to zero. Every holding time before a jump is finite almost surely, so absorption occurs in finite time almost surely. This observation is useful, but the first-jump recurrence itself follows by conditioning and does not require substituting an assumption of eventual absorption into the recurrence.

Worked Recurrences

Worked Example: One Interior State

Consider $S=\{0,1,2\}$, with $0$ and $2$ absorbing for the stopped process. From state $1$, let $\lambda_1=3$ and $\mu_1=2$. Find the probability of reaching $2$ before $0$, starting from $1$.

The generator of the stopped process, in state order $0,1,2$, is $$ Q=\begin{bmatrix} 0&0&0\\ 2&-5&3\\ 0&0&0 \end{bmatrix}. $$

The row sums are $0$, $2-5+3=0$, and $0$, as required for a generator. At state $1$, the total jump rate is $3+2=5$. The upward jump probability is $3/5$, and the downward jump probability is $2/5$. The boundary values are $h_0=0$ and $h_2=1$, so the first-jump recurrence gives $$ h_1=\frac35h_2+\frac25h_0 =\frac35(1)+\frac25(0)=\frac35. $$

Check: Substituting into the recurrence gives $\frac35\cdot1+\frac25\cdot0=\frac35=h_1$. The increment form gives $\Delta_1=h_1-h_0=\frac35$ and $\Delta_2=h_2-h_1=\frac25$; indeed, $\Delta_2=(\mu_1/\lambda_1)\Delta_1=(2/3)(3/5)=2/5$.

Worked Example: Two Interior States with Unequal Rate Ratios

Let $S=\{0,1,2,3\}$, with absorbing endpoints for the stopped process. Use rates $(\lambda_1,\mu_1)=(2,1)$ and $(\lambda_2,\mu_2)=(1,3)$. Find the probabilities of hitting $3$ before $0$ from states $1$ and $2$.

The generator is $$ Q=\begin{bmatrix} 0&0&0&0\\ 1&-3&2&0\\ 0&3&-4&1\\ 0&0&0&0 \end{bmatrix}. $$

Its row sums are $0$, $1-3+2+0=0$, $0+3-4+1=0$, and $0$. With $h_0=0$ and $h_3=1$, the recurrence at state $1$ is $$ h_1=\frac23h_2+\frac13h_0=\frac23h_2. $$

At state $2$, the upward jump probability is $1/4$ and the downward jump probability is $3/4$, so $$ h_2=\frac14h_3+\frac34h_1=\frac14+\frac34h_1. $$

Substitute the first equation into the second: $h_2=\frac14+\frac34(\frac23h_2)=\frac14+\frac12h_2$. Subtracting $\frac12h_2$ gives $\frac12h_2=\frac14$, hence $h_2=\frac12$. Then $h_1=\frac23(\frac12)=\frac13$.

Check: At state $1$, $\frac23h_2+\frac13h_0=\frac23(\frac12)+0=\frac13=h_1$. At state $2$, $\frac14h_3+\frac34h_1=\frac14+\frac34(\frac13)=\frac14+\frac14=\frac12=h_2$. The increments are $\Delta_1=\frac13$, $\Delta_2=\frac12-\frac13=\frac16$, and $\Delta_3=1-\frac12=\frac12$. The rate-ratio checks are $\Delta_2=(1/2)\Delta_1=\frac16$ at state $1$, and $\Delta_3=3\Delta_2=\frac12$ at state $2$.

Worked Example: Equal Birth and Death Rates

On $S=\{0,1,2,3\}$, suppose both rates equal $1$ at states $1$ and $2$, and stop at the endpoints. Find the probability of hitting $3$ before $0$ from each interior state.

The generator is $$ Q=\begin{bmatrix} 0&0&0&0\\ 1&-2&1&0\\ 0&1&-2&1\\ 0&0&0&0 \end{bmatrix}. $$

Each row sums to zero: the interior rows give $1-2+1=0$, and the endpoint rows are zero. The two recurrence equations, using the boundary values $h_0=0$ and $h_3=1$, are $$ h_1=\frac12h_2,\qquad h_2=\frac12h_3+\frac12h_1=\frac12+\frac12h_1. $$

Substitute $h_1=\frac12h_2$ into the second equation: $h_2=\frac12+\frac14h_2$. Therefore $\frac34h_2=\frac12$, so $h_2=\frac23$, and then $h_1=\frac12(\frac23)=\frac13$.

Check: The first recurrence gives $\frac12(\frac23)=\frac13=h_1$. The second gives $\frac12(1)+\frac12(\frac13)=\frac12+\frac16=\frac23=h_2$. The increments are all equal: $\Delta_1=\frac13$, $\Delta_2=\frac23-\frac13=\frac13$, and $\Delta_3=1-\frac23=\frac13$. This agrees with $\Delta_{i+1}=(\mu_i/\lambda_i)\Delta_i=\Delta_i$ at each interior state.

What the Rate-Ratio Form Does—and Does Not—Say

The recurrence uses jump probabilities, which are rate ratios. If both rates at one state are multiplied by the same positive constant, their jump probabilities do not change. For example, changing $(\lambda_i,\mu_i)=(1,1)$ to $(5,5)$ changes the expected holding time at that state but leaves the chance of an upward jump equal to $1/2$ and the chance of a downward jump equal to $1/2$. The first-step equation for the hitting probability is therefore unchanged. More generally, multiplying both rates at each interior state by a state-dependent positive factor preserves every local birth-to-death ratio, and hence preserves the recurrence.

This does not mean that the rates are irrelevant. Changing only one of the two rates changes the relative jump probabilities. For instance, at state $2$ in the second example, the upward probability is $1/(1+3)=1/4$, not $1/2$. A larger death rate there reduces the chance of progressing toward the upper boundary. The rate-ratio relation records this effect through $\Delta_3=3\Delta_2$.

The division in the increment formula requires $\lambda_i>0$. The weighted recurrence $$ \lambda_i(h_{i+1}-h_i)=\mu_i(h_i-h_{i-1}) $$ can still be written when a rate is zero, but the ratio form may not be valid. For example, if $\lambda_i=0$ and $\mu_i>0$, the displayed equation forces $h_i=h_{i-1}$, while division by $\lambda_i$ is impossible. The positive-rate assumptions avoid these degenerate cases and ensure that either direction can be taken from every interior state. When $N=1$, there are no interior states and the boundary values alone answer the question.

Common mistake. A tempting but incorrect calculation is to take the probability of an upward jump at the starting state as the probability of ever reaching the upper endpoint. In the second example, this would give $h_1=2/(2+1)=2/3$. That is only the chance of moving from $1$ to $2$ on the first jump; it ignores the possibility of returning to $0$ or making further jumps. The correction is to condition on both possible next states and use their continuation probabilities: $h_1=(2/3)h_2+(1/3)h_0$, together with the recurrence at state $2$. Solving both equations gives $h_1=1/3$, not $2/3$.

For a finite interval, the endpoint conditions and the interior recurrence form a linear system. The examples show how to derive and check that system; in more states, the rate-ratio relation organizes how neighboring increments depend on one another. Any proposed values should satisfy both the boundary values and every interior recurrence. Checking only the first jump probability at the starting state is not enough.

Check Your Understanding

Use first-jump conditioning and verify your answers by substitution.

For a process on $\{0,1,2\}$ with $\lambda_1=1$ and $\mu_1=3$, what is the probability of hitting $2$ before $0$ from state $1$?
In a process on $\{0,1,2,3\}$, suppose $(\lambda_1,\mu_1)=(1,2)$ and $(\lambda_2,\mu_2)=(2,1)$. Write the two interior recurrence equations and the increment relations.
Why does multiplying both rates at a state by the same positive constant leave its first-jump probabilities unchanged?
For the rates in the second worked example, substitute $h_1=1/3$, $h_2=1/2$, $h_0=0$, and $h_3=1$ into both first-jump equations.
What part of the rate-ratio argument fails if a birth rate $\lambda_i$ is zero?

Hitting Probabilities From First-Step Recurrences

What You'll Learn