Tutorials › Stochastic Processes › First-Step Equations for Finite Ruin Probabilities

Gambler's Ruin · Tutorial 301 of 994

First-Step Equations for Finite Ruin Probabilities

Learn how to derive and verify the first-step equations for the probability of reaching the upper boundary before ruin.

Intermediate 12 min read

What You'll Learn

  • Define the probability of reaching the upper boundary before zero
  • Set the correct probability values at both endpoints
  • Derive the interior recurrence by conditioning on the first play
  • Check transition rows and first-step equations in finite examples
  • Recognize why the direction of each transition matters

The First Play Separates the Boundary Event

In a finite gambler’s ruin game, the fortune changes by one unit at each play and the game ends when the fortune reaches either zero or a fixed upper boundary. The probability of reaching the upper boundary first depends on the first play in a simple way: after that play, the game is at one of two neighboring fortunes, and the same probability question starts again from there. This produces a recurrence, together with boundary values that encode what happens when the game starts at an endpoint.

Fix an integer \(N\ge1\) and a number \(p\in[0,1]\), and put \(q=1-p\). On a probability space \((\Omega,\mathcal{F},\mathbb{P})\), let \((X_n)_{n\ge0}\) be the discrete-time fortune process with state space \(S=\{0,1,\ldots,N\}\). From an interior state \(i\in\{1,\ldots,N-1\}\), the next fortune is \(i+1\) with probability \(p\) and \(i-1\) with probability \(q\). The endpoints 0 and \(N\) are absorbing. The process is a Markov chain, and we use the natural filtration \(\mathcal{F}_n=\sigma(X_0,\ldots,X_n)\) when describing its history.

For \(i\in S\), write \(\mathbb{P}_i\) for probabilities when \(X_0=i\). Define the boundary hitting times \(T_0=\inf\{n\ge0:X_n=0\}\) and \(T_N=\inf\{n\ge0:X_n=N\}\), with \(\inf\varnothing=\infty\). The event of interest is strictly \(T_N<T_0\): the upper boundary must be reached before zero. In particular, starting at 0 does not count as reaching \(N\) first, while starting at \(N\) does.

Definition: The finite upper-boundary hitting probability from fortune \(i\) is \(h_i=\mathbb{P}_i(T_N<T_0)\), for \(i\in S\). It satisfies \(h_0=0\) and \(h_N=1\). For an interior state \(i\), its first-step equation is \(h_i=p h_{i+1}+q h_{i-1}\).

When \(N=1\), there are no interior states, so only the two boundary values apply. The recurrence is relevant when \(N\ge2\), for each \(i=1,\ldots,N-1\). The transition matrix is row-stochastic: probability \(p\) points upward and probability \(q\) points downward, and \(p+q=1\). At an endpoint, absorption means the chain stays there with probability one.

Why the Recurrence Holds

Proposition (First-step equations): For the finite gambler’s ruin chain just defined, the probabilities \(h_i=\mathbb{P}_i(T_N<T_0)\) satisfy \(h_0=0\), \(h_N=1\), and, for every interior state \(i\), \(h_i=p h_{i+1}+q h_{i-1}\). These equations are necessary identities for the hitting probabilities; this proposition does not assert that they have a unique solution.

Proof. If the chain starts at 0, then \(T_0=0\), so the strict inequality \(T_N<T_0\) is false and \(h_0=0\). If it starts at \(N\), then \(T_N=0\) and \(T_0\ge1\), so \(T_N<T_0\) and \(h_N=1\). Now fix an interior state \(i\). The first step is to \(i+1\) with probability \(p\) and to \(i-1\) with probability \(q\). Conditional on either first-step destination, the Markov property says that the probability of subsequently reaching \(N\) before 0 is the corresponding hitting probability from that destination. The two destinations are distinct, and their probabilities sum to one. Therefore total probability over these two cases gives \(h_i=p h_{i+1}+q h_{i-1}\). This argument also applies when \(p=0\) or \(q=0\): the zero-probability branch contributes zero, without requiring conditioning on an event of probability zero. \(\square\)

The equation has no added constant. It is a probability, not an expected number of plays: after the first move, the event is evaluated from the new state, with its existing probability \(h_{i+1}\) or \(h_{i-1}\). The boundary values are essential because they tell us the probability once an endpoint has already been reached.

Worked Example: A Biased Game Starting One Unit Above Zero

Take \(N=3\), with \(p=\tfrac23\) and \(q=\tfrac13\). The state space is \(S=\{0,1,2,3\}\). In that state order, the transition matrix is

$$ P= \begin{bmatrix} 1&0&0&0\\ \tfrac13&0&\tfrac23&0\\ 0&\tfrac13&0&\tfrac23\\ 0&0&0&1 \end{bmatrix}. $$

Before using it, check the row sums: the endpoint rows each sum to \(1\), and each interior row sums to \(\tfrac13+\tfrac23=1\). From state 1, a win moves to state 2 and a loss moves to state 0. The first-step equation is therefore \(h_1=\tfrac23 h_2+\tfrac13 h_0\). Substituting the boundary value \(h_0=0\) gives \(h_1=\tfrac23h_2\). From state 2, the equation is \(h_2=\tfrac23h_3+\tfrac13h_1\), with \(h_3=1\).

Check: The coefficient of \(h_2\) in the equation for \(h_1\) is the probability of moving upward, \(\tfrac23\); the coefficient of \(h_0\) is the probability of moving downward, \(\tfrac13\). The two coefficients sum to \(\tfrac23+\tfrac13=1\), matching the first row of transition probabilities for state 1.

Worked Example: A Game with a More Likely Loss

Take \(N=4\), \(p=\tfrac14\), and \(q=\tfrac34\), with states ordered as \(0,1,2,3,4\). The transition matrix is

$$ P= \begin{bmatrix} 1&0&0&0&0\\ \tfrac34&0&\tfrac14&0&0\\ 0&\tfrac34&0&\tfrac14&0\\ 0&0&\tfrac34&0&\tfrac14\\ 0&0&0&0&1 \end{bmatrix}. $$

Check each row before using the matrix: the first and last rows sum to \(1\), and each interior row sums to \(\tfrac34+\tfrac14=1\). Starting at state 2, the first play moves to state 3 with probability \(\tfrac14\) and to state 1 with probability \(\tfrac34\). Hence \(h_2=\tfrac14h_3+\tfrac34h_1\). At state 1, the equation is \(h_1=\tfrac14h_2+\tfrac34h_0=\tfrac14h_2\). At state 3, it is \(h_3=\tfrac14h_4+\tfrac34h_2=\tfrac14+\tfrac34h_2\).

Check: In the equation for state 2, the upward destination is 3 and the downward destination is 1, exactly as the matrix row \((0,\tfrac34,0,\tfrac14,0)\) specifies. The row sum is \(\tfrac34+\tfrac14=1\). These equations describe the hitting probabilities without yet solving the system.

Worked Example: A One-Play Game

Let \(N=2\), \(p=q=\tfrac12\), and start at state 1. The state space is \(S=\{0,1,2\}\), and the transition matrix is

$$ P= \begin{bmatrix} 1&0&0\\ \tfrac12&0&\tfrac12\\ 0&0&1 \end{bmatrix}. $$

The row sums are \(1\), \(\tfrac12+\tfrac12=1\), and \(1\). From state 1, a win reaches 2 and a loss reaches 0. Thus \(h_1=\tfrac12h_2+\tfrac12h_0=\tfrac12\cdot1+\tfrac12\cdot0=\tfrac12\).

Check by listing outcomes: The first play is the only play before absorption. Exactly one of its two equally likely outcomes reaches 2 before 0, so the probability is \(\tfrac12\), as in the equation. The boundary probabilities used are \(h_2=1\) and \(h_0=0\).

Absorption and the Meaning of “Before”

For \(0<p<1\), a finite game reaches one of its endpoints almost surely. Here is a direct argument, which clarifies that the first-step equations concern a game that eventually stops. Let \(T=\min(T_0,T_N)\), the first time either endpoint is reached.

Proposition (Almost-sure absorption): For the finite gambler’s ruin chain with \(N\ge1\), fixed \(p,q\ge0\), and \(p+q=1\), the endpoint hitting time \(T\) is finite almost surely from every initial state.

Proof. From any interior state, \(N\) consecutive wins reach \(N\) within at most \(N\) plays, and \(N\) consecutive losses reach 0 within at most \(N\) plays. At least one of \(p^N\) and \(q^N\) is positive, since \(p+q=1\); set \(\delta=\max(p^N,q^N)>0\). Thus, from any interior state, the probability of absorption within the next \(N\) plays is at least \(\delta\). If the game has not stopped by time \(kN\), its state is interior. Applying the Markov property at time \(kN\) gives \(\mathbb{P}_i(T>(k+1)N)\le(1-\delta)\mathbb{P}_i(T>kN)\). Iterating this inequality yields \(\mathbb{P}_i(T>kN)\le(1-\delta)^k\), which tends to zero as \(k\to\infty\). Therefore \(\mathbb{P}_i(T=\infty)=0\). If the initial state is already an endpoint, \(T=0\), so the result holds there as well. \(\square\)

Almost-sure absorption does not change the strict event in the definition: the upper boundary must be reached first, not merely at some time. In the finite nearest-neighbor game, starting from an interior state, the chain cannot jump from one boundary to the other in a single play. The first boundary hit is therefore either 0 or \(N\), and exactly one is reached first.

Common Mistake: Reversing the Transition Weights

A frequent error is to attach \(p\) to the downward destination. In the second example, from state 2 the matrix row is \((0,\tfrac34,0,\tfrac14,0)\), so a loss moves to state 1 with probability \(\tfrac34\), while a win moves to state 3 with probability \(\tfrac14\). The incorrect equation \(h_2=\tfrac14h_1+\tfrac34h_3\) swaps those probabilities. The correct equation is \(h_2=\tfrac14h_3+\tfrac34h_1\). A reliable check is to name the destination of each outcome before writing its probability-weighted term, then verify that the coefficients sum to one.

The recurrence is useful because it converts a path-dependent event—eventually reaching one boundary before another—into local equations involving neighboring states. Its structure is determined by the game’s transition rule and boundary convention. The equations alone do not display a solved formula: the next mathematical task is to analyze their solutions, taking care to establish when the boundary-value system determines the hitting probabilities uniquely.

Check Your Understanding

For each question, keep the strict event \(T_N<T_0\) and use the stated transition probabilities.

  1. For \(N=5\), what are the boundary values \(h_0\) and \(h_5\), and what is the first-step equation at state 3 when the win probability is \(p\)?
  2. A game has \(N=4\), \(p=\tfrac25\), and \(q=\tfrac35\). Write the first-step equation for \(h_2\), identifying which destination receives each coefficient.
  3. For \(N=2\) and \(p=\tfrac34\), write the equation for \(h_1\) and substitute the two endpoint values. What probability does it give?
  4. Why is a branch with probability zero omitted by its zero coefficient rather than treated as conditioning on that branch?
  5. In the absorption argument, why is \(\max(p^N,q^N)>0\) when \(p+q=1\)?