Drifted Diffusions with Boundary Rewards and Running Costs

From Exit Rewards to a Boundary-Value ODE

Consider a diffusion moving inside a bounded interval. It earns a running cost while it remains inside, then receives a reward determined by the boundary it reaches first. The associated expectation depends on the starting point. The Feynman–Kac principle for this exit problem says that the expectation solves an ordinary differential equation (ODE): the diffusion generator acts on the unknown function, the running cost appears as a source term, and the boundary rewards become boundary values.

Work on a filtered probability space $(\Omega,\mathcal{F},(\mathcal{F}_t)_{t\ge0},\mathbb{P})$ carrying a standard Brownian motion $B_t$. Fix real constants $\mu$, $\sigma>0$, and endpoints $a<b$. Starting from $x\in(a,b)$, define the process and its first exit time by

X_t=x+\mu t+\sigma B_t,\qquad \tau=\inf\{t\ge0:X_t\notin(a,b)\}.

This is the explicit strong solution of $dX_t=\mu\,dt+\sigma\,dB_t$: it is constructed using the given Brownian motion, and the drift coefficient is $\mu$ while the diffusion coefficient is $\sigma$. All expectations below are under $\mathbb{P}$, with the starting point $x$ indicated by $\mathbb{E}_x$. Let $g_a,g_b$ be finite boundary rewards and let $c:[a,b]\to\mathbb{R}$ be a bounded continuous running cost. The total reward is $g_a$ if exit occurs at $a$, $g_b$ if it occurs at $b$, plus the accumulated cost $\int_0^\tau c(X_s)\,ds$. Since paths are continuous, $X_\tau\in\{a,b\}$ almost surely.

Definition: The exit-reward value is $$ U(x)=\mathbb{E}_x\left[g(X_\tau)+\int_0^\tau c(X_s)\,ds\right], \qquad g(a)=g_a,\quad g(b)=g_b. $$ The candidate boundary-value problem is $$ \frac{\sigma^2}{2}u''(x)+\mu u'(x)+c(x)=0\quad(a<x<b), \qquad u(a)=g_a,\quad u(b)=g_b. $$

The running cost has a plus sign in the expectation and therefore appears as $+c$ in the generator equation $\mathcal{L}u+c=0$. For example, a positive constant cost accumulates positively and gives a nonnegative expected exit-time contribution when the boundary rewards are zero. This problem uses an Ito integral; it does not use a Stratonovich integral or a change to a risk-neutral measure.

Why the Exit Time Is Integrable

To use the ODE solution at the random time $\tau$, we need control of that time. In a bounded interval, the constant-coefficient diffusion exits with a geometric tail on integer time steps. For $n\ge0$, conditional on $\mathcal{F}_n$ and on $\tau>n$, the event

B_{n+1}-B_n>\frac{b-a+|\mu|}{\sigma}

forces $X_{n+1}>b$: indeed $X_n>a$, the drift over that step is $\mu\ge-|\mu|$, and the displayed Brownian increment contributes more than $b-a+|\mu|$. By continuity, exit must then occur by time $n+1$. The increment is independent of $\mathcal{F}_n$, so this event has a fixed probability $p>0$, independent of $n$ and the position in the interval. It follows inductively that $\mathbb{P}_x(\tau>n)\le(1-p)^n$. In particular, $\mathbb{E}_x[\tau]\le\sum_{n\ge0}\mathbb{P}_x(\tau>n)<\infty$. Thus the bounded running cost has an integrable total accumulation.

The Exit Representation and Its Proof

Theorem: Suppose $u\in C^2([a,b])$ satisfies $$ \frac{\sigma^2}{2}u''+\mu u'+c=0\text{ on }(a,b), \qquad u(a)=g_a,\quad u(b)=g_b. $$ Then, for every $x\in(a,b)$, $$ u(x)=\mathbb{E}_x\left[g(X_\tau)+\int_0^\tau c(X_s)\,ds\right]. $$

Proof. Fix $t<\infty$ and apply Ito’s formula to $u(X_{t\wedge\tau})$. The function is $f(y)=u(y)$, with derivatives $f'(y)=u'(y)$ and $f''(y)=u''(y)$. Since $dX_s=\mu\,ds+\sigma\,dB_s$, the second-order correction is $\tfrac12\sigma^2u''(X_s)\,ds$; equivalently, $dB_s\,dB_s=ds$, $dB_s\,ds=0$, and $ds\,ds=0$. The resulting identity is

u(X_{t\wedge\tau})=u(x)+ \int_0^{t\wedge\tau}\left(\mu u'(X_s)+\frac{\sigma^2}{2}u''(X_s)\right)ds +\int_0^t \mathbf{1}_{\{s\le\tau\}}\sigma u'(X_s)\,dB_s.

The stochastic-integral integrand is progressively measurable and bounded on each finite time interval: $u'$ is bounded on $[a,b]$, and the indicator restricts evaluation to the stopped path. Hence it belongs to $\mathcal{L}^2$, the class of progressively measurable $H$ with $\mathbb{E}\int_0^t H_s^2\,ds<\infty$. By the Ito isometry the integral is square-integrable; the martingale property of the Ito integral gives expectation zero. Using the ODE in the time integral therefore yields

u(x)=\mathbb{E}_x\left[u(X_{t\wedge\tau})+ \int_0^{t\wedge\tau}c(X_s)\,ds\right].

As $t\to\infty$, $\tau<\infty$ almost surely, so $u(X_{t\wedge\tau})\to u(X_\tau)=g(X_\tau)$. The function $u$ is bounded on $[a,b]$, and $\left|\int_0^{t\wedge\tau}c(X_s)\,ds\right|\le\|c\|_\infty\tau$, whose expectation is finite. Dominated convergence now gives the claimed representation. $\square$

This proof verifies the representation for a classical ODE solution; it does not rely on treating a local martingale as if it were automatically a true martingale. On each finite time interval the Ito integral is square-integrable and is a true martingale, and the passage to the exit time is justified separately by boundedness and $\mathbb{E}_x[\tau]<\infty$.

Existence and Uniqueness of the ODE Solution

Proposition: For continuous $c$ on $[a,b]$, the stated boundary-value problem has exactly one $C^2$ solution.

Proof. The linear ODE has a unique solution for each specified pair of initial values $u(a)$ and $u'(a)$, since its coefficients are constant and its forcing term is continuous. First consider the homogeneous equation $\frac{\sigma^2}{2}v''+\mu v'=0$ with $v(a)=v(b)=0$. The exit representation just proved applies with zero boundary rewards and zero running cost, so $v(x)=\mathbb{E}_x[0]=0$ for every $x\in(a,b)$; continuity gives $v\equiv0$ on $[a,b]$.

Now solve the inhomogeneous initial-value problem with $u(a)=g_a$ and arbitrary slope $u'(a)=s$. Its value at $b$ depends affinely on $s$, by linearity of the ODE. The coefficient of $s$ is $v(b)$, where $v$ solves the homogeneous equation with $v(a)=0$ and $v'(a)=1$. This coefficient cannot vanish: if $v(b)=0$, the homogeneous uniqueness just proved would imply $v\equiv0$, contradicting $v'(a)=1$. Thus there is exactly one slope $s$ giving $u(b)=g_b$. This constructs the solution and proves its uniqueness. $\square$

Worked Exit Problems

Worked Example: Probability of Exiting at the Upper Boundary

Take $a=0$, $b=2$, $\mu=1$, $\sigma=1$, and rewards $g_0=0$, $g_2=1$, with $c=0$. The value is the probability of exiting at $2$, because the boundary reward is one there and zero at $0$. The ODE is $\tfrac12u''+u'=0$, with $u(0)=0$ and $u(2)=1$.

Writing $v=u'$, the equation gives $\tfrac12v'+v=0$, so $v(x)=Ce^{-2x}$. Integrating gives $u(x)=A+Be^{-2x}$. The boundary conditions give $A+B=0$ and $A+Be^{-4}=1$. Hence $B(e^{-4}-1)=1$, so $B=-1/(1-e^{-4})$ and $A=1/(1-e^{-4})$. Therefore

u(x)=\frac{1-e^{-2x}}{1-e^{-4}}.

Check: $u(0)=(1-1)/(1-e^{-4})=0$, and $u(2)=(1-e^{-4})/(1-e^{-4})=1$. Also $u'(x)=2e^{-2x}/(1-e^{-4})$ and $u''(x)=-4e^{-2x}/(1-e^{-4})$, so $\tfrac12u''+u'=0$. The representation theorem identifies this solution with the desired exit probability.

Worked Example: Expected Running Cost Until Exit

Take $a=0$, $b=1$, $\mu=0$, $\sigma=1$, zero boundary rewards, and constant running cost $c(x)=1$. The value is $\mathbb{E}_x[\tau]$. The ODE and boundary conditions are $\tfrac12u''+1=0$, $u(0)=u(1)=0$.

The ODE gives $u''=-2$. Integrating twice gives $u(x)=-x^2+A x+B$. From $u(0)=0$ we obtain $B=0$; from $u(1)=0$ we obtain $-1+A=0$, so $A=1$. Thus

$$ u(x)=x(1-x). $$

Check: $u(0)=0$, $u(1)=1-1=0$, $u'(x)=1-2x$, and $u''(x)=-2$. Substitution gives $\tfrac12(-2)+1=0$. At $x=\tfrac12$, the expected exit time is $u(\tfrac12)=\tfrac12(1-\tfrac12)=\tfrac14$, a positive value consistent with a positive running cost.

Worked Example: Drift, Unequal Boundary Rewards, and a Running Cost

Take $a=0$, $b=1$, $\mu=1$, $\sigma=1$, boundary rewards $g_0=2$, $g_1=5$, and constant running cost $c(x)=1$. The ODE is $\tfrac12u''+u'+1=0$. A particular solution is $-x$, since its first derivative is $-1$, its second derivative is $0$, and $\tfrac12(0)+(-1)+1=0$. The homogeneous solutions are $1$ and $e^{-2x}$, so $u(x)=A+Be^{-2x}-x$.

At zero, $A+B=2$. At one, $A+Be^{-2}-1=5$, or $A+Be^{-2}=6$. Subtracting the first equation from the second gives $B(e^{-2}-1)=4$. Therefore $B=-4/(1-e^{-2})$, and $A=2-B=2+4/(1-e^{-2})$. The solution is

u(x)=2+\frac{4}{1-e^{-2}}-\frac{4e^{-2x}}{1-e^{-2}}-x.

Check: at zero, the constant and exponential terms combine to $2+\frac{4-4}{1-e^{-2}}=2$. At one, they combine to $2+\frac{4-4e^{-2}}{1-e^{-2}}=2+4=6$, and subtracting one gives $5$. Also $u'=\frac{8e^{-2x}}{1-e^{-2}}-1$ and $u''=-\frac{16e^{-2x}}{1-e^{-2}}$. Consequently $\tfrac12u''+u'+1=-\frac{8e^{-2x}}{1-e^{-2}}+\frac{8e^{-2x}}{1-e^{-2}}-1+1=0$. The theorem says this value is the expected boundary reward plus the expected time-integrated unit cost.

Interpretation and a Common Pitfall

The ODE separates the problem into three ingredients: $\frac{\sigma^2}{2}u''+\mu u'$ records the local effect of diffusion and drift; $c$ records the cost accumulated per unit time; and $g_a,g_b$ specify what is paid upon exit. Setting $c=0$ recovers a boundary-reward problem such as an exit probability. Setting both boundary rewards to zero and $c=1$ gives the expected exit time. A nonzero drift changes the first-derivative term, so the driftless ODE cannot generally be reused unchanged.

Common pitfall: stopping does not by itself justify taking expectations. A tempting but incomplete argument is to apply Ito’s formula at $t\wedge\tau$, let $t\to\infty$, and discard the stochastic integral because it has mean zero. That conclusion needs justification: an unrestricted local martingale need not be a true martingale. Here, for every finite $t$, the integrand $\mathbf{1}_{\{s\le\tau\}}\sigma u'(X_s)$ is bounded and lies in $\mathcal{L}^2$, so its Ito integral is a square-integrable martingale with zero expectation. The infinite-time limit is then handled using $\mathbb{E}_x[\tau]<\infty$ and dominated convergence, not by assuming the stopped local martingale remains a martingale without conditions.

The argument is specific to a bounded interval, constant coefficients, bounded continuous running cost, and a classical solution on the closed interval. On unbounded domains or with less regular data, the needed integrability and boundary behavior require additional conditions. The exit representation is not a license to ignore those conditions.

Check Your Understanding

Use the generator equation and its boundary conditions to answer these questions.

For constant drift $\mu$, volatility $\sigma>0$, and zero running cost, write the ODE for the probability of exiting $(a,b)$ at $b$, including both boundary values.
Explain why a positive constant running cost appears with a plus sign in $\mathcal{L}u+c=0$, even though it is added in the expectation.
For zero drift, unit volatility, interval $(0,L)$, zero boundary rewards, and unit running cost, solve the ODE for the expected exit time.
In the Ito verification proof, why is the stochastic integral up to $t\wedge\tau$ a true martingale for each finite $t$?
What fails in the verification argument if one simply says that every local martingale has expectation zero?