From Infinite-Horizon Rewards to an Elliptic Equation
For a diffusion, the generator describes the instantaneous change in expectations of smooth functions. Discounting accumulated rewards over all future times turns that generator into an elliptic operator: the relevant equation is \((\lambda-\mathcal L)u=g\), where \(\lambda>0\) is the discount rate and \(g\) is the reward rate. The resulting function \(u(x)\) is the expected total discounted reward when the process starts at \(x\).
Work on a filtered probability space \((\Omega,\mathcal{F},(\mathcal{F}_t)_{t\ge0},\mathbb{P})\) carrying a standard Brownian motion \(B_t\). Consider the one-dimensional diffusion \(dX_t=\mu(X_t)\,dt+\sigma(X_t)\,dB_t\), with \(X_0=x\). Assume \(\mu\) and \(\sigma\) are globally Lipschitz and have at most linear growth: for some finite \(C\), \(|\mu(y)|+|\sigma(y)|\le C(1+|y|)\). These conditions give a unique strong solution for each initial state \(x\). Its generator, acting on twice continuously differentiable functions, is \(\mathcal L f(y)=\mu(y)f'(y)+\tfrac12\sigma(y)^2f''(y)\).
The integral in this definition is an ordinary time integral, not an Itô integral. If \(g\) is bounded, it is absolutely integrable pathwise and in expectation, since \(\mathbb{E}_x[\int_0^\infty e^{-\lambda t}|g(X_t)|\,dt]\le \|g\|_\infty/\lambda\). When we use an Itô integral below, its integrand is progressively measurable and belongs to \(\mathcal L^2\) on every finite interval: this means \(\mathbb{E}\int_0^T H_s^2\,ds<\infty\) for every \(T<\infty\).
The Verification Formula
Proof. Fix \(T<\infty\). Apply Itô’s formula to \(e^{-\lambda t}u(X_t)\). For the function \(F(t,y)=e^{-\lambda t}u(y)\), the derivatives are \(F_t=-\lambda e^{-\lambda t}u(y)\), \(F_y=e^{-\lambda t}u'(y)\), and \(F_{yy}=e^{-\lambda t}u''(y)\). The explicit second-order correction is \(\tfrac12F_{yy}\sigma(X_t)^2\,dt\). Thus \(d(e^{-\lambda t}u(X_t))=e^{-\lambda t}(\mathcal Lu(X_t)-\lambda u(X_t))\,dt+e^{-\lambda t}\sigma(X_t)u'(X_t)\,dB_t\). The drift is \(-e^{-\lambda t}g(X_t)\,dt\) by the equation. The stochastic integrand is progressively measurable; boundedness of \(u'\) and \(\sigma(X_t)\) on a finite horizon is not guaranteed by linear growth of \(\sigma\), but the diffusion moment bound gives \(\mathbb{E}_x\int_0^T e^{-2\lambda t}\sigma(X_t)^2u'(X_t)^2\,dt<\infty\). Hence this integrand belongs to \(\mathcal L^2\), and its Itô integral is a square-integrable martingale: its increments have conditional mean zero given the earlier filtration. Integrating and taking expectations therefore gives \(\mathbb{E}_x[e^{-\lambda T}u(X_T)]-u(x)=-\mathbb{E}_x[\int_0^T e^{-\lambda t}g(X_t)\,dt]\). Since \(u\) is bounded, \(|\mathbb{E}_x[e^{-\lambda T}u(X_T)]|\le e^{-\lambda T}\|u\|_\infty\to0\). Since \(g\) is bounded, the finite-horizon reward integrals converge in expectation to the infinite-horizon integral, with tail bounded by \(\|g\|_\infty e^{-\lambda T}/\lambda\). Taking \(T\to\infty\) proves the identity. \(\square\)
There is also a converse under suitable semigroup regularity. Write \(P_tg(x)=\mathbb{E}_x[g(X_t)]\). If \(u(x)=\int_0^\infty e^{-\lambda t}P_tg(x)\,dt\), the following conditions suffice: \(P_tg\) is twice continuously differentiable in \(x\), \(\partial_tP_tg=\mathcal L P_tg\), differentiation and \(\mathcal L\) may be passed through the discounted integral, and \(e^{-\lambda t}P_tg(x)\to0\) as \(t\to\infty\). These are regularity and integrability assumptions, not automatic consequences of merely having a diffusion.
Proof. By the assumptions allowing the generator to pass under the integral, \(\mathcal Lu(x)=\int_0^\infty e^{-\lambda t}\mathcal L P_tg(x)\,dt=\int_0^\infty e^{-\lambda t}\partial_tP_tg(x)\,dt\). Integrating by parts on \([0,T]\) gives \(\int_0^T e^{-\lambda t}\partial_tP_tg(x)\,dt=e^{-\lambda T}P_Tg(x)-g(x)+\lambda\int_0^T e^{-\lambda t}P_tg(x)\,dt\), because \(P_0g(x)=g(x)\). Let \(T\to\infty\), using the assumed vanishing terminal term and convergence of the integrals. Then \(\mathcal Lu=-g+\lambda u\), which is exactly \((\lambda-\mathcal L)u=g\). \(\square\)
For bounded classical solutions covered by the verification theorem, the representation also gives uniqueness: if \(u_1\) and \(u_2\) both solve the equation with the same \(g\), their difference \(v=u_1-u_2\) solves \((\lambda-\mathcal L)v=0\). Applying the theorem with reward zero gives \(v(x)=\mathbb{E}_x[\int_0^\infty e^{-\lambda t}0\,dt]=0\).
Worked Resolvents
Worked Example: Constant Reward Under Brownian Motion
Let \(X_t=x+B_t\), so \(\mu=0\), \(\sigma=1\), and \(\mathcal Lf=\tfrac12f''\). Take the constant reward \(g(y)=c\), where \(c\) is any real constant, and \(\lambda>0\). Directly from the definition, \(R_\lambda g(x)=\mathbb{E}_x[\int_0^\infty e^{-\lambda t}c\,dt]=c\int_0^\infty e^{-\lambda t}dt=c/\lambda\). The candidate \(u(y)=c/\lambda\) has \(u'=u''=0\), so \(\mathcal Lu=0\), and \(\lambda u-\mathcal Lu=c\). Thus the elliptic equation gives the same answer. This includes \(c=0\), for which both the reward and the solution are zero.
Check: substituting the candidate gives \(\lambda(c/\lambda)-\tfrac12(0)=c\), including when \(c=0\).
Worked Example: An Oscillatory Reward for Brownian Motion
Again take \(X_t=x+B_t\), and let \(g(y)=\cos(ky)\) for a real number \(k\). Since \(B_t\) is normal with mean \(0\) and variance \(t\), its characteristic function gives \(\mathbb{E}_x[\cos(kX_t)]=e^{-k^2t/2}\cos(kx)\). Therefore \(R_\lambda g(x)=\int_0^\infty e^{-\lambda t}e^{-k^2t/2}\cos(kx)\,dt=\cos(kx)/(\lambda+k^2/2)\). For the PDE check, set \(A=1/(\lambda+k^2/2)\) and \(u(y)=A\cos(ky)\). Then \(u'(y)=-Ak\sin(ky)\) and \(u''(y)=-Ak^2\cos(ky)\). Hence \(\mathcal Lu=-Ak^2\cos(ky)/2\), and \((\lambda-\mathcal L)u=A(\lambda+k^2/2)\cos(ky)=\cos(ky)=g(y)\).
Check: at \(k=0\), the reward is identically \(1\) and the formula becomes \(R_\lambda g=1/\lambda\), matching the constant-reward calculation. The PDE substitution also reduces to \(\lambda(1/\lambda)=1\).
Worked Example: Quadratic Reward for an Ornstein–Uhlenbeck Diffusion
Consider \(dX_t=-aX_t\,dt+\sigma\,dB_t\), with \(a>0\), \(\sigma>0\), and \(X_0=x\). Its generator is \(\mathcal Lf(y)=-ayf'(y)+\tfrac12\sigma^2f''(y)\). Take \(g(y)=y^2\) and \(\lambda>0\). The explicit solution is \(X_t=e^{-at}x+\sigma\int_0^t e^{-a(t-s)}\,dB_s\). The stochastic integral has mean zero and variance \(\sigma^2\int_0^t e^{-2a(t-s)}ds=\sigma^2(1-e^{-2at})/(2a)\). Thus \(\mathbb{E}_x[X_t^2]=e^{-2at}x^2+\sigma^2(1-e^{-2at})/(2a)\). Integrating each term against \(e^{-\lambda t}\) gives \(R_\lambda g(x)=x^2/(\lambda+2a)+\frac{\sigma^2}{2a}(1/\lambda-1/(\lambda+2a))=x^2/(\lambda+2a)+\sigma^2/(\lambda(\lambda+2a))\).
To verify the equation, put \(A=1/(\lambda+2a)\) and \(C=\sigma^2/(\lambda(\lambda+2a))\), so \(u(y)=Ay^2+C\), \(u'(y)=2Ay\), and \(u''(y)=2A\). Then \(\mathcal Lu(y)=-2aAy^2+\sigma^2A\), and \((\lambda-\mathcal L)u(y)=(\lambda+2a)Ay^2+\lambda C-\sigma^2A=y^2\). Indeed \((\lambda+2a)A=1\), while \(\lambda C=\sigma^2/(\lambda+2a)=\sigma^2A\).
The bounded-solution verification theorem does not directly apply here because \(u\) and \(g\) are unbounded. The expectation identity is nevertheless valid: the displayed second moment is bounded in \(t\) by \(x^2+\sigma^2/(2a)\), so the discounted reward is integrable. The same explicit moment formula computes the resolvent directly. In an Itô verification, the stochastic integrand is \(e^{-\lambda t}\sigma u'(X_t)=2A\sigma e^{-\lambda t}X_t\); its squared expectation is integrable on every finite interval by the second-moment bound. The terminal term vanishes because \(\mathbb{E}_x[|u(X_T)|]\) is bounded uniformly in \(T\), while \(e^{-\lambda T}\to0\).
Check: at \(x=0\), the answer is \(\sigma^2/(\lambda(\lambda+2a))\). The direct integral at \(x=0\) is \(\frac{\sigma^2}{2a}(1/\lambda-1/(\lambda+2a))=\frac{\sigma^2}{2a}\frac{2a}{\lambda(\lambda+2a)}\), the same value.
Interpretation and a Common Pitfall
The operator \(\lambda-\mathcal L\) combines two effects. The term \(\lambda u\) reflects discounting at every instant; \(-\mathcal Lu\) accounts for the diffusion’s evolution. The resolvent is therefore a stationary, or elliptic, counterpart to the time-dependent Feynman–Kac representation: rather than specifying a payoff at a fixed terminal time, it accumulates a running reward over all future times with exponential discount.
Common pitfall: losing the discount or reversing the generator sign. For the constant reward \(g=1\), the correct answer is \(u=1/\lambda\), since \(\int_0^\infty e^{-\lambda t}dt=1/\lambda\). Writing \((\lambda+\mathcal L)u=g\) instead of \((\lambda-\mathcal L)u=g\) gives the wrong general equation. For example, with Brownian motion and \(g(y)=\cos(ky)\), the correct denominator is \(\lambda+k^2/2\), not \(\lambda-k^2/2\): direct integration of \(e^{-(\lambda+k^2/2)t}\) fixes both the sign and the denominator. Also, for unbounded rewards, discounting alone does not justify every step; moment and terminal-limit conditions must still be checked.
Check Your Understanding
Use the definition of the resolvent and the operator \(\mathcal L\) in each question.
- For Brownian motion and \(g(y)=\sin(ky)\), find the resolvent and verify the equation \((\lambda-\mathcal L)u=g\).
- For a bounded classical solution in the verification theorem, identify the terminal term and explain why its expectation tends to zero.
- For the Ornstein–Uhlenbeck example, compute the resolvent when \(g(y)=y\) by first finding \(\mathbb{E}_x[X_t]\).
- State the semigroup assumptions used to pass from the resolvent expectation to the elliptic equation.
- Explain why the bounded-reward estimate \(\|g\|_\infty/\lambda\) does not apply to the quadratic reward \(g(y)=y^2\), and identify the moment bound used instead.