Running Rewards and Inhomogeneous Parabolic Equations

From a Source Term to an Accumulated Reward

A terminal payoff depends only on the state at a fixed future time. A running reward, by contrast, is earned throughout the time interval, so its contribution must accumulate along the diffusion path. The corresponding backward equation is inhomogeneous: the source term \(f\) appears in the PDE and, with the appropriate sign, as a time integral in the expectation formula. The purpose here is to derive that sign and formula under conditions that also ensure the stochastic integral has zero conditional mean.

Work on a filtered probability space \((\Omega,\mathcal{F},(\mathcal{F}_t)_{t\ge0},\mathbb{P})\) carrying a Brownian motion \(B_t\). On a finite interval \([0,T]\), consider the SDE \(dX_s=\mu(s,X_s)\,ds+\sigma(s,X_s)\,dB_s\). Here \(\mu\) is the drift coefficient and \(\sigma\) is the diffusion coefficient. Assume they are deterministic, continuous in time, globally Lipschitz in the state variable uniformly in time, and satisfy linear growth bounds. Assume the SDE has a strong solution: it is adapted to the given filtration and driven by the Brownian motion on this space. A weak solution instead allows the probability space and Brownian motion to be chosen as part of the solution. Pathwise uniqueness means two solutions driven by the same Brownian motion and with the same initial condition agree almost surely; uniqueness in law means their distributions agree. The representation below assumes a strong solution and makes no separate uniqueness claim.

For this diffusion, the generator acting on a twice differentiable function \(\varphi\) is \(\mathcal{L}_s\varphi(x)=\mu(s,x)\varphi'(x)+\tfrac12\sigma(s,x)^2\varphi''(x)\). All expectations below are under \(\mathbb{P}\); no change to a risk-neutral measure \(\mathbb{Q}\) is involved. Every stochastic integral in the derivation is an Itô integral. Its integrand must be progressively measurable and belong to \(\mathcal{L}^2\) on the interval in question, meaning \(\mathbb{E}\int_0^T H_s^2\,ds<\infty\). The multiplication rules used in Itô’s formula are \(dB_s\,dB_s=ds\), \(dB_s\,ds=0\), and \(ds\,ds=0\); they encode quadratic variation and are not the ordinary chain rule.

For a process started at \(X_t=x\), write \(\mathbb{E}_{t,x}\) for expectation under its law with that initial condition. Define \(A_{t,s}=\int_t^s c(r,X_r)\,dr\). The candidate representation is \[ u(t,x)=\mathbb{E}_{t,x}\left[e^{-A_{t,T}}g(X_T)+\int_t^T e^{-A_{t,s}}f(s,X_s)\,ds\right]. \] The exponential weights terminal data and each reward by the accumulated coefficient up to the time it is received. The following proof explains why the running term has a plus sign in this expectation, even though \(f\) appears with a plus sign in the PDE.

A Martingale Fact Needed in the Derivation

Proof. First take an elementary adapted integrand, constant on each interval of a deterministic partition with its value on that interval measurable at the left endpoint. The integral over any such interval is that measurable value multiplied by a Brownian increment independent of the past and with mean zero. Therefore its conditional expectation given the information at the left endpoint is zero. Adding intervals, and conditioning successively, gives \(\mathbb{E}[M_T\mid\mathcal{F}_t]=M_t\) at partition times. The Itô isometry gives \(\mathbb{E}[M_t^2]=\mathbb{E}\int_0^t H_s^2\,ds<\infty\). For general \(H\) in the stated class, approximate it in the norm \((\mathbb{E}\int_0^T H_s^2\,ds)^{1/2}\) by elementary adapted integrands. The Itô isometry gives convergence of their integrals in \(L^2\), and conditional expectation is a contraction in \(L^2\). Passing to the limit proves the conditional-expectation identity and square integrability for \(M\). \(\square\)

Representation Theorem and Derivation

Proof. Fix \(t\) and define \(A_{t,s}=\int_t^s c(r,X_r)\,dr\) for \(s\in[t,T]\). Since \(c\) is bounded, the exponential \(e^{-A_{t,s}}\) is bounded on this finite interval. Apply Itô’s formula to the function \(u(s,x)\) along \(X_s\). Its time derivative is \(u_t\), its first spatial derivative is \(u_x\), and its second spatial derivative is \(u_{xx}\); the quadratic-variation correction is explicitly \(\tfrac12\sigma(s,X_s)^2u_{xx}(s,X_s)\,ds\). Thus \[ du(s,X_s)=\left(u_t+\mu u_x+\tfrac12\sigma^2u_{xx}\right)(s,X_s)\,ds +u_x(s,X_s)\sigma(s,X_s)\,dB_s =(c u-f)(s,X_s)\,ds+u_x(s,X_s)\sigma(s,X_s)\,dB_s, \] where the PDE gives the second equality. The finite-variation factor satisfies \(d(e^{-A_{t,s}})=-c(s,X_s)e^{-A_{t,s}}\,ds\). The product rule has no quadratic-covariation term between this finite-variation factor and the diffusion term. Therefore \[ d\big(e^{-A_{t,s}}u(s,X_s)\big) =-e^{-A_{t,s}}f(s,X_s)\,ds +e^{-A_{t,s}}u_x(s,X_s)\sigma(s,X_s)\,dB_s. \] The stochastic integrand is progressively measurable. It is bounded because \(c\) and \(u_x\sigma\) are bounded and \(T\) is finite, so its squared time integral has finite expectation. By the lemma, its Itô integral is a true square-integrable martingale, not merely a local martingale. Integrating from \(t\) to \(T\), using \(u(T,X_T)=g(X_T)\), and taking conditional expectation given \(\mathcal{F}_t\) gives \[ e^{-A_{t,T}}g(X_T)-u(t,X_t) =-\int_t^T e^{-A_{t,s}}f(s,X_s)\,ds +\int_t^T e^{-A_{t,s}}u_x(s,X_s)\sigma(s,X_s)\,dB_s. \] The last integral has conditional expectation zero. Rearranging proves the asserted formula. The integrability assumptions also make the terminal and running terms integrable, since \(c,f,g\) are bounded and \(T\) is finite. \(\square\)

This is an Itô representation, not an ordinary chain-rule calculation. In particular, the second derivative term in \(\mathcal{L}_t u\) comes from Brownian quadratic variation. The argument proves the formula for the stated classical solutions; it does not assert that every expectation-defined function is classical without additional regularity assumptions.

Worked Representations

Why the Sign and Martingale Conditions Matter

The drift calculation determines the sign of the reward. The PDE says that the drift of the weighted terminal-value process \(e^{-A_{t,s}}u(s,X_s)\) is \(-e^{-A_{t,s}}f(s,X_s)\,ds\). Adding the accumulated reward \(\int_t^s e^{-A_{t,r}}f(r,X_r)\,dr\) cancels that drift. The resulting process is a martingale because its stochastic integrand is square-integrable under the stated assumptions. It is this cancellation—not a guess about whether the reward should have a plus or minus sign—that yields the formula.

Common mistake: reversing the sign of the running term. A tempting but incorrect expression is \(\mathbb{E}_{t,x}[e^{-A_{t,T}}g(X_T)-\int_t^T e^{-A_{t,s}}f(s,X_s)\,ds]\). For the second example, it gives \(-\int_0^2e^{-s}\,ds=-(1-e^{-2})\), whereas direct integration and the PDE both give \(1-e^{-2}\). The correction is to add the running reward: integrating the drift \(-e^{-A}f\) to the terminal time and rearranging moves that term to the other side with a plus sign.

A second pitfall is to call every stochastic integral a martingale without checking its integrand. Itô’s formula initially produces a local-martingale term in general. Here boundedness of \(c\) and \(u_x\sigma\), together with finite \(T\), puts the integrand in \(\mathcal{L}^2\); the lemma then verifies the true martingale property and justifies taking conditional expectations. Without a sufficient integrability condition, that step cannot be inferred from a local-martingale claim alone.