Choosing Forward or Backward Kolmogorov Equations

Why the Direction of the Equation Matters

In a continuous-time state model, a transition probability depends on both the state occupied at the starting time and the state being observed at the ending time. The Kolmogorov equations describe how those probabilities change when either endpoint moves. Choosing the right equation is therefore not a matter of choosing a different model: it is a matter of matching the equation’s time variable and boundary condition to the information in the problem.

This distinction is especially useful when a problem specifies a starting population distribution, a particular destination state, or transition intensities that vary with time. The methods here are aligned with the SOA Exam ALTAM (Advanced Long-Term Actuarial Mathematics) syllabus.

Definition: For a finite-state continuous-time Markov model, let \(q_{ij}(t)\) be the intensity of transition from state \(i\) to state \(j\) at time \(t\), for \(i\ne j\). The generator \(Q(t)\) has off-diagonal entries \(q_{ij}(t)\) and diagonal entries \(q_{ii}(t)=-\sum_{j\ne i}q_{ij}(t)\). Thus every row of \(Q(t)\) sums to zero. Let \(P(s,t)\) be the transition matrix from time \(s\) to time \(t\), with entry \(p_{ij}(s,t)\) equal to the probability of being in state \(j\) at time \(t\), given state \(i\) at time \(s\).

Throughout, probability distributions are row vectors. If the initial distribution is \(\boldsymbol{\alpha}(s)\), then the distribution at time \(t\) is \(\boldsymbol{\alpha}(s)P(s,t)\). This convention fixes the order of matrix multiplication in the equations below.

The Forward and Backward Equations

The forward equation moves the ending time \(t\), holding the starting time \(s\) fixed. The backward equation moves the starting time \(s\), holding the ending time \(t\) fixed. For a time-dependent generator, their matrix forms are:

\frac{\partial}{\partial t}P(s,t)=P(s,t)Q(t), \qquad \frac{\partial}{\partial s}P(s,t)=-Q(s)P(s,t).

The boundary condition is \(P(s,s)=I\), where \(I\) is the identity matrix. In component form, the forward equation is

\frac{\partial}{\partial t}p_{ij}(s,t) =\sum_k p_{ik}(s,t)q_{kj}(t),

and the backward equation is

\frac{\partial}{\partial s}p_{ij}(s,t) =-\sum_k q_{ik}(s)p_{kj}(s,t).

The equations are both valid descriptions of the same transition probabilities. The useful one depends on what is easiest to specify: forward equations naturally propagate an initial distribution, while backward equations naturally calculate a terminal-state probability or value for each possible starting state.

Identify the quantity.
Decide whether the requested result is a transition probability, a distribution across states, or a probability of a specified terminal event.

Choose the time direction.
Use the forward equation when advancing probabilities from a known initial distribution. Use the backward equation when the terminal condition is simple and the desired result varies by starting state.

Set boundary conditions and check.
At equal times, \(P(s,s)=I\). Verify that probabilities are between zero and one and that each row of \(P(s,t)\) sums to one.

Why the Equations Hold

The forward equation follows by considering a small increase \(h\) in the ending time. In that short interval, the probability of one transition from state \(k\) to state \(j\) is \(q_{kj}(t)h+o(h)\); the probability of two or more transitions is \(o(h)\). Conditioning on the state at time \(t\) gives

p_{ij}(s,t+h) =\sum_k p_{ik}(s,t)\left(\mathbf{1}_{\{k=j\}}+q_{kj}(t)h\right)+o(h).

Subtract \(p_{ij}(s,t)\), divide by \(h\), and let \(h\) decrease to zero. The result is \(\partial p_{ij}(s,t)/\partial t=\sum_k p_{ik}(s,t)q_{kj}(t)\), the forward equation. For the backward equation, condition instead on the first small interval after the starting time \(s\). Starting in \(i\), the model either stays in \(i\) or moves to a state \(k\), after which the remaining transition probability is \(p_{kj}(s+h,t)\). Expanding this one-step relation and taking the same limit yields \(\partial p_{ij}(s,t)/\partial s=-\sum_k q_{ik}(s)p_{kj}(s,t)\). The minus sign appears because increasing the starting time shortens the interval over which transitions can occur.

Worked Calculations

Worked Example: A Two-State Model Using the Forward Equation

A hypothetical benefit model has states 0 and 1. The intensity from 0 to 1 is \(0.4\) per year, and the intensity from 1 to 0 is \(0.2\) per year. The intensities are constant. Find the probability of being in state 1 after 3 years, starting in state 0.

Method. Propagate the probability of state 1 from its known initial value using the forward equation. Write \(p(t)=p_{01}(0,t)\). The probability of being in state 0 at time \(t\) is \(1-p(t)\). Transitions into state 1 occur at rate \(0.4\) while in state 0; transitions out occur at rate \(0.2\) while in state 1. Thus

p'(t)=0.4(1-p(t))-0.2p(t)=0.4-0.6p(t), \qquad p(0)=0.

The equilibrium value solves \(0.4-0.6p=0\), so it is \(2/3\). The solution of the differential equation is \(p(t)=\frac{2}{3}(1-e^{-0.6t})\). At \(t=3\), the exponent is \(-0.6(3)=-1.8\), and \(e^{-1.8}=0.1652988882\). Therefore

p_{01}(0,3)=\frac{2}{3}(1-0.1652988882) =\frac{2}{3}(0.8347011118) =0.5564674079.

Rounded to six decimal places, the answer is 0.556467. It lies between zero and one, as required. The method avoids treating the initial transition into state 1 as the only possible path: after entering state 1, the process can return to state 0 and enter state 1 again.

Worked Example: A Terminal-State Probability Using the Backward Equation

A hypothetical three-state process moves only along the path \(A\to B\to D\). The intensity from \(A\) to \(B\) is \(1\) per year, the intensity from \(B\) to \(D\) is \(1\) per year, and \(D\) is absorbing. Find the probability of being in \(D\) after 2 years when starting in \(A\).

Method. Use the backward equation because the destination condition is simple: at the terminal time, the probability of being in \(D\) is 1 if already in \(D\), and 0 otherwise. Let \(u_i(s)\) be the probability of being in \(D\) at time 2 given state \(i\) at time \(s\). The terminal conditions are \(u_A(2)=u_B(2)=0\) and \(u_D(2)=1\). The backward equations are

u_A'(s)=u_A(s)-u_B(s),\qquad u_B'(s)=u_B(s)-u_D(s),\qquad u_D'(s)=0.

It is convenient to use the remaining time \(r=2-s\). Starting in \(B\), the process reaches \(D\) within \(r\) years with probability \(1-e^{-r}\), so \(u_B(2-r)=1-e^{-r}\). For a process starting in \(A\), reaching \(D\) requires two successive exponential waiting times, each with rate 1. Equivalently, its probability \(g(r)=u_A(2-r)\) satisfies \(g(0)=0\) and

g'(r)=1-e^{-r}-g(r).

Multiplying by \(e^r\) gives \((e^rg(r))'=e^r-1\). Integrating from 0 to \(r\),

e^rg(r)=e^r-1-r, \qquad g(r)=1-e^{-r}(1+r).

At \(r=2\), \(e^{-2}=0.1353352832\), so \(e^{-2}(1+2)=0.4060058496\), and \(g(2)=1-0.4060058496=0.5939941504\). Rounded to six decimal places, the required probability is 0.593994. The value is less than 1 because both transitions must occur before the two-year endpoint.

Worked Example: A Time-Varying Intensity into an Absorbing State

A hypothetical two-state model has state \(A\) and absorbing state \(B\). The only transition has intensity \(\lambda(t)=0.1+0.2t\) per year, where \(t\) is measured in years from time 0. Starting in \(A\), find the probability of being in \(B\) at time 1.

Method. Use the forward equation for the probability \(p(t)\) of being in \(B\). Since \(B\) is absorbing, the probability of remaining in \(A\) is \(1-p(t)\), giving \(p'(t)=(0.1+0.2t)(1-p(t))\), with \(p(0)=0\). The probability of still being in \(A\) solves \(S'(t)=-\lambda(t)S(t)\), \(S(0)=1\), and hence

S(1)=\exp\left(-\int_0^1(0.1+0.2t)\,dt\right).

The cumulative intensity is \(0.1(1)+0.2(1^2)/2=0.1+0.1=0.2\). Therefore \(S(1)=e^{-0.2}=0.8187307531\), and the probability of being in \(B\) is

$$ p(1)=1-S(1)=1-0.8187307531=0.1812692469. $$

Rounded to four decimal places, the answer is 0.1813. The calculation checks because the state probabilities sum to \(0.8187307531+0.1812692469=1\). The rate is integrated over time; using only its endpoint value would not represent the stated model.

Common Exam Trap: Reversing the Transition Rates

For a two-state model with rate \(\alpha\) from 0 to 1 and rate \(\beta\) from 1 to 0, the forward equation for \(p_{01}(0,t)\) is

p'(t)=\alpha(1-p(t))-\beta p(t).

A common error is to use \(\beta\) for the flow into state 1 and \(\alpha\) for the flow out. In the first worked example, that reversed calculation would give \(p'(t)=0.2(1-p)-0.4p\), and then \(p(3)=\frac{1}{3}(1-e^{-1.8})=0.278234\). This is incorrect: \(\beta=0.2\) is the rate out of state 1, not into it. The correct flow rates depend on the state occupied immediately before the transition. Always read each off-diagonal generator entry as “from the row state to the column state.”

Choosing and Checking a Solution

A problem may ask for a probability from one specified starting state to one specified destination. Either equation can be used in principle, but the boundary conditions can make one much simpler. If the initial distribution is known and the generator is constant, forward evolution is often direct. If a terminal reward or destination event is specified for every terminal state, backward evolution can be convenient because it propagates that terminal condition to each possible starting state.

With time-varying intensities, retain the time argument in the generator. In general, one must solve the stated differential equations in their proper order; matrix generators at different times need not commute, so a constant-generator matrix-exponential shortcut cannot automatically be used. For a single transition into an absorbing state, the survival probability has the simpler form \(e^{-\int_s^t\lambda(u)\,du}\); for a more general model, solve the forward or backward system with the appropriate boundary conditions.

Key takeaway: Forward equations evolve the ending time and multiply \(P(s,t)\) by \(Q(t)\) on the right. Backward equations evolve the starting time and multiply \(Q(s)\) by \(P(s,t)\) on the left, with a minus sign. State the convention, time variable, and boundary condition before solving.

Check Your Understanding

For each problem, identify the relevant transition direction and use the stated time units.

A two-state model has constant intensities \(0.3\) per year from state 0 to state 1 and \(0.2\) per year from state 1 to state 0. Find the probability of being in state 1 after 3 years, starting in state 0. (Answer: 0.4661)
State \(B\) is absorbing, and the only transition intensity from \(A\) to \(B\) is \(0.2\) per year. Find the probability of being in \(B\) after 4 years, starting in \(A\). (Answer: 0.5507)
A two-state model has constant intensities \(0.5\) per year from state 0 to state 1 and \(0.25\) per year from state 1 to state 0. Find the probability of being in state 1 after 2 years, starting in state 1. (Answer: 0.7410)
A process moves only from \(A\) to \(B\) to absorbing state \(D\). The respective transition intensities are \(2\) and \(1\) per year. Find the probability of being in \(D\) after 1 year, starting in \(A\). (Answer: 0.3996)
State \(B\) is absorbing, with transition intensity \(\lambda(t)=0.05+0.1t\) per year from \(A\), where \(t\) is in years from time 0. Find the probability of being in \(B\) at time 2, starting in \(A\). (Answer: 0.2592)

What You'll Learn