Recurring Payments After Repeated Transitions

Introduction

Some state-contingent benefits are payable once, when a policyholder first enters a specified state. Other benefits are payable each time a transition occurs. For example, a hypothetical care contract might pay a fixed amount whenever a person moves from independent living to a care state, including after a return to independent living and a later move back into care. The possibility of repeated payments changes the valuation: later qualifying transitions must also be included, with their own probabilities and discount factors.

This tutorial develops a method for valuing those repeated transition payments in a continuous-time Markov model. It is aligned with the SOA Exam ALTAM (Advanced Long-Term Actuarial Mathematics) syllabus.

Definition: A recurrent transition benefit is a payment made whenever a specified transition occurs, including on later occurrences of that transition. If $b_{ij}$ is paid each time the process moves from state $i$ to state $j$, the payment is triggered by every $i\to j$ transition, not only the first one.

Model and Valuation Formula

Assume a finite-state, time-homogeneous continuous-time Markov chain with transition intensities $q_{ij}$ for $i\ne j$. While in state $i$, the process moves to state $j$ at intensity $q_{ij}$; write $q_i=\sum_{j\ne i}q_{ij}$ for the total intensity of leaving $i$. The chain has no simultaneous transitions, and the transition intensities and benefit amounts are fixed. A payment $b_{ij}$ is made immediately when an $i\to j$ transition occurs. The effective force of interest is a constant $\delta>0$, so a payment at time $t$ has discount factor $e^{-\delta t}$. The initial state is specified.

Let $p_i(t)$ be the probability that the process is in state $i$ at time $t$, given the initial state. The expected discounted time spent in $i$ over a period ending at $T$ is

U_i(T)=\int_0^T e^{-\delta t}p_i(t)\,dt.

For an infinite horizon, write $U_i=\int_0^\infty e^{-\delta t}p_i(t)\,dt$. With a finite-state chain, bounded transition intensities and benefits, and $\delta>0$, the discounted expected payment total is finite.

Theorem: The expected present value of payments made at every transition $i\to j$, over $[0,T]$, is

\operatorname{EPV}(T)=\sum_i\sum_{j\ne i} b_{ij}q_{ij}\int_0^T e^{-\delta t}p_i(t)\,dt =\sum_i\sum_{j\ne i} b_{ij}q_{ij}U_i(T).

For an infinite horizon, replace $U_i(T)$ with $U_i$. A transition can occur repeatedly, so this formula counts all its occurrences in expectation. It does not treat the transition as a one-time event.

Why the Formula Holds

The key is that a transition from $i$ to $j$ can occur only while the process occupies state $i$. Conditional on being in $i$ at time $t$, its probability of moving to $j$ during a short interval of length $h$ is $q_{ij}h+o(h)$. Thus, to first order, the expected payment in that interval is $b_{ij}p_i(t)q_{ij}h$. Discounting at time $t$ and summing these contributions over time gives an integral.

Identify the eligible transitions.
For each ordered pair $i,j$, record the amount $b_{ij}$ paid whenever $i\to j$ occurs. Set $b_{ij}=0$ for transitions that produce no payment.

Find discounted occupancy.
For each source state $i$, calculate $U_i(T)=\int_0^T e^{-\delta t}p_i(t)\,dt$, or the infinite-horizon integral when appropriate.

Convert occupancy to discounted transition counts.
The expected discounted count of $i\to j$ transitions is $q_{ij}U_i(T)$.

Multiply by the benefit and add.
Multiply each discounted transition count by its payment amount, then sum over the specified transitions.

The short-interval argument is also a justification of the intensity factor in the formula: $p_i(t)$ alone measures the chance of occupying the source state, not the chance of making a particular transition. The factor $q_{ij}$ is needed to convert time in that state into expected transitions of the specified type.

Two-State Occupancy for an Infinite Horizon

For a useful closed form, consider two states $A$ and $B$, with intensities $\alpha$ for $A\to B$ and $\beta$ for $B\to A$. Starting in $A$, define $U_A$ and $U_B$ as the infinite-horizon discounted occupancy times. The state probability equations, integrated with discounting, give

(\delta+\alpha)U_A-\beta U_B=1,\qquad -\alpha U_A+(\delta+\beta)U_B=0.

The 1 in the first equation represents the initial unit of probability in $A$; the second equation has no such term because the process starts in $A$. Solving the second equation gives $U_B=\alpha U_A/(\delta+\beta)$. Substituting into the first yields

U_A=\frac{\delta+\beta}{\delta(\delta+\alpha+\beta)},\qquad U_B=\frac{\alpha}{\delta(\delta+\alpha+\beta)}.

These formulas are for an initial state of $A$. If the initial state is $B$, the corresponding equations have 0 and 1, respectively, on their right-hand sides. The occupancy values must match the stated initial condition.

Worked Example: Repeated Payments in a Two-State Model

A hypothetical contract begins in state $A$. The process alternates between $A$ and $B$, with intensities $\alpha=0.20$ per year for $A\to B$ and $\beta=0.30$ per year for $B\to A$. It pays $1,000 at every $A\to B$ transition and $400 at every $B\to A$ transition. The force of interest is $\delta=0.05$ per year. Find the expected present value over the infinite horizon.

Method. Calculate discounted occupancy in each source state, then multiply by the corresponding transition intensity and payment. This includes every occurrence of both transitions.

The common denominator is

\delta(\delta+\alpha+\beta)=0.05(0.05+0.20+0.30)=0.05(0.55)=0.0275.

Starting in $A$, the discounted occupancy times are

U_A=\frac{0.05+0.30}{0.0275} =\frac{0.35}{0.0275} =12.7272727\ldots, \qquad U_B=\frac{0.20}{0.0275} =7.2727273\ldots.

The expected discounted number of $A\to B$ transitions is $0.20(12.7272727\ldots)=2.5454545\ldots$. Its present value contribution is $1{,}000(2.5454545\ldots)=\$2{,}545.4545\ldots$.

The expected discounted number of $B\to A$ transitions is $0.30(7.2727273\ldots)=2.1818182\ldots$. Its contribution is $400(2.1818182\ldots)=\$872.7273\ldots$. Therefore,

\operatorname{EPV} =2{,}545.4545\ldots+872.7273\ldots =\$3{,}418.1818\ldots \approx \$3{,}418.18.

Check. The contributions use the unrounded occupancy times; the displayed intermediate values are not rounded before multiplication. The final answer is rounded to the nearest cent. A common mistake is to count only the first $A\to B$ transition and omit later cycles, or to multiply a benefit by occupancy without also multiplying by its transition intensity.

When the Process Can Terminate

A recurrent payment rule can also apply in a model with an absorbing state. An absorbing state has no exits, so no further transition benefits arise after entry. For a small model, it can be convenient to value from each state using first-transition conditioning. If the process is currently in $i$, its time to leave is exponential with rate $q_i$. The discount-adjusted probability factor for a transition to $j$ is $q_{ij}/(\delta+q_i)$. Consequently, if $V_i$ is the expected present value starting in $i$,

V_i=\sum_{j\ne i}\frac{q_{ij}}{\delta+q_i}\bigl(b_{ij}+V_j\bigr).

This equation includes the benefit paid at the next transition and the value after arriving in the next state. It remains valid when a state can be revisited: the continuation value $V_j$ includes all future transitions, including later returns to $i$. For a transition to an absorbing state with no subsequent benefits, set its continuation value to zero.

Worked Example: Recurrent Benefits Before Absorption

A hypothetical long-term care contract has states $A$, $B$, and $D$, where $D$ is absorbing. From $A$, the intensities are $0.20$ per year to $B$ and $0.10$ per year to $D$. From $B$, they are $0.30$ per year to $A$ and $0.20$ per year to $D$. Each $A\to B$ transition pays $500, each $B\to A$ transition pays $200, each $A\to D$ transition pays $1,000, and each $B\to D$ transition pays $800. The force of interest is $\delta=0.10$ per year. Find the expected present value starting in $A$.

Method. Use first-transition conditioning, retaining the continuation value after every nonabsorbing transition. Let $V_A$ and $V_B$ denote the values starting in $A$ and $B$. The total exit intensities are $q_A=0.20+0.10=0.30$ and $q_B=0.30+0.20=0.50$.

From $A$, the discount-adjusted factors for the two possible exits are $0.20/(0.10+0.30)=0.50$ and $0.10/(0.10+0.30)=0.25$. Thus

V_A=0.50(500+V_B)+0.25(1{,}000) =0.50V_B+500.

From $B$, the corresponding factors are $0.30/(0.10+0.50)=0.50$ and $0.20/(0.10+0.50)=1/3$. Therefore,

V_B=0.50(200+V_A)+\frac{1}{3}(800) =0.50V_A+366.6667.

Substitute the second equation into the first: $V_A=0.50(0.50V_A+366.6667)+500 =0.25V_A+683.3333$. Hence $0.75V_A=683.3333$, so $V_A=911.1111$. Then $V_B=0.50(911.1111)+366.6667=822.2223$, with the unrounded values giving $V_B=822.2222\ldots$.

The expected present value starting in $A$ is therefore $911.11, rounded to the nearest cent. Substitution checks the result: $0.50(500+822.2222\ldots)+0.25(1{,}000)=661.1111\ldots+250=911.1111\ldots$. The $B\to A$ continuation is essential: omitting it would incorrectly treat the first return to $A$ as the end of all benefits.

First Entry Is Not Recurrence

A first-entry benefit pays only on the first qualifying transition. A recurrent benefit pays on every qualifying transition, including after the process leaves and later returns to the source state. The distinction is a contractual rule, not a consequence of the transition rates. In a Markov model, ordinary state occupancy is sufficient when the payment amount and eligibility depend only on the current transition. If the amount depends on how many times the transition has occurred, the count may need to be included as an additional state variable.

Worked Example: Entry Benefit Versus Repeated Benefit

A hypothetical process starts in $A$, with intensity $0.25$ per year for $A\to B$ and $0.25$ per year for $B\to A$. The force of interest is also $\delta=0.25$ per year. A benefit of $300 is paid at every $A\to B$ transition, and no benefit is paid at $B\to A$. Find the expected present value.

Method. Use discounted occupancy in state $A$, because every payment is triggered by an $A\to B$ transition. Here

U_A=\frac{0.25+0.25}{0.25(0.25+0.25+0.25)} =\frac{0.50}{0.1875} =\frac{8}{3}.

The expected discounted count of $A\to B$ transitions is $0.25(8/3)=2/3$. The expected present value is $300(2/3)=\$200.00$.

Check and trap. If the benefit were payable only on the first entry into $B$, the discounted probability factor for that first transition would be $0.25/(0.25+0.25)=0.50$, giving $300(0.50)=\$150.00$. That is not the contract described: the recurring benefit is $200.00 because later returns to $A$ permit additional $A\to B$ payments.

Common Exam Traps

Counting only the first occurrence. A payment at every qualifying transition requires all recurrent occurrences, not just a first-entry probability.
Using the wrong source state. The transition $i\to j$ is generated while in $i$, so its expected discounted count is $q_{ij}U_i$, not $q_{ij}U_j$.
Omitting the intensity. Discounted occupancy is time in a state, not a transition count. Multiply occupancy by the relevant transition intensity.
Ignoring the initial state. Discounted occupancy differs depending on where the process starts. Use equations or formulas consistent with that condition.
Applying a recurring rule after termination. Once the process enters an absorbing state with no exits, no later transitions or associated payments occur.
Mixing time units. Transition intensities and the force of interest must use the same time unit, such as per year.

Key takeaway: For a fixed payment made at every $i\to j$ transition, multiply the payment by the transition intensity $q_{ij}$ and the discounted expected time spent in source state $i$. Sum across eligible transitions, using the correct initial state and contract horizon.

Check Your Understanding

For each problem, include every qualifying transition and use the stated initial state.

A two-state process starts in $A$, has $A\to B$ intensity $0.10$ and $B\to A$ intensity $0.20$ per year, and has force of interest $\delta=0.10$. A $500 benefit is paid at every $A\to B$ transition, with no payment on the reverse transition. Find the infinite-horizon expected present value. (Answer: $375.00)
A two-state process starts in $B$, has $A\to B$ intensity $0.10$ and $B\to A$ intensity $0.20$ per year, and has $\delta=0.10$. A $250 benefit is paid at every $B\to A$ transition. Find the infinite-horizon expected present value. (Answer: $250.00)
A process starts in $A$, has $A\to B$ intensity $0.20$, $B\to A$ intensity $0.30$ per year, and $\delta=0.10$. It pays $200 at every $A\to B$ transition and $500 at every $B\to A$ transition. Find the infinite-horizon expected present value. (Answer: $766.67)
States $A$ and $B$ can transition to each other or to absorbing state $D$. The rates per year are $A\to B:0.20$, $A\to D:0.10$, $B\to A:0.30$, and $B\to D:0.20$. With $\delta=0.10$, payments are $300, $500, $100, and $400, respectively, at those transitions. Find the expected present value starting in $B$. (Answer: $427.78)
A two-state process starts in $A$, has both transition intensities equal to $0.25$ per year, and $\delta=0.25$. It pays $300 at every $A\to B$ transition and $100 at every $B\to A$ transition. Find the infinite-horizon expected present value. (Answer: $233.33)

What You'll Learn