Deriving the Multi-State Thiele System

Why a Multi-State Reserve Needs a System of Equations

In a single-state model, the reserve changes with interest, ongoing cash flows, and the possibility of an insured event. In a multi-state model, the same principles apply, but a transition can move the policy into a state with a different reserve. A reserve calculation must therefore account for both the cash paid at the transition and the reserve that applies immediately afterward.

This tutorial derives the reserve equation for each state directly from a short-interval balance. The resulting equations are coupled: the derivative of the reserve in one state can depend on reserves in several destination states. The derivation applies to a continuous-time Markov model with specified transition intensities and cash flows. It is aligned with the SOA Exam ALTAM (Advanced Long-Term Actuarial Mathematics) syllabus.

Definition: Let $i$ denote a state of a finite-state continuous-time Markov model. The state-specific prospective reserve $V_i(t)$ is the expected present value, at time $t$ and conditional on being in state $i$, of future outgo less future income under the contract. Let $\mu_{ij}(t)$ be the transition intensity from state $i$ to a different state $j$. Let $b_{ij}(t)$ be the lump sum paid by the insurer at that transition, positive for an outgo and negative for an income. Let $c_i(t)$ be the net continuous income rate to the insurer in state $i$: premiums less expenses and continuous benefits. The force of interest is $\delta(t)$.

These signs are important. A positive $b_{ij}$ increases the insurer’s outgo at a transition, whereas a positive $c_i$ is income and reduces the liability reserve. The reserve $V_j(t)$ is the prospective liability just after transition into state $j$. A state that terminates the contract may have reserve zero, but that is a contract-specific condition, not a requirement of the equation.

Deriving the Equation Over a Short Interval

Assume that, conditional on the current state, transition probabilities over a short interval of length $h$ satisfy $\Pr(i\to j\text{ in }[t,t+h))=\mu_{ij}(t)h+o(h)$. The probability of no transition is $1-\sum_{j\ne i}\mu_{ij}(t)h+o(h)$, and the probability of two or more transitions is $o(h)$. Assume continuous cash-flow rates and interest are locally well behaved, so first-order expansions are valid. Simultaneous transitions are excluded.

The prospective reserve is outgo less income. Thus, if the policy is in state $i$ at time $t$, continuous net income $c_i(t)h$ during the interval reduces the liability by that amount. If a transition to $j$ occurs, the insurer pays $b_{ij}(t)$ immediately and then faces the reserve $V_j(t+h)$. If there is no transition, the reserve at the end of the interval is $V_i(t+h)$.

Write the conditional balance.
Discount the expected end-of-interval liability back to time $t$, include transition payments, and subtract the continuous income received during the interval.

Use first-order probabilities.
Replace each transition probability by $\mu_{ij}(t)h+o(h)$, and the no-transition probability by $1-\sum_{j\ne i}\mu_{ij}(t)h+o(h)$.

Expand and collect terms.
Use $V_i(t+h)=V_i(t)+V_i'(t)h+o(h)$ and $e^{-\delta(t)h}=1-\delta(t)h+o(h)$. Subtract $V_i(t)$, divide by $h$, and let $h$ tend to zero.

The conditional balance in step 1, with time arguments suppressed where they are evaluated at $t$, is

V_i(t)=-c_i(t)h+e^{-\delta(t)h}\left[\left(1-\sum_{j\ne i}\mu_{ij}(t)h\right)V_i(t+h)+\sum_{j\ne i}\mu_{ij}(t)h\left(b_{ij}(t)+V_j(t+h)\right)\right]+o(h).

The transition term contains both $b_{ij}$, the immediate payment, and $V_j(t+h)$, the liability that remains afterward. Expanding the right-hand side to first order gives

V_i(t)+h\left[-c_i(t)+V_i'(t)-\delta(t)V_i(t)+\sum_{j\ne i}\mu_{ij}(t)\left(b_{ij}(t)+V_j(t)-V_i(t)\right)\right]+o(h).

The left-hand side is $V_i(t)$. Therefore the coefficient of $h$ must be zero. Rearranging gives the multi-state Thiele equation for state $i$:

V_i'(t)=\delta(t)V_i(t)+c_i(t)+\sum_{j\ne i}\mu_{ij}(t)\left(V_i(t)-V_j(t)-b_{ij}(t)\right).

Theorem: Under the stated short-interval assumptions, the reserve in each state satisfies $V_i'=\delta V_i+c_i+\sum_{j\ne i}\mu_{ij}(V_i-V_j-b_{ij})$. Apply the equation separately to each state, using that state’s continuous cash-flow rate and all its possible outgoing transitions. The equation does not by itself specify terminal or boundary conditions; those depend on the contract.

Reading the Terms

The interest term $\delta V_i$ reflects the change in the value of the liability as time passes. The term $c_i$ is positive for net income to the insurer and negative for net continuous outgo. For a possible transition $i\to j$, the contribution is its intensity multiplied by the change in liability associated with the event: the current-state reserve less the destination-state reserve and the immediate payment.

If there are no transition payments, set $b_{ij}=0$. If a transition terminates the contract with no further liability, use $V_j=0$, while retaining any payment due at that transition. If interest, intensities, or cash-flow rates vary with time, use their values at $t$; the same first-order derivation applies when those functions are sufficiently regular.

Worked Example: Active-State Reserve With Disability and Death Transitions

Consider a hypothetical policy with active, disabled, and dead states. Interest is at a constant force $\delta=0.03$ per year. The model is a time-homogeneous continuous-time Markov chain: from the active state, disability occurs with intensity $0.04$ per year and death with intensity $0.02$ per year; conditional holding times are exponential, with these competing transition intensities. The death state is absorbing. At the time considered, $V_{\text{active}}=\$8{,}000$, $V_{\text{disabled}}=\$11{,}000$, and $V_{\text{dead}}=0$. In the active state, the annual premium rate is $\$1{,}500$ and the annual expense rate is $\$300$. There is no payment on disability, and a $\$20{,}000$ death benefit is paid on transition to death. Find the instantaneous derivative of the active-state reserve.

Method. Apply the state-specific equation, with $c_{\text{active}}=1{,}500-300=\$1{,}200$ per year. The transition payments are $b_{\text{active,disabled}}=0$ and $b_{\text{active,dead}}=\$20{,}000$.

\begin{aligned} V_{\text{active}}' &=0.03(8{,}000)+1{,}200\\ &\quad+0.04(8{,}000-11{,}000-0) +0.02(8{,}000-0-20{,}000)\\ &=240+1{,}200-120-240\\ &=\$1{,}080\text{ per year}. \end{aligned}

The interest contribution is $\$240$ per year. Disability contributes $-\$120$ per year because the disabled-state reserve exceeds the active-state reserve. Death contributes $-\$240$ per year because the death payment plus the change in reserve makes that transition costly on this reserve basis. The total derivative is positive: at this instant the active-state liability is increasing at $\$1{,}080$ per year.

Check. The transition terms sum to $-\$360$ per year, while interest and net premium income sum to $\$1{,}440$ per year; their difference is $\$1{,}080$. The common mistake avoided here is treating the death benefit as income or omitting the current-state reserve from the transition term.

Worked Example: Continuous Care Outgo in a Disabled State

A hypothetical long-term care contract has active, disabled, and dead states. The force of interest is constant at $0.04$ per year. While disabled, the insured recovers to active with intensity $0.25$ per year or dies with intensity $0.05$ per year; holding time in the disabled state is exponential with total exit intensity $0.30$ per year. At the time considered, $V_{\text{disabled}}=\$15{,}000$, $V_{\text{active}}=\$9{,}000$, and $V_{\text{dead}}=0$. The insurer pays continuous care benefits of $\$6{,}000$ per year and incurs expenses of $\$300$ per year while disabled. A $\$12{,}000$ death benefit is paid at death, and recovery has no transition payment. Calculate $V_{\text{disabled}}'$.

Method. Continuous benefits and expenses are outgo, so $c_{\text{disabled}}=-(6{,}000+300)=-\$6{,}300$ per year. Use $b_{\text{disabled,active}}=0$ and $b_{\text{disabled,dead}}=\$12{,}000$.

\begin{aligned} V_{\text{disabled}}' &=0.04(15{,}000)-6{,}300\\ &\quad+0.25(15{,}000-9{,}000-0) +0.05(15{,}000-0-12{,}000)\\ &=600-6{,}300+1{,}500+150\\ &=-\$4{,}050\text{ per year}. \end{aligned}

Recovery contributes $0.25(\$6{,}000)=\$1{,}500$ per year because recovery moves from a $\$15{,}000$ reserve to a $\$9{,}000$ reserve with no payment. The death-transition contribution is $0.05(\$3{,}000)=\$150$ per year. The substantial ongoing care outgo makes the derivative negative overall.

Check. Combining the terms gives $600-6{,}300+1{,}500+150=-4{,}050$. The sign of the continuous benefit is essential: it is outgo, not income to the insurer.

Worked Example: Three Possible Transitions From One State

A hypothetical contract has states $X$, $Y$, $Z$, and death. At the instant considered, the force of interest is $0.02$ per year, and $V_X=\$12{,}000$, $V_Y=\$9{,}000$, $V_Z=\$16{,}000$, and $V_{\text{dead}}=0$. The time-homogeneous Markov model has outgoing intensities from $X$ of $0.10$ per year to $Y$, $0.05$ per year to $Z$, and $0.01$ per year to death; the holding time in $X$ is exponential with total exit intensity $0.16$ per year. The insurer receives net continuous income of $\$500$ per year in $X$. The transition payments are $\$1{,}000$ on $X\to Y$, zero on $X\to Z$, and a $\$25{,}000$ death benefit on $X\to\text{dead}$. Find the reserve derivative in $X$, then estimate the reserve $0.01$ years later if the policy remains in $X$.

Method. Apply the equation to state $X$, keeping each destination state and transition payment paired with its own intensity.

\begin{aligned} V_X' &=0.02(12{,}000)+500\\ &\quad+0.10(12{,}000-9{,}000-1{,}000)\\ &\quad+0.05(12{,}000-16{,}000-0)\\ &\quad+0.01(12{,}000-0-25{,}000)\\ &=240+500+200-200-130\\ &=\$610\text{ per year}. \end{aligned}

For a short interval $h=0.01$ years, use the first-order approximation $V_X(t+h)\approx V_X(t)+V_X'(t)h$, conditional on still being in $X$. Thus $V_X(t+h)\approx \$12{,}000+\$610(0.01)=\$12{,}006.10$.

Check. The three transition contributions total $200-200-130=-130$ dollars per year; interest plus net income contribute $240+500=740$, leaving $610$. The short-interval estimate is a first-order approximation, not an exact finite-time solution.

Common Sign and Direction Traps

A frequent error is to use $V_j-V_i+b_{ij}$ in place of $V_i-V_j-b_{ij}$. That reverses the reserve-change and payment signs. For the first worked example, this incorrect substitution would give

$$ 240+1{,}200+0.04(11{,}000-8{,}000)+0.02(0-8{,}000+20{,}000) =1{,}800. $$

The correct derivative is $\$1{,}080$ per year, as calculated above. The incorrect result treats the movement to a higher reserve as increasing liability change in the wrong direction and treats the death payment as if it reduced the transition cost. Return to the liability definition: an immediate insurer payment is positive outgo, while moving from the current reserve to a destination reserve changes liability by destination reserve less current reserve. The derivative equation uses the negative of that transition change, which produces $V_i-V_j-b_{ij}$.

A second trap is to use only the intensity-weighted destination reserve. The transition event includes both the payment at the event and the post-transition liability. A third is to mix annual intensities with an interval measured in months without converting units. If time is measured in years, $\delta$, each $\mu_{ij}$, and each continuous cash-flow rate must also be expressed per year.

Key takeaway: For every state, combine interest and net continuous income with one term for each outgoing transition. Each transition term is its intensity multiplied by current reserve minus destination reserve minus immediate transition payment. Keep the sign convention, state destination, and time unit consistent.

Check Your Understanding

For each problem, use the liability-reserve convention defined above. Unless stated otherwise, payments and rates are annual, and use the instantaneous Thiele equation.

A state has reserve $\$5{,}000$, force of interest $0.03$, and net continuous income $\$400$ per year. It transitions to a state with reserve $\$7{,}000$ at intensity $0.20$, with no transition payment. Find the reserve derivative. (Answer: $\$150$ per year)
A state has reserve $\$10{,}000$, force of interest $0.05$, premium income $\$1{,}200$ per year, and expenses of $\$300$ per year. Death has intensity $0.04$, pays $\$15{,}000$, and leaves zero reserve. Find the derivative. (Answer: $\$1{,}200$ per year)
A state has reserve $\$20{,}000$, force of interest $0.02$, and continuous net outgo of $\$1{,}000$ per year. Recovery has intensity $0.03$, destination reserve $\$15{,}000$, and transition payment $\$500$. Death has intensity $0.01$, zero destination reserve, and a $\$30{,}000$ payment. Find the derivative. (Answer: $-\$565$ per year)
In a state $i$, there are two possible transitions, to states $j$ and $k$. Write the reserve equation when the intensities are $\mu_{ij}$ and $\mu_{ik}$, the payments are $b_{ij}$ and $b_{ik}$, and net continuous income is $c_i$. (Answer: $V_i'=\delta V_i+c_i+\mu_{ij}(V_i-V_j-b_{ij})+\mu_{ik}(V_i-V_k-b_{ik})$)
A reserve derivative is $\$800$ per year at time $t$, and the current reserve is $\$9{,}500$. Using a first-order approximation, estimate the reserve $0.02$ years later, conditional on remaining in the same state. (Answer: $\$9{,}516.00$)

What You'll Learn