Deriving Care-State Probabilities from Kolmogorov Equations

From Transition Intensities to Probability Equations

A long-term care intensity matrix describes the instantaneous rates of moving among health and care states. It does not, by itself, give the probability of occupying each state after a specified duration. The forward Kolmogorov equations connect the two: they describe how state probabilities change as probability flows along the permitted transitions. This tutorial derives and applies those equations, aligned with the SOA Exam ALTAM (Advanced Long-Term Actuarial Mathematics) syllabus.

Use a finite set of states, including death if mortality is part of the model. Let $Q(t)=[q_{ij}(t)]$ be the intensity matrix at time $t$, where $q_{ij}(t)$, for $i\ne j$, is the instantaneous transition intensity from state $i$ to state $j$. The diagonal entry is $q_{ii}(t)=-\sum_{j\ne i}q_{ij}(t)$, so each row sums to zero. A transition that is not permitted has intensity zero.

Definition: Let $\boldsymbol{p}(t)=(p_1(t),\ldots,p_n(t))$ be a row vector, where $p_j(t)$ is the probability of occupying state $j$ at time $t$. If the process has transition intensity matrix $Q(t)$, the forward Kolmogorov equations are $$ \boldsymbol{p}'(t)=\boldsymbol{p}(t)Q(t). $$ The initial condition is $\boldsymbol{p}(0)=\boldsymbol{p}_0$, the specified distribution of states at time zero.

The row-vector convention matters: probabilities are on the left and the intensity matrix is on the right. In component form, the equation for state $j$ is

p_j'(t)=\sum_{i\ne j}p_i(t)q_{ij}(t)-p_j(t)\sum_{k\ne j}q_{jk}(t).

The first sum is the total instantaneous flow into state $j$, from all other states. The second term is the flow out of state $j$. The equation is a rate of change, not itself a probability. In particular, $p_j'(t)$ can be negative when probability is leaving the state faster than it is entering.

Why the Forward Equation Holds

For a small time interval of length $h$, the probability of one transition from $i$ to $j$ is $q_{ij}(t)h+o(h)$. The probability of two or more transitions is $o(h)$, under the usual finite-intensity continuous-time Markov model. Conditional on being in state $j$ at time $t$, the probability of remaining there over the interval is $1-\sum_{k\ne j}q_{jk}(t)h+o(h)$. Thus, to first order,

p_j(t+h)=p_j(t)\left(1-\sum_{k\ne j}q_{jk}(t)h\right) +\sum_{i\ne j}p_i(t)q_{ij}(t)h+o(h).

Subtract $p_j(t)$, divide by $h$, and let $h$ decrease to zero. The resulting limit is the component forward equation above. This argument also explains why the equation accounts for all possible incoming transitions rather than only the most direct route into a care state.

The model assumes that the state description is sufficient to determine the transition intensities: conditional on the current state (and time, if intensities vary with time), the future transition behavior does not require the earlier path. For time-homogeneous intensities, $Q(t)=Q$ is constant and the solution can be written $\boldsymbol{p}(t)=\boldsymbol{p}_0e^{Qt}$. For time-varying intensities, the differential equation still applies, but in general one cannot replace its solution with $\boldsymbol{p}_0e^{Qt}$ for a single constant matrix.

Worked Examples

Worked Example: Healthy State, Care State, and Death

Method: Write the row-generator matrix, form the forward equations, and solve the resulting differential equations. A hypothetical insured begins healthy. The transitions are healthy to care at intensity $\lambda=0.20$ per year, healthy to death at intensity $0.10$ per year, and care to death at intensity $0.10$ per year. There is no recovery. Find the state probabilities after one year.

Order the states as healthy $H$, care $C$, and dead $D$. The diagonal entries are the negatives of the total outgoing intensities, so

Q= \begin{pmatrix} -0.30&0.20&0.10\\ 0&-0.10&0.10\\ 0&0&0 \end{pmatrix}, \qquad \boldsymbol{p}(0)=(1,0,0).

The equations are $p_H'=-0.30p_H$, $p_C'=0.20p_H-0.10p_C$, and $p_D'=0.10p_H+0.10p_C$. The healthy-state equation gives $p_H(t)=e^{-0.30t}$, so

p_H(1)=e^{-0.30}=0.740818.

For the care state, use the integrating factor $e^{0.10t}$. Since $p_C(0)=0$,

p_C(t)=e^{-0.10t}\int_0^t 0.20e^{0.10s}e^{-0.30s}\,ds =e^{-0.10t}(1-e^{-0.20t}).

At one year, $e^{-0.10}=0.904837$, $e^{-0.20}=0.818731$, and $1-e^{-0.20}=0.181269$. Therefore $p_C(1)=0.904837(0.181269)=0.164019$. Death is absorbing, and the probabilities across all three states must sum to one, so $p_D(1)=1-0.740818-0.164019=0.095163$.

The answer is $(p_H(1),p_C(1),p_D(1))=(0.740818,0.164019,0.095163)$, to six decimal places. The values are nonnegative and sum to $1.000000$, as required. A common mistake is to use $0.20$ as the total rate of leaving $H$; the healthy state also has a death transition, so its total exit intensity is $0.20+0.10=0.30$.

Worked Example: Two Care Levels with Mortality

Method: Solve the first two state equations, then use probability conservation to obtain the last living-state probability. In a hypothetical model, an insured begins healthy $H$, may move from $H$ to care level $C_1$ at intensity $0.20$ per year, and may deteriorate from $C_1$ to $C_2$ at intensity $0.30$ per year. Each living state has death intensity $0.10$ per year. There is no recovery. Find all probabilities after one year.

In state order $H,C_1,C_2,D$, the generator is

Q= \begin{pmatrix} -0.30&0.20&0&0.10\\ 0&-0.40&0.30&0.10\\ 0&0&-0.10&0.10\\ 0&0&0&0 \end{pmatrix}, \qquad \boldsymbol{p}(0)=(1,0,0,0).

The healthy-state equation gives $p_H(t)=e^{-0.30t}$, hence $p_H(1)=e^{-0.30}=0.740818$. The care-level $C_1$ equation is $p_{C_1}'=0.20p_H-0.40p_{C_1}$. With $p_{C_1}(0)=0$, an integrating factor gives

p_{C_1}(t)=0.20e^{-0.40t}\int_0^t e^{0.10s}\,ds =2(e^{-0.30t}-e^{-0.40t}).

At one year, $e^{-0.40}=0.670320$. Thus $p_{C_1}(1)=2(0.740818-0.670320)=2(0.070498)=0.140996$.

Because every living state has the same death intensity $0.10$, the total probability of still being alive at time $t$ is $e^{-0.10t}$. Therefore $p_D(1)=1-e^{-0.10}=1-0.904837=0.095163$, and $p_{C_2}(1)=0.904837-0.740818-0.140996=0.023023$.

The probabilities are $p_H(1)=0.740818$, $p_{C_1}(1)=0.140996$, $p_{C_2}(1)=0.023023$, and $p_D(1)=0.095163$, to six decimal places. Their sum is $1.000000$. The method avoids treating the probability in $C_2$ as simply the probability of being in $C_1$ times $0.30$: transition intensity is a rate, and an insured must first reach $C_1$ and then make the next transition before the valuation time.

Worked Example: Initial Mixture and a Short-Time Approximation

Method: Multiply the initial row vector by the generator to obtain the instantaneous probability changes, then use a first-order Euler step. This is an approximation, not an exact solution. In a hypothetical model, states are healthy $H$, care level $C_1$, care level $C_2$, and dead $D$. Intensities per year are $H\to C_1:0.12$, $H\to D:0.08$, $C_1\to H:0.20$, $C_1\to C_2:0.30$, $C_1\to D:0.10$, $C_2\to C_1:0.15$, and $C_2\to D:0.25$. At time zero, the state probabilities are $(0.60,0.30,0.10,0)$. Approximate the probabilities after $0.10$ year.

The total exit intensities are $0.20$ from $H$, $0.60$ from $C_1$, and $0.40$ from $C_2$. Thus

Q= \begin{pmatrix} -0.20&0.12&0&0.08\\ 0.20&-0.60&0.30&0.10\\ 0&0.15&-0.40&0.25\\ 0&0&0&0 \end{pmatrix}.

Apply $\boldsymbol{p}'(0)=\boldsymbol{p}(0)Q$, calculating each component:

\begin{aligned} p_H'(0)&=0.60(-0.20)+0.30(0.20)=-0.060,\\ p_{C_1}'(0)&=0.60(0.12)+0.30(-0.60)+0.10(0.15)=-0.093,\\ p_{C_2}'(0)&=0.30(0.30)+0.10(-0.40)=0.050,\\ p_D'(0)&=0.60(0.08)+0.30(0.10)+0.10(0.25)=0.103. \end{aligned}

The derivatives sum to $-0.060-0.093+0.050+0.103=0$, consistent with conservation of total probability. For $\Delta t=0.10$ year, the first-order approximation is $\boldsymbol{p}(\Delta t)\approx\boldsymbol{p}(0)+\Delta t\,\boldsymbol{p}'(0)$. The four components are $0.60-0.006=0.594$, $0.30-0.0093=0.2907$, $0.10+0.005=0.105$, and $0+0.0103=0.0103$.

The approximate probabilities are $(0.5940,0.2907,0.1050,0.0103)$, to four decimal places, and sum to $1.0000$. The common trap is mixing time units: the intensities are per year, so the step is $0.10$ year, not $0.10$ month. Also, the result is only a first-order approximation; for a larger interval, solve the differential system rather than assuming this Euler step is exact.

Checks, Interpretation, and Limits

For a finite-state model that includes all possible states, the rows of $Q(t)$ sum to zero. Consequently,

\frac{d}{dt}\sum_jp_j(t)=\boldsymbol{p}(t)Q(t)\boldsymbol{1}=0,

where $\boldsymbol{1}$ is a column vector of ones. If the initial probabilities sum to one, they continue to sum to one. This is a useful arithmetic check, but it is not sufficient by itself: probabilities must also be nonnegative, and the state ordering in $\boldsymbol{p}$ must match the ordering in $Q$.

The component equations are often easiest to build by drawing attention to the flows. For a care state, list every permitted transition into it and every transition out of it. Recovery is an inflow to a healthier state and an outflow from the care state; deterioration is an inflow to a more severe care state and an outflow from the less severe state; death is an outflow from each living state and an inflow to the absorbing death state. If death is omitted, the living-state probabilities generally will not sum to one when mortality is positive.

Check the diagonal: each diagonal is minus the sum of that row’s off-diagonal intensities.
Check the direction: with a row probability vector, use $\boldsymbol{p}'=\boldsymbol{p}Q$, not $Q\boldsymbol{p}$.
Check units: $p_j'(t)$ has units of probability per unit time, so the time increment must use the same units as the intensities.
Check the assumptions: constant-matrix solutions require time-homogeneous intensities; state-dependent intensities may also require the state definitions to capture duration or other relevant history.

These equations are useful because later actuarial calculations need probabilities of occupying each state at each time. A mistaken flow direction, omitted mortality transition, or incorrect exit rate changes those probabilities and can therefore affect valuations built from them. The forward equations provide a transparent way to verify how each permitted transition contributes.

Check Your Understanding

For each problem, use the stated state order and transition assumptions. Probabilities and rates in answers are rounded as shown.

A hypothetical model has states healthy $H$, care $C$, and dead $D$, with intensities $H\to C=0.15$ and $H\to D=0.05$ per year, and $C\to D=0.05$ per year. The insured starts in $H$. Give $Q$ and $\boldsymbol{p}'(0)$. (Answer: $Q=\begin{pmatrix}-0.20&0.15&0.05\\0&-0.05&0.05\\0&0&0\end{pmatrix}$; $\boldsymbol{p}'(0)=(-0.20,0.15,0.05)$ per year.)
Two living states $H,C$ have intensities $H\to C=0.10$ and $C\to H=0.20$ per year, with no other transitions. If $\boldsymbol{p}(0)=(0.70,0.30)$, find $\boldsymbol{p}'(0)$. (Answer: $(-0.010,0.010)$ per year.)
A hypothetical insured starts healthy. The transitions are $H\to C=0.10$, $H\to D=0.20$, and $C\to D=0.20$, all per year, with no recovery. Find the probabilities of $H,C,D$ at $t=2$. (Answer: $p_H(2)=e^{-0.60}=0.5488$, $p_C(2)=e^{-0.40}(1-e^{-0.20})=0.1215$, $p_D(2)=0.3297$.)
For states $H,C,D$, let $Q=\begin{pmatrix}-0.25&0.20&0.05\\0.10&-0.30&0.20\\0&0&0\end{pmatrix}$ and $\boldsymbol{p}(0)=(0.40,0.50,0.10)$. Calculate the three components of $\boldsymbol{p}'(0)$. (Answer: $(-0.050,-0.070,0.120)$ per year.)
For a time-homogeneous model with state order $H,C,D$, let $Q=\begin{pmatrix}-0.30&0.20&0.10\\0.10&-0.40&0.30\\0&0&0\end{pmatrix}$ per year and $\boldsymbol{p}(0)=(1,0,0)$. Find $\boldsymbol{p}''(0)$, using $\boldsymbol{p}''(0)=\boldsymbol{p}(0)Q^2$. (Answer: $(0.11,-0.14,0.03)$ per year squared.)

What You'll Learn