State Occupancy Probabilities from an Initial Distribution

Why the Initial Distribution Matters

A transition probability describes where a process may be at a future time, conditional on its state now. In many actuarial models, however, the starting state is not known with certainty. A covered person might already be in one of several states when observation begins, or a portfolio might contain policies distributed across states. The occupancy probabilities at a later time must then account for both the starting distribution and the transitions from each possible starting state.

This tutorial develops that calculation using the row-vector convention. The model is a continuous-time Markov chain: conditional on its current state, the future transition behaviour depends on the current time and state, not on the earlier path. Unless stated otherwise, rates are constant over time and measured per year.

Definition: For states \(1,\ldots,n\), the initial distribution is the row vector \(p(0)=(p_1(0),\ldots,p_n(0))\), where \(p_i(0)=P(X(0)=i)\), \(p_i(0)\geq 0\), and \(\sum_i p_i(0)=1\). The transition probability \(P_{ij}(0,t)\) is the probability of being in state \(j\) at time \(t\), given state \(i\) at time 0. The occupancy distribution at time \(t\) is the row vector \(p(t)\), whose \(j\)th entry is \(P(X(t)=j)\).

Evolving a Row Distribution

Write \(P(0,t)\) for the transition probability matrix from time 0 to time \(t\). Its \((i,j)\) entry is \(P_{ij}(0,t)\), so rows correspond to starting states and columns to destination states. For any initial distribution, the occupancy distribution is

p(t)=p(0)P(0,t), \qquad p_j(t)=\sum_{i=1}^{n}p_i(0)P_{ij}(0,t).

The component formula is the law of total probability: to be in state \(j\) at time \(t\), the process must have started in some state \(i\), an event with probability \(p_i(0)\), and then moved from \(i\) to \(j\), with conditional probability \(P_{ij}(0,t)\). Summing over all possible initial states gives the result.

For a time-homogeneous model, let \(Q\) be its intensity matrix. Its off-diagonal entry \(q_{ij}\) is the transition intensity from state \(i\) to state \(j\); the diagonal entry is \(q_{ii}=-\sum_{j\ne i}q_{ij}\). The forward equation for the transition matrix is \(P'(0,t)=P(0,t)Q\). Multiplying on the left by the fixed initial row vector gives the distribution-level forward equation:

$$ p'(t)=p(0)P'(0,t)=p(0)P(0,t)Q=p(t)Q. $$

Thus one can obtain occupancy probabilities either by calculating the relevant transition probabilities and multiplying by \(p(0)\), or by solving the forward equations for \(p(t)\) directly. The initial condition for the latter method is \(p(0)\), not a distribution concentrated in a particular state unless the problem says so. For time-inhomogeneous intensities, the corresponding forward equation is \(p'(t)=p(t)Q(t)\), with the transition matrix for that time-dependent model.

Key result: With the row-vector convention, the initial distribution multiplies the transition matrix on the left: \(p(t)=p(0)P(0,t)\). This is a weighted average of the transition-matrix rows, with weights given by the probabilities of the initial states.

Worked Occupancy Calculations

In each example, the state order used for vectors and matrices is stated explicitly. Intermediate exponentials are retained in the calculation; final probabilities are reported to four decimal places.

Worked Example: A Two-State Model With an Uncertain Starting State

A hypothetical contract is classified as either Active, state \(A\), or Disabled, state \(B\). The transition intensities are \(A\to B=0.4\) per year and \(B\to A=0.2\) per year. Transitions follow a time-homogeneous continuous-time Markov chain. At time 0, the distribution is \(p(0)=(0.70,0.30)\), in the order \((A,B)\). Find the occupancy distribution after 2 years.

Method. First obtain the two-state transition probabilities, then weight their rows by the initial probabilities. For rates \(a\) from \(A\) to \(B\) and \(b\) from \(B\) to \(A\), let \(k=a+b\). The probability of being in \(A\) at time \(t\), conditional on starting in \(A\), is \(b/k+(a/k)e^{-kt}\); conditional on starting in \(B\), it is \((b/k)(1-e^{-kt})\). Here \(a=0.4\), \(b=0.2\), and \(k=0.6\).

At \(t=2\), \(e^{-kt}=e^{-1.2}=0.3011942119\), so \(1-e^{-kt}=0.6988057881\). The two probabilities of being in \(A\) are

P_{AA}(0,2)=\frac{0.2}{0.6}+\frac{0.4}{0.6}e^{-1.2} =0.5341294746, \qquad P_{BA}(0,2)=\frac{0.2}{0.6}(1-e^{-1.2}) =0.2329352627.

The other entries in those rows are \(P_{AB}(0,2)=1-P_{AA}(0,2)=0.4658705254\) and \(P_{BB}(0,2)=1-P_{BA}(0,2)=0.7670647373\). Therefore,

P(0,2)= \begin{pmatrix} 0.5341294746&0.4658705254\\ 0.2329352627&0.7670647373 \end{pmatrix}.

Multiply the initial row vector by this matrix:

$$ p_A(2)=0.70(0.5341294746)+0.30(0.2329352627) =0.3738906322+0.0698805788 =0.4437712110, $$

$$ p_B(2)=0.70(0.4658705254)+0.30(0.7670647373) =0.3261093678+0.2301194212 =0.5562287890. $$

To four decimal places, \(p(2)=(0.4438,0.5562)\). The entries are nonnegative and sum to 1, providing a useful arithmetic check. The common mistake avoided here is treating the initial distribution as though every contract started in \(A\); that would use only the first row of \(P(0,2)\), not the weighted combination of both rows.

Worked Example: Occupancy in a Sequential Model With an Absorbing State

A hypothetical care model has three states: Independent \(A\), Receiving care \(B\), and Deceased \(C\). The only transitions are \(A\to B\) at intensity \(0.5\) per year and \(B\to C\) at intensity \(0.25\) per year; \(C\) is absorbing. The chain is time-homogeneous. At time 0, the distribution in the order \((A,B,C)\) is \((0.20,0.50,0.30)\). Find the distribution after 2 years.

Method. Calculate the transition probabilities from each initial state, then use the initial probabilities as weights. Starting in \(A\), the probability of still being in \(A\) after 2 years is \(e^{-0.5(2)}=e^{-1}=0.3678794412\). The probability of being in \(B\) is the probability of leaving \(A\) at time \(s\) and remaining in \(B\) until time 2:

P_{AB}(0,2) =\int_0^2 0.5e^{-0.5s}e^{-0.25(2-s)}\,ds =2(e^{-0.5}-e^{-1}) =0.4773024380.

Since the only other destination is \(C\), \(P_{AC}(0,2)=1-0.3678794412-0.4773024380=0.1548181208\). Starting in \(B\), the probability of remaining in \(B\) is \(P_{BB}(0,2)=e^{-0.25(2)}=e^{-0.5}=0.6065306597\), and \(P_{BC}(0,2)=1-e^{-0.5}=0.3934693403\). Starting in \(C\), the process remains there. Thus

P(0,2)= \begin{pmatrix} 0.3678794412&0.4773024380&0.1548181208\\ 0&0.6065306597&0.3934693403\\ 0&0&1 \end{pmatrix}.

Weight each row by its initial probability. The occupancy probability of \(A\) is

$$ p_A(2)=0.20(0.3678794412)=0.0735758882. $$

The occupancy probability of \(B\) is

$$ p_B(2)=0.20(0.4773024380)+0.50(0.6065306597) =0.0954604876+0.3032653299 =0.3987258175. $$

The occupancy probability of \(C\) is

$$ p_C(2)=0.20(0.1548181208)+0.50(0.3934693403)+0.30 =0.0309636242+0.1967346702+0.3000000000 =0.5276982944. $$

To four decimal places, \(p(2)=(0.0736,0.3987,0.5277)\). The unrounded values sum to 1.0000000001 because of displayed decimal precision. The mistake to avoid is using the transition probabilities conditional on starting in \(A\) for the entire group; the given initial distribution includes people already in \(B\) or \(C\).

Worked Example: Different Exit Rates From Two Transient States

A hypothetical monitoring model has two non-absorbing states, \(A\) and \(B\), and an absorbing state \(C\). The only transitions are \(A\to C\) at intensity \(0.2\) per year and \(B\to C\) at intensity \(0.5\) per year. The model is time-homogeneous. The initial distribution in the order \((A,B,C)\) is \((0.40,0.35,0.25)\). Find the occupancy distribution after 2 years.

Method. In each transient state, the waiting time to the sole exit is exponential with that state's stated intensity. Therefore, the probability of remaining in \(A\) for 2 years is \(e^{-0.2(2)}=e^{-0.4}=0.6703200460\), and the probability of remaining in \(B\) is \(e^{-0.5(2)}=e^{-1}=0.3678794412\). The transition matrix is

P(0,2)= \begin{pmatrix} e^{-0.4}&0&1-e^{-0.4}\\ 0&e^{-1}&1-e^{-1}\\ 0&0&1 \end{pmatrix}.

The probability of occupying \(A\) is contributed only by those who started in \(A\) and did not exit:

p_A(2)=0.40e^{-0.4} =0.40(0.6703200460) =0.2681280184.

Similarly, only those who started in \(B\) and did not exit remain in \(B\):

p_B(2)=0.35e^{-1} =0.35(0.3678794412) =0.1287578044.

The probability of occupying \(C\) includes the initial \(0.25\) already there and the portions of the initial \(A\) and \(B\) groups that have exited:

\begin{aligned} p_C(2) &=0.25+0.40(1-e^{-0.4})+0.35(1-e^{-1})\\ &=0.25+0.1318719816+0.2212421956\\ &=0.6031141772. \end{aligned}

To four decimal places, \(p(2)=(0.2681,0.1288,0.6031)\). The values sum to 1 at the displayed precision. A common error is to apply one exit rate to both transient states; the survival probability in each state uses its own intensity.

Using the Forward Equation Directly

The distribution-level forward equation can be especially convenient when the intensities are known and the desired result is a set of occupancy probabilities. For a time-homogeneous model, form \(Q\) using the stated state order and solve \(p'(t)=p(t)Q\) subject to the full initial row vector. For example, in the two-state model with rates \(a\) and \(b\), the equation for the first component is

$$ p_A'(t)=-a p_A(t)+b p_B(t) =-a p_A(t)+b(1-p_A(t)). $$

The second equality uses the two-state probability identity \(p_B(t)=1-p_A(t)\). Rearranging gives \(p_A'(t)=b-(a+b)p_A(t)\). Its equilibrium value is \(b/(a+b)\), and solving the first-order equation with initial value \(p_A(0)\) gives

p_A(t)=\frac{b}{a+b} +\left(p_A(0)-\frac{b}{a+b}\right)e^{-(a+b)t}.

This formula explicitly shows how the initial distribution matters: the initial value \(p_A(0)\) determines the coefficient of the exponential term. It also provides a direct check on a matrix calculation for a two-state model. The formula requires constant transition intensities between the two states; it should not be applied unchanged when the intensities vary with time.

Common Convention Trap

With row vectors, the order of multiplication is not interchangeable. The expression \(P(0,t)p(0)^{\mathsf T}\) treats the initial distribution as a column vector and uses the transition matrix in the wrong orientation for the convention here. For the first worked example, using that incorrect product gives a first component of

$$ 0.5341294746(0.70)+0.4658705254(0.30)=0.5136517898, $$

which is not the correct \(p_A(2)\). The correct calculation uses the \(A\)-destination column, weighted by the probabilities of starting in each state:

p_A(2)=0.70P_{AA}(0,2)+0.30P_{BA}(0,2)=0.4437712110.

Before multiplying, check that each row of \(P(0,t)\) is indexed by an initial state, each column by a destination state, and that the initial distribution is a row vector. Also check that the initial probabilities are used as weights, rather than silently replacing them with a certain starting state.

Key takeaway: Occupancy probabilities combine uncertainty about the initial state with subsequent state transitions. State the vector orientation, state order, initial distribution, and time point before calculating; then check that the resulting entries are valid probabilities.

Check Your Understanding

For each problem, use the stated state order and report the final probabilities to four decimal places.

A two-state chain has intensities \(A\to B=0.1\) and \(B\to A=0.3\) per year. The initial distribution is \((0.40,0.60)\). Find \(p_A(2)\). (Answer: \(0.5927\))
State \(A\) transitions to absorbing state \(B\) at intensity \(0.4\) per year. The initial distribution is \((0.60,0.40)\). Find the occupancy distribution after 1.5 years. (Answer: \((0.3293,0.6707)\))
States \(A\) and \(B\) transition only to absorbing state \(C\), at intensities \(0.25\) and \(0.5\) per year, respectively. The initial distribution is \((0.20,0.50,0.30)\). Find the distribution after 2 years. (Answer: \((0.1213,0.1839,0.6948)\))
A sequential model has \(A\to B=0.4\) and \(B\to C=0.2\) per year, with \(C\) absorbing. Starting in \(A\) with certainty, find the distribution after 1 year. (Answer: \((0.6703,0.2968,0.0329)\))
A two-state chain has intensities \(A\to B=0.2\) and \(B\to A=0.2\) per year. Its initial distribution is \((0.80,0.20)\). Find \(p_A(3)\). (Answer: \(0.5904\))

What You'll Learn