Common Transition Probability Calculation Errors

Why Transition Calculations Go Wrong

A transition probability is a probability of being in a specified state at a specified time, conditional on the state at an earlier time. Several common errors arise from changing one part of that statement without noticing: treating a rate as a probability, switching the row and column interpretation, skipping an intermediate state, or interpreting endpoint observations as information about every event along a continuous-time path. The calculations may look plausible while answering a different question.

The calculations below use finite-state Markov models. A process is Markov if, conditional on its current state, its future state distribution does not depend on its earlier history. This assumption is essential when composing transition matrices. Unless stated otherwise, matrices use the row-to-column convention: row \(i\), column \(j\) gives the probability of moving from state \(i\) to state \(j\).

Definition: For a Markov process with states \(1,\ldots,n\), \(p_{ij}(s,t)=P(X_t=j\mid X_s=i)\) is the transition probability from state \(i\) at time \(s\) to state \(j\) at time \(t\). The transition matrix \(P(s,t)\) has entries \(p_{ij}(s,t)\), with starting states indexing rows and ending states indexing columns. Each row sums to 1 when the process remains within the stated state space.

Rates Are Not Finite-Time Probabilities

A transition intensity is a rate, measured in inverse time. A transition probability over an interval is dimensionless and depends on the length of that interval as well as on the model. In particular, an intensity of \(0.4\) per year does not mean that the probability of a transition in one year is exactly \(0.4\). The approximation that a short-interval transition probability is about the intensity multiplied by the interval length is only a small-time approximation; it is not a general finite-time calculation.

Worked Example: Converting a Constant Intensity to a Probability

Consider a hypothetical two-state process, Active and Disabled. While Active, the individual moves to Disabled at a constant intensity of \(0.4\) per year. Disabled is absorbing, so return to Active is impossible. Find the probability of being Disabled after two years, given that the individual is Active initially.

Method and assumptions. The waiting time to the only possible transition is exponential with rate \(\lambda=0.4\) per year. The probability of no transition in \(t\) years is therefore \(e^{-\lambda t}\). Because Disabled is absorbing, being Disabled at two years is equivalent to having made the transition by then.

The probability of remaining Active for two years is \(e^{-0.4(2)}=e^{-0.8}=0.4493\).

Thus the probability of being Disabled at two years is \(1-0.4493=0.5507\), to four decimal places.

Check. The two state probabilities sum to \(0.4493+0.5507=1.0000\). The result is between 0 and 1, as required.

Common mistake avoided. Setting the two-year probability equal to \(0.4(2)=0.8\) confuses an intensity-times-time calculation with an exact probability. In this model, that value is not the transition probability; the exact probability is \(1-e^{-0.8}=0.5507\).

Read the Matrix Before Multiplying

Under the row-to-column convention, if the state distribution at a time is written as a row vector \(\boldsymbol{\pi}\), then propagation over an interval uses \(\boldsymbol{\pi}P\). For two consecutive intervals, the first interval's matrix is on the left and the second interval's matrix is on the right. This rule depends on both the row-vector convention and the Markov property.

Theorem: For a Markov process and times \(s\leq u\leq t\), the Chapman–Kolmogorov composition rule is

p_{ij}(s,t)=\sum_k p_{ik}(s,u)p_{kj}(u,t), \qquad P(s,t)=P(s,u)P(u,t).

The sum ranges over every possible state \(k\) at the intermediate time \(u\). For a time-inhomogeneous Markov process, the matrices may differ across intervals; the chronological multiplication order is unchanged.

To see why the rule holds, condition on the state at time \(u\). The events \(X_u=k\) partition the possibilities. By the Markov property, once \(X_u=k\) is specified, the conditional probability of being in state \(j\) at time \(t\) is \(p_{kj}(u,t)\), regardless of the history before \(u\). Hence

P(X_t=j\mid X_s=i) =\sum_k P(X_t=j\mid X_u=k, X_s=i)P(X_u=k\mid X_s=i) =\sum_k p_{kj}(u,t)p_{ik}(s,u).

Without the Markov property, the first conditional probability in the middle expression can depend on earlier history, and this matrix composition need not be valid.

Worked Example: Using the Intermediate State and Correct Matrix Order

A hypothetical three-state Markov model has states A, B, and C. Its transition matrices over the first and second intervals are

P_1= \begin{pmatrix} 0.6&0.3&0.1\\ 0.2&0.5&0.3\\ 0.1&0.2&0.7 \end{pmatrix}, \qquad P_2= \begin{pmatrix} 0.5&0.4&0.1\\ 0.1&0.6&0.3\\ 0.2&0.2&0.6 \end{pmatrix}.

The rows refer to the state at the start of each interval and columns to the state at its end. Starting in A, find the probability of ending in C after both intervals.

Method. Apply the first-interval matrix, then the second-interval matrix. Summing over the possible intermediate states gives

p_{AC}=(0.6)(0.1)+(0.3)(0.3)+(0.1)(0.6) =0.06+0.09+0.06=0.21.

The three terms correspond, respectively, to being in A, B, or C after the first interval, then moving to C during the second interval. The row-vector calculation gives the same result: the initial distribution \((1,0,0)\) becomes \((0.6,0.3,0.1)\) after the first interval, whose probability of C after the second is \(0.6(0.1)+0.3(0.3)+0.1(0.6)=0.21\).

Check and trap. Each matrix has nonnegative entries and rows summing to 1. Reversing the order would instead use the second interval first; its A-to-C entry after that incorrect multiplication is \(0.5(0.1)+0.4(0.3)+0.1(0.7)=0.24\), not \(0.21\). That calculation describes a different order of intervals.

Endpoint Events and Path Events Are Different

A transition matrix over an interval describes the state at the interval's endpoint, conditional on the starting state. It does not, by itself, say whether the process entered or left a state between those endpoints. Therefore an endpoint probability such as \(P(X_t=C\mid X_s=A)\) is not generally the probability of having visited C at any time between \(s\) and \(t\).

There is an important distinction between a discrete-time chain, where the only times relevant to “visits” are the specified observation times, and a continuous-time process, whose state can change between observations. For a discrete-time chain, transition matrices determine the probability of visiting a state at one or more of those discrete observation times. For a continuous-time path event, additional path or first-passage information may be needed. Do not infer a continuous-time ever-visited probability from endpoint matrices alone.

Worked Example: Probability of an Observed Visit Is Not an Endpoint Probability

Use the two matrices in the previous example, but now suppose they describe a discrete-time Markov chain observed at times \(0,1,2\). The chain starts in A. Find (a) the probability that the state at time 2 is C and (b) the probability that C is observed at time 1 or time 2.

(a) Endpoint at time 2. Sum over the state at time 1, as before: \(P(X_2=C\mid X_0=A)=0.6(0.1)+0.3(0.3)+0.1(0.6)=0.21\).

(b) Observed visit by time 2. C is observed at time 1 with probability \(0.1\). If the state at time 1 is A or B, the probability of being in C at time 2 is \(0.1\) or \(0.3\), respectively. These cases are disjoint from the event of observing C at time 1, so

P(X_1=C\text{ or }X_2=C\mid X_0=A) =0.1+0.6(0.1)+0.3(0.3) =0.1+0.06+0.09=0.25.

Check. The observed-visit probability \(0.25\) exceeds the endpoint probability \(0.21\), because some paths are in C at time 1 and not in C at time 2.

Common mistake avoided. Calling \(0.25\) the probability of ever visiting C along a continuous-time path would not be justified by these matrices. Here it is specifically the probability of observing C at one of the two discrete observation times. A continuous-time process could enter and leave C between observations without appearing in either endpoint state.

Three-Interval Calculations and Path Restrictions

For more than two intervals, retain every intermediate state in an endpoint calculation. A particular path restriction is a different calculation: it uses only the state sequences that satisfy that restriction. The distinction prevents a second common error—confusing the probability of one specified sequence with the probability of all paths that share the same starting and ending states.

Worked Example: Endpoint Probability Versus One Specified Sequence

A hypothetical two-state, discrete-time Markov chain has states A and B. Over three consecutive intervals its transition matrices are

P_1= \begin{pmatrix}0.7&0.3\\0.2&0.8\end{pmatrix}, \quad P_2= \begin{pmatrix}0.6&0.4\\0.5&0.5\end{pmatrix}, \quad P_3= \begin{pmatrix}0.9&0.1\\0.3&0.7\end{pmatrix}.

Starting in A, calculate (a) the probability of ending in B after three intervals and (b) the probability of the particular sequence A, B, A, B at times 0, 1, 2, 3.

(a) Endpoint probability. After interval 1 the state distribution is \((0.7,0.3)\). After interval 2 it is \((0.7(0.6)+0.3(0.5),\;0.7(0.4)+0.3(0.5))=(0.57,0.43)\). For the final interval, the probability of B is

$$ 0.57(0.1)+0.43(0.7)=0.057+0.301=0.358. $$

(b) Specified sequence. The Markov property gives the probability of this one sequence as the product of its successive transition probabilities: \(P(A\to B\to A\to B)=0.3(0.5)(0.1)=0.015\).

Check and trap. The sequence probability \(0.015\) is less than the endpoint probability \(0.358\), as expected: ending in B includes many other sequences. Multiplying only the transitions in the specified sequence answers part (b), not part (a).

A Practical Error-Checking Routine

Before finalizing a transition calculation, state the model and check what event the question asks for. The following routine catches most errors in these calculations.

Identify the quantity.
Decide whether the question concerns a finite-time probability, an endpoint state, a specified sequence, or a visit event.

Check the assumptions.
For matrix composition, establish that the process is Markov. For an intensity-to-probability conversion, identify the transition model, time units, and interval length.

Fix the convention.
For row-to-column matrices and row-vector distributions, use chronological multiplication from left to right in the written product: first-interval matrix, then second-interval matrix.

Include every required state or path.
Sum over all intermediate states for an endpoint probability. Restrict the sum only when the question specifies a path condition.

Run basic checks.
Transition probabilities must be between 0 and 1; complete rows must sum to 1; and a probability of an observed visit cannot be smaller than the probability of ending in that visited state.

These checks are especially useful in multi-state insurance models, where a transition may represent a change in health, employment, or benefit status. A mistaken interpretation can affect a later valuation even when the arithmetic itself is accurate. The key is to preserve the model assumptions and the exact event definition throughout the calculation. These distinctions are aligned with the SOA Exam ALTAM (Advanced Long-Term Actuarial Mathematics) syllabus.

Check Your Understanding

For each question, identify the event and apply the stated transition convention carefully.

A hypothetical two-state process moves from Working to Disabled at a constant intensity of \(0.25\) per year, with Disabled absorbing. Starting Working, find the probability of being Disabled after 3 years. Give the answer to four decimals. (Answer: \(1-e^{-0.75}=0.5276\).)
A discrete-time Markov chain has transition matrices \(P_1=\begin{pmatrix}0.8&0.2\\0.1&0.9\end{pmatrix}\) and \(P_2=\begin{pmatrix}0.7&0.3\\0.4&0.6\end{pmatrix}\), with rows and columns ordered A, B. Starting in A, find the probability of ending in B after both intervals. (Answer: \(0.8(0.3)+0.2(0.6)=0.36\).)
For the matrices in question 2, find the probability of the specific discrete-time sequence A, B, B. (Answer: \(0.2(0.6)=0.12\).)
A discrete-time chain starts in A, and its first transition matrix has A row \((0.6,0.3,0.1)\) for states A, B, C. Its second matrix has A, B, C entries in the C column equal to \(0.1,0.3,0.6\), respectively. Find the probability of ending in C after two steps. (Answer: \(0.6(0.1)+0.3(0.3)+0.1(0.6)=0.21\).)
A continuous-time process is observed only at times 0 and 1. Its transition matrix says \(P(X_1=C\mid X_0=A)=0.2\). Can this entry alone determine the probability that the process visited C at any time during the interval? State what it does determine. (Answer: No; it determines the probability of being in C at time 1 given A at time 0, not the probability of visiting C at any time between.)

What You'll Learn