Why Condition on an Intermediate State?
A transition from state \(i\) at time \(s\) to state \(j\) at a later time \(t\) may occur through several possible states at an intermediate time \(u\). To find the probability of the endpoint \(j\), we can separate the possibilities according to which state is occupied at \(u\). The Chapman-Kolmogorov relationship formalizes this argument.
The key is to condition on the state at the intermediate time, not to assume a particular route or number of transitions between the endpoints. Once that intermediate state is specified, the Markov property says that the future transition probability depends on the current state and time, not on the earlier history.
Take three ordered times \(s\leq u\leq t\). The state \(X_u\) must be one of the states in the model. The events \(X_u=k\), as \(k\) ranges over the state space, are mutually exclusive and exhaustive. They therefore provide a partition on which to condition.
The Conditioning Proof
For each intermediate state \(k\), the probability of ending in \(j\) and occupying \(k\) at time \(u\), conditional on being in \(i\) at time \(s\), can be written as a product:
The first factor is a conditional probability of the future, given both the earlier state and the intermediate state. By the Markov property, once \(X_u=k\) is known, the earlier condition \(X_s=i\) does not change the distribution of the future. Thus that factor is \(P(X_t=j\mid X_u=k)=p_{kj}(u,t)\). The second factor is \(p_{ik}(s,u)\).
Adding these joint probabilities over the partition gives the endpoint probability:
For a countably infinite state space, the sum is over all states and is interpreted as a convergent sum of nonnegative probabilities. If a state \(k\) has zero probability at time \(u\), its contribution is zero; no conditional probability given that zero-probability event is needed.
The events \(X_u=k\) are mutually exclusive and cover every possible state at time \(u\).
Sum the endpoint probabilities contributed by each intermediate-state event.
Given \(X_u=k\), replace the conditional future probability given both \(X_u=k\) and \(X_s=i\) by the transition probability from \(k\).
Time-Homogeneous and Time-Inhomogeneous Notation
The proof does not require constant transition intensities or time homogeneity. In a time-inhomogeneous model, transition probabilities can depend on both the starting and ending times, so the two intervals in the formula retain their own time arguments: \(p_{ik}(s,u)\) and \(p_{kj}(u,t)\).
If the model is time-homogeneous, transition probabilities depend only on elapsed duration. Writing \(p_{ij}(r)\) for a transition over duration \(r\), the same result becomes
In particular, when the initial time is zero, this is often written \(p_{ij}(t)=\sum_k p_{ik}(u)p_{kj}(t-u)\). This shorter form is valid only when time homogeneity has been assumed. The conditioning proof itself remains the same in either setting.
Worked Example: Conditioning Across Three Intermediate States
A hypothetical Markov state model has states \(A,B,C\), and \(G\). At time 0 the system is in \(A\). At time 1, the probabilities of states \(A,B,C\), respectively, are \(0.50, 0.30, 0.20\). From those states at time 1, the probabilities of being in \(G\) at time 3 are \(0.20, 0.60, 0.10\), respectively. Find the probability of being in \(G\) at time 3. Assume the supplied probabilities are transition probabilities for this model; no interest or mortality assumption is involved.
Method. Condition on the state at time 1. The three intermediate-state probabilities sum to \(0.50+0.30+0.20=1.00\), so they cover all possibilities.
Through \(A\), the contribution is \(0.50(0.20)=0.10\). Through \(B\), it is \(0.30(0.60)=0.18\). Through \(C\), it is \(0.20(0.10)=0.02\).
Adding the mutually exclusive contributions gives \(0.10+0.18+0.02=0.30\). Therefore, the probability of being in \(G\) at time 3 is \(0.30\). The check is that the result is between 0 and 1 and is the weighted average of the three conditional probabilities.
Common mistake avoided: The three probabilities of reaching \(G\) from \(A,B,C\) cannot simply be added. Each must be weighted by the probability of occupying its corresponding intermediate state.
Worked Example: An Absorbing State as an Intermediate State
Consider a hypothetical long-term care model with states Active, Disabled, and Dead. At time 2, conditional on being Active at time 0, the probabilities of Active, Disabled, and Dead are \(0.70, 0.20, 0.10\). From those respective states at time 2, the probabilities of being Disabled at time 5 are \(0.15, 0.50, 0\). Dead is absorbing. Find the probability of being Disabled at time 5, given Active at time 0.
Method. Condition on the state at time 2. The intermediate probabilities sum to \(0.70+0.20+0.10=1.00\).
The contribution through Active is \(0.70(0.15)=0.105\). The contribution through Disabled is \(0.20(0.50)=0.100\). The contribution through Dead is \(0.10(0)=0\), because the model specifies that Dead is absorbing.
The total is \(0.105+0.100+0=0.205\). Thus the probability of being Disabled at time 5 is \(0.205\), or \(20.5\%\). The zero contribution through Dead is consistent with absorption: the system cannot leave Dead to enter Disabled.
Common mistake avoided: Being Disabled at time 2 does not guarantee being Disabled at time 5. Its contribution uses the conditional probability \(0.50\) of the later state, not just the intermediate probability \(0.20\).
Worked Example: Including a Zero-Contribution State
A hypothetical Markov model has four possible states \(A,B,C,D\) at an intermediate time. Given the initial state and time, their probabilities are \(0.40,0.30,0.20,0.10\). From these states, the probabilities of reaching a specified target state by the final time are \(0.10,0.50,0.60,0\), respectively. Find the overall target probability and identify the contribution through \(D\).
Method. Apply the law of total probability, weighting each conditional target probability by the probability of its intermediate state.
The four contributions are \(0.40(0.10)=0.04\), \(0.30(0.50)=0.15\), \(0.20(0.60)=0.12\), and \(0.10(0)=0\). Their sum is \(0.04+0.15+0.12+0=0.31\).
The overall target probability is \(0.310\), and the contribution through \(D\) is \(0\). The state \(D\) must still be included in the partition even though its contribution is zero. As a check, the intermediate probabilities sum to 1.00 and the weighted result lies between the smallest and largest conditional target probabilities, 0 and 0.60.
Common mistake avoided: Adding the conditional probabilities \(0.10+0.50+0.60+0\) gives \(1.20\), which is not the endpoint probability. Those values must be weighted by the probabilities of their respective intermediate states.
What the Relationship Does—and Does Not—Say
The sum describes all ways the process can be at the specified endpoint, grouped according to its state at one selected time. It does not say that the process made exactly one transition in either interval. There may have been no state change, one change, or several changes before or after \(u\); each transition probability already accounts for the model's behavior over its full interval.
The relationship also concerns the state at time \(u\), not whether a state was ever visited. For example, conditioning on \(X_u=k\) says the process is in \(k\) at that particular time. It does not include paths that visited \(k\) earlier and left before \(u\). Keeping the event being conditioned on precise is essential when using the formula.
Common Exam Traps
- Forgetting the weights. Conditional probabilities of the final state must be multiplied by the probabilities of the intermediate states, then summed.
- Leaving out a state with a zero contribution. A zero term does not change the total, but recognizing it shows that the intermediate states form a complete partition.
- Using only one intermediate route. Unless all other intermediate-state probabilities are zero, every possible state at the conditioning time contributes to the total.
- Dropping time arguments without justification. In a time-inhomogeneous model, \(p_{ij}(s,t)\) generally cannot be replaced by a function of \(t-s\) alone.
- Confusing an endpoint event with an ever-visited event. The formula concerns the state at time \(t\), with conditioning on the state at time \(u\).
Check Your Understanding
For each problem, condition on the stated intermediate state and show the weighted contributions before checking the answer.
- At an intermediate time, a model is in states \(A,B,C\) with probabilities \(0.20,0.50,0.30\). The conditional probabilities of reaching target \(J\) from those states are \(0.40,0.20,0.80\). Find the probability of reaching \(J\). (Answer: \(0.420\))
- At time 4, a system is in \(R\) with probability \(0.60\) and \(S\) with probability \(0.40\). From \(R\) and \(S\), the probabilities of state \(T\) at time 7 are \(0.25\) and \(0.50\), respectively. Find the probability of \(T\) at time 7. (Answer: \(0.350\))
- In a time-homogeneous model, the probabilities of occupying intermediate states \(K_1,K_2,K_3\) after one time unit from \(A\) are \(0.20,0.50,0.30\). The respective probabilities of reaching \(B\) over the next two time units are \(0.10,0.40,0.60\). Use Chapman-Kolmogorov to find \(p_{AB}(3)\). (Answer: \(0.400\))
- At an intermediate time, four states have probabilities \(0.40,0.30,0.20,0.10\). Their conditional probabilities of reaching the target are \(0.10,0.50,0.60,0\). Find the total target probability and the contribution from the fourth state. (Answer: \(0.310\) total; contribution from the fourth state is \(0\))
- The intermediate-state probabilities for \(A,B,C\) are \(0.25,0.35,0.40\). The conditional target probabilities from \(A,B\) are \(0.20,0.40\), and the overall target probability is \(0.36\). Find the conditional target probability from \(C\). (Answer: \(0.425\))