Why Model Assumptions Matter
Kolmogorov equations describe how probabilities move between states over time. Their use is not justified merely because a problem lists several states and rates. The equations require a suitable state description, transition intensities that define valid probabilities, and a process whose future evolution is determined by the current state and time rather than unrecorded history.
A model may be time-homogeneous, with intensities that are constant over time, or time-inhomogeneous, with intensities that vary with calendar time. Both can support Kolmogorov equations. The distinction matters because changing rates do not by themselves invalidate the model, whereas rates that depend on omitted information can invalidate a state-only Markov model.
For states \(i\) and \(j\), write \(p_{ij}(s,t)=P(X(t)=j\mid X(s)=i)\). Under a finite-state continuous-time Markov model, the transition intensity from \(i\) to \(j\), for \(i\ne j\), is
The matrix \(Q(t)=(q_{ij}(t))\) is the generator. Under the row-vector convention, its off-diagonal entries are transition intensities and its diagonal entry is \(q_{ii}(t)=-\sum_{j\ne i}q_{ij}(t)\). Thus every row sums to zero. For the usual finite-state model, intensities must be nonnegative off the diagonal and finite; suitable regularity, such as piecewise continuity or local integrability, allows the transition probabilities to be obtained from the Kolmogorov equations. At a time where a piecewise-defined intensity jumps, the differential equation is interpreted on either side, with transition probabilities continued across the jump.
These requirements are substantive. A negative off-diagonal entry cannot describe a transition rate. A row that does not sum to zero fails to balance probability outflow from a state. And if a rate depends on time spent in a state, but that duration is not included in the state or otherwise represented, the process may not be Markov in the stated states.
Justifying the Forward Equation
A short-interval argument shows why the generator appears in the forward equation. For a finite-state process with valid intensities, the probability of moving from \(i\) to \(j\ne i\) during a short interval of length \(h\) is \(q_{ij}(t)h+o(h)\). The probability of remaining in \(i\) is \(1+q_{ii}(t)h+o(h)\). The probability of two or more transitions in that interval is of smaller order than \(h\), under the usual finite-intensity assumptions.
Let \(p(t)\) be a row vector of state probabilities. The probability of being in state \(j\) at \(t+h\) is the sum, over possible states \(i\) at \(t\), of the probability of being in \(i\) times the short-interval probability of moving to \(j\). In vector form this gives
Subtract \(p(t)\), divide by \(h\), and take the limit as \(h\) decreases to zero. At times where the derivative exists, the result is the forward equation:
The argument depends on the Markov property and the short-interval intensity description. If either fails for the specified states, inserting a matrix into this equation does not repair the underlying model. For a finite state space with a constant generator, the model is time-homogeneous; when the generator varies with time, the forward equation still applies under appropriate regularity, but a single constant-generator solution cannot be assumed.
Worked Example: Checking Whether a Proposed Generator Is Valid
A hypothetical policy model has states Active, Disabled, and Dead, in that order. A proposed constant generator, with rates measured per year, is
Determine whether this is a valid generator, and find the instantaneous derivatives of the state probabilities when the initial distribution is \(p(0)=(0.6,0.3,0.1)\).
Method and generator check. A generator under the row convention must have nonnegative off-diagonal entries and each row must sum to zero. The row sums are \(-0.30+0.20+0.10=0\), \(0.15-0.40+0.25=0\), and \(0.05+0.05-0.10=0\). All six off-diagonal entries are nonnegative. Therefore \(Q\) is a valid generator.
Compute the derivative. The forward equation gives \(p'(0)=p(0)Q\). Component by component,
The derivatives sum to \(-0.130+0.005+0.125=0\), as probability conservation requires. A negative derivative for the Active probability is not itself a defect; it means that probability is initially flowing out of Active faster than it is flowing in.
Common mistake avoided. Do not require every component of \(p'(0)\) to be nonnegative. It is the off-diagonal generator entries that must be nonnegative; probability derivatives may have either sign.
When the State Description Is Not Markov
The Markov property concerns the information contained in the state, not whether the process seems intuitively random. If two policies are both labelled Disabled but their future recovery probabilities differ because one has been disabled longer, then “Disabled” alone does not determine the future transition law. The state description has omitted a relevant feature of the history.
This issue can arise when transition rates depend on duration in a state, time since a prior event, or an unrecorded characteristic that affects transition behavior. Possible remedies include adding information to the state, such as duration or a suitably defined duration category, or using a different modelling framework. An expanded state model must itself be specified carefully: simply adding a label does not automatically establish the Markov property.
Aggregation can cause the same difficulty. Suppose a model combines two subgroups into one displayed state, but the subgroups have different transition intensities. If the mix of subgroups among policies in that displayed state changes with history, the aggregate transition probabilities may depend on how the group was reached. The aggregated process need not be Markov, even when the more detailed process is Markov.
Worked Example: Detecting Omitted Duration
In a hypothetical disability model, a disabled policy recovers at intensity \(0.5\) per year during its first year of disability and at intensity \(0.1\) per year thereafter. Compare the probability of recovery in the next half-year for two policies observed at the same calendar time: one has been disabled for \(0.25\) years and the other for 2 years.
Method. For a constant recovery intensity \(\lambda\) over an interval of length \(t\), the probability of no recovery is \(e^{-\lambda t}\), so the recovery probability is \(1-e^{-\lambda t}\). This follows from the survival equation \(S'(t)=-\lambda S(t)\), \(S(0)=1\), whose solution is \(S(t)=e^{-\lambda t}\).
For the policy disabled for \(0.25\) years, the next \(0.5\) years remain within the first year of disability. Its recovery probability is \(1-e^{-0.5(0.5)}=1-e^{-0.25}=0.2212\), to four decimal places.
For the policy disabled for 2 years, the lower intensity applies throughout the next half-year. Its recovery probability is \(1-e^{-0.1(0.5)}=1-e^{-0.05}=0.0488\), to four decimal places.
At the same calendar time, both policies are in the displayed state Disabled, but their conditional future probabilities differ. Therefore the two-state description Active/Disabled is not Markov under this duration-dependent rule. Common mistake avoided: applying a single recovery intensity to every policy labelled Disabled ignores duration and does not represent the stated model.
Time Dependence Does Not Invalidate the Equations
A process can be Markov while its intensities change with calendar time. In that case the transition probabilities are conditional on the current time as well as the current state, and the generator is written \(Q(t)\). The forward equation uses the current-time generator. What fails is the extra assumption that one constant matrix applies throughout the entire period.
A piecewise-constant generator is an especially clear case. The Kolmogorov equation can be solved on each interval using that interval’s generator, with the probabilities at the end of one interval serving as the initial probabilities for the next. A jump in a rate at a known calendar time is not, by itself, a violation of the Markov property.
Worked Example: Applying a Time-Varying Intensity
A hypothetical two-state model has Active and Dead states. Death is absorbing. A person starts Active at time 0. The death intensity is \(0.2\) per year for \(0\le t<1\), then \(0.4\) per year for \(1\le t\le2\). Find the probabilities of being Active and Dead at time 2.
Method. The generator is \(Q(t)=\begin{pmatrix}-\lambda(t)&\lambda(t)\\0&0\end{pmatrix}\). While Active, the probability of remaining Active solves \(S'(t)=-\lambda(t)S(t)\), so across intervals it is the exponential of the negative integrated intensity. The stated rates are nonnegative and finite, and the generator rows sum to zero on both intervals.
The integrated intensity from 0 to 2 is \(0.2(1)+0.4(1)=0.6\). Therefore
The probabilities sum to \(0.5488+0.4512=1.0000\). Using the first year’s rate \(0.2\) over both years would instead give \(e^{-0.4}=0.6703\) for Active, which is wrong because it ignores the specified increase in intensity. Common mistake avoided: time-inhomogeneity is allowed, but the actual calendar-time rate must be used on each interval.
Practical Assumption Checks
Before applying a Kolmogorov equation, check the model in a fixed order. First, identify the states precisely and ask whether they include the information governing future transitions. Next, test the Markov condition: given the current state and time, should histories that led to that state have the same conditional future transition law? Then inspect the generator and the time specification.
Each state must have a clear meaning, and the states must describe the modeled process without relevant distinctions being silently discarded.
Ask whether the state and current time determine the transition behavior, or whether duration, prior events, or hidden subgroups still matter.
Every off-diagonal intensity must be nonnegative, and each diagonal entry must be the negative sum of the other entries in its row.
Decide whether the intensities are constant, time-varying, or piecewise-defined. Use the corresponding equation and respect interval boundaries.
For a finite-state model, these checks provide a practical basis for deciding whether the standard equations are appropriate. They do not establish that a model is a good representation of a real population; that requires evidence and judgement beyond the algebraic validity of the generator. In actuarial applications, this distinction is important: a mathematically valid equation can still encode an inadequate state structure or an unsuitable intensity assumption.
Check Your Understanding
For each problem, check both the probability-model assumptions and the arithmetic.
- A proposed two-state generator, in the order In Force and Lapsed, is \(Q=\begin{pmatrix}-0.18&0.18\\0&0\end{pmatrix}\), with rates per year. Is it a valid generator? (Answer: Yes; off-diagonal entries are nonnegative and both row sums are zero.)
- A three-state model has \(Q=\begin{pmatrix}-0.2&0.2&0\\0.1&-0.1&0\\0&0&0\end{pmatrix}\) and initial distribution \(p(0)=(0.7,0.3,0)\). Find \(p'(0)\). (Answer: \((-0.11,\ 0.11,\ 0)\) per year.)
- A hypothetical Active-to-Dead model has death intensity \(0.1\) per year for the first two years and \(0.25\) per year for the third year. Starting Active at time 0, find the Active and Dead probabilities at time 3. Give probabilities to four decimals. (Answer: Active \(e^{-0.45}=0.6376\); Dead \(1-e^{-0.45}=0.3624\).)
- Recovery intensity is \(0.4\) per year during the first year of disability and \(0.1\) per year thereafter. At the same calendar time, compare the next-half-year recovery probabilities for people disabled for \(0.5\) years and for 2 years. Give probabilities to four decimals. (Answer: \(1-e^{-0.2}=0.1813\) and \(1-e^{-0.05}=0.0488\); a state that records only Disabled is insufficient.)
- A model combines two subgroups into one state. Their transition intensities out of that state differ, and the subgroup mix depends on the history by which a policy entered it. Is the aggregate state necessarily Markov? What is the modelling concern? (Answer: No; the future transition law can depend on the entry history through the changing subgroup mix, so the aggregate state may omit relevant information.)