Sojourn-Time-Conditioned Predictions in Semi-Markov Models

Why Time Already in a State Can Matter

At a landmark, knowing a person’s current state may not be enough to predict what happens next. The probability of leaving a state can depend on how long the person has already occupied it. For example, if a transition becomes more likely as time in a state increases, someone who has already remained there for a long time can have a different near-term outlook from someone who entered that state recently.

A semi-Markov prediction makes this elapsed time explicit. It conditions on the current state and its sojourn age, then accounts for the possibility of later transitions. It differs from a Markov prediction based only on current state, which assumes that the future distribution is unchanged by how long the process has occupied that state.

Definition: The sojourn age in state \(i\) is the time elapsed since the process most recently entered \(i\). A sojourn-time-conditioned state-occupation probability is the probability that the process occupies state \(k\) after a specified additional time \(h\), conditional on being in state \(i\) now and having sojourn age \(u\) now. Write this probability as \(p_{ik}(h\mid u)\).

The time origin for \(h\) is the landmark, not entry into the current state: \(h\) measures time from now to the prediction horizon, whereas \(u\) is time already spent in the current state. These quantities must not be interchanged. The target is occupation of a state at the horizon, not the probability of ever entering that state.

Hazards and the First Transition

Assume a finite-state, time-homogeneous semi-Markov process. Conditional on the current state and its sojourn age, the instantaneous transition rate to another state depends on that age, not on unrecorded earlier history or calendar time. Once a transition occurs, the sojourn clock for the new state starts at zero. The state set is exhaustive for the prediction, and transitions are generated by the specified rates. These assumptions allow repeated transitions to be handled recursively.

For \(i\ne j\), let \(\lambda_{ij}(w)\) be the transition-specific hazard at sojourn age \(w\) in state \(i\). It is the instantaneous rate of a transition from \(i\) to \(j\), conditional on still occupying \(i\) at age \(w\). The total exit hazard is \(\lambda_i(w)=\sum_{j\ne i}\lambda_{ij}(w)\). Thus, the probability of remaining in \(i\) for at least another \(v\) time units, given current age \(u\), is determined by the hazards over ages \(u\) through \(u+v\).

S_i(u,v)=\exp\left\{-\int_u^{u+v}\lambda_i(w)\,dw\right\}.

The density of the first transition occurring \(v\) time units from the landmark and going to \(j\) is \(S_i(u,v)\lambda_{ij}(u+v)\). The hazard is evaluated at age \(u+v\), not at \(v\), because the process has already accumulated \(u\) units in \(i\). This first-transition distribution is the ingredient needed to predict occupation after multiple possible transitions.

Result: For a time-homogeneous semi-Markov process with the clock reset on entry to each new state, the occupation probabilities satisfy

p_{ik}(h\mid u) = \mathbf{1}\{i=k\}S_i(u,h) + \sum_{j\ne i}\int_0^h S_i(u,v)\lambda_{ij}(u+v)\, p_{jk}(h-v\mid 0)\,dv.

The first term is the probability of occupying \(k\) at the horizon without leaving the current state; it is nonzero only when \(k=i\). In the integral, the process first leaves \(i\) at time \(v\), enters \(j\), and then has \(h-v\) time units for subsequent transitions. The final factor therefore starts from state \(j\) at sojourn age zero. The equation includes paths with more than one transition through the recursively defined probabilities \(p_{jk}(h-v\mid0)\).

Worked Predictions

Worked Example: Remaining in a State With an Increasing Exit Rate

Suppose a process is in state \(A\) at a landmark, having been there for \(u=2\) months. The only possible transition is from \(A\) to an absorbing state \(E\), with hazard \(\lambda_{AE}(w)=0.10+0.02w\) per month at sojourn age \(w\) months. Find the probability of still occupying \(A\) after \(h=4\) more months, and the probability of occupying \(E\).

The cumulative exit hazard is integrated over ages 2 to 6 months:

\int_2^6(0.10+0.02w)\,dw =0.10(6-2)+0.01(6^2-2^2) =0.40+0.32=0.72.

Therefore, the probability of no transition during the next four months is \(S_A(2,4)=e^{-0.72}\approx0.4868\). Since \(E\) is the only alternative state and is absorbing, its horizon occupation probability is \(1-0.4868=0.5132\). The two probabilities sum to one. Someone just entering \(A\) would use an integral from age zero instead, so the prediction depends on the age at the landmark.

Worked Example: Competing Destinations From the Current State

At a landmark, a hypothetical process has been in state \(A\) for 2 months. It can next enter either \(B\) or \(C\). Let the total exit hazard at age \(w\) be \(0.10+0.05w\) per month, with 60% of the exit hazard assigned to \(A\to B\) and 40% to \(A\to C\). Find the probabilities of occupying \(A\), \(B\), and \(C\) two months after the landmark, assuming \(B\) and \(C\) are absorbing.

The cause-specific hazards are \(\lambda_{AB}(w)=0.6(0.10+0.05w)\) and \(\lambda_{AC}(w)=0.4(0.10+0.05w)\). The integrated total hazard from current age 2 to age 4 is

\int_2^4(0.10+0.05w)\,dw =0.10(2)+0.025(4^2-2^2) =0.20+0.30=0.50.

Thus \(P(A\text{ at }h=2)=e^{-0.50}\approx0.6065\). The probability of any exit is \(1-e^{-0.50}\approx0.3935\). Because the destination-specific hazards are fixed shares of the total hazard at every age, the probability of exiting to each destination is the same share of that exit probability:

P(B\text{ at }h=2)=0.60(1-e^{-0.50})\approx0.2361,\qquad P(C\text{ at }h=2)=0.40(1-e^{-0.50})\approx0.1574.

The probabilities are approximately \(0.6065\), \(0.2361\), and \(0.1574\), which sum to 1.0000 up to rounding. This share calculation works here because the relative cause-specific hazards are constant over age and both destination states are absorbing. In general, destination probabilities require integrating each cause-specific first-transition density; they are not necessarily constant shares of the overall exit probability.

Worked Example: Occupation After a Transition to a Second Transient State

Consider states \(A\), \(B\), and absorbing state \(E\). At the landmark the process has been in \(A\) for 3 months. From \(A\), it transitions to \(B\) at constant rate \(0.30\) per month or to \(E\) at constant rate \(0.10\) per month. Once it enters \(B\), it transitions to \(E\) at constant rate \(0.20\) per month. Find the occupation probabilities after 2 more months.

The exit rate from \(A\) is \(0.40\) per month and does not depend on sojourn age. Thus, conditioning on having already spent 3 months in \(A\) does not change its remaining waiting-time distribution in this example: the probability of still occupying \(A\) after 2 months is \(e^{-0.40(2)}=e^{-0.8}\approx0.4493\).

The probability of entering \(B\) by the horizon and still occupying it then is an integral over the time \(v\) of the \(A\to B\) transition. The transition density at \(v\) is \(0.30e^{-0.40v}\); after entering \(B\), the probability of staying there for the remaining \(2-v\) months is \(e^{-0.20(2-v)}\). Therefore,

P(B\text{ at }h=2) = \int_0^2 0.30e^{-0.40v}e^{-0.20(2-v)}\,dv = 1.5e^{-0.4}(1-e^{-0.4}) \approx0.3315.

Since the three states are exhaustive and \(E\) is absorbing, its occupation probability is the remaining probability:

P(E\text{ at }h=2) =1-e^{-0.8}-1.5e^{-0.4}(1-e^{-0.4}) \approx1-0.4493-0.3315=0.2192.

The result is approximately \(0.4493\) in \(A\), \(0.3315\) in \(B\), and \(0.2192\) in \(E\), summing to 1.0000 to the displayed precision. This calculation distinguishes being in \(B\) at the horizon from ever having entered \(B\): some processes enter \(B\) and then reach \(E\) before the horizon.

Why the Semi-Markov Conditioning Matters

A current-state-only Markov prediction is appropriate when the future conditional distribution depends on the current state but not on the sojourn age. In a continuous-time Markov model, constant transition-specific hazards within each state provide this memoryless structure. The third example has precisely this property: although the landmark records three months already spent in \(A\), the constant exit rates imply the same remaining-time distribution as at age zero.

When hazards vary with sojourn age, that simplification generally fails. In the first example, the hazard rises with age, so a process currently at age 2 faces a different hazard profile over the next four months from a newly entered process. Applying a current-state-only transition matrix without justification would discard information relevant to the prediction. Conversely, conditioning on sojourn age does not automatically improve prediction: the age must be defined correctly, be available at the landmark, and be linked to a credible model for the transition hazards.

Specify the landmark information.
Record the current state \(i\), its sojourn age \(u\), and the additional prediction horizon \(h\).

Model the first exit.
Use the transition-specific hazards to calculate survival in \(i\) and the destination-specific density of the first transition.

Continue after transition.
Reset the sojourn age to zero in the destination state and account for possible subsequent transitions using the renewal equation.

Check interpretation and totals.
Confirm that each quantity is occupation at the horizon, that the conditioning time is correct, and that probabilities across exhaustive states sum to one.

Diagnostics, Comparisons, and Limitations

Several checks can reveal a faulty prediction. First, the integrated hazard must use the age interval from \(u\) to \(u+h\) for the initial state; integrating from zero silently treats the landmark subject as a new entrant. Second, destination-specific hazards must add to the total exit hazard. Third, the renewal integral must include the probability of remaining in each destination state for the time left after entry. Omitting that factor estimates entry by the horizon, not occupation at the horizon. Finally, for a closed and exhaustive state system, the occupation probabilities at a fixed horizon should sum to one, apart from numerical rounding.

The semi-Markov approach is not interchangeable with every alternative. A current-state-only Markov model is simpler and can be suitable when the transition process is memoryless or when duration has no useful predictive contribution. The semi-Markov model retains elapsed duration explicitly, but needs adequate information to estimate how transition rates vary with duration. A model that conditions on still more history may be needed if sojourn age alone does not capture the relevant dependence. The earlier material on conditioning information at a landmark distinguishes what may be used for prediction from information that is unavailable at that time.

The semi-Markov formulation also has limits. It assumes that, conditional on state and sojourn age, the transition law does not depend on additional unmodelled history; that the sojourn clock is reset upon entry; and, in the equations above, that hazards are not varying with calendar time. If hazards also depend on calendar time or landmark covariates, the prediction must represent those dependencies rather than use the time-homogeneous equation unchanged. Sparse observations at particular durations can make estimated age-specific hazards unstable, and predictions far beyond supported follow-up can be sensitive to extrapolation.

Common Mistakes

Starting the age clock at the landmark. This replaces current age \(u\) with zero. Use the actual time since entry into the current state.
Confusing event occurrence with horizon occupation. A transition into a state before \(h\) does not imply occupation there at \(h\), because another transition may occur in the meantime.
Using a current-state-only prediction by default. It is justified only when the relevant conditional transition law does not depend on duration, or when that simplification is otherwise defensible.
Forgetting the post-transition process. After the first transition, subsequent occupation probabilities must account for the new state’s transition law and remaining time.

Key takeaway: A semi-Markov state-occupation prediction conditions on both the current state and the time already spent there. Calculate the first transition using hazards over the correct age interval, then account for subsequent transitions before interpreting the probability at the prediction horizon.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A process has occupied state \(R\) for 5 weeks at the landmark. What age interval is used to calculate its probability of remaining in \(R\) for the next 3 weeks?
If the process can transition from \(R\) to either \(S\) or \(T\), what must the two cause-specific hazards sum to?
Explain why the probability of entering \(S\) by a horizon need not equal the probability of occupying \(S\) at that horizon.
In a model with constant transition-specific hazards within each state, does elapsed time in the current state change the remaining-time distribution? Explain.

Solutions. (1) Integrate the total exit hazard over ages 5 to 8 weeks, because the current age is 5 and the additional horizon is 3. (2) They must sum to the total exit hazard from \(R\) at each sojourn age: \(\lambda_R(w)=\lambda_{RS}(w)+\lambda_{RT}(w)\). (3) A process may enter \(S\) and then leave it before the horizon, so entry and occupation at the horizon are different events. (4) No. Constant hazards imply the memoryless property, so the remaining-time distribution is unchanged by elapsed time in the state.