Fitting Semi-Markov Models with Clock-Reset Risk Sets

Why Reset the Transition Clock?

A transition hazard can depend on when follow-up began, or on how long a person has occupied the state they are currently in. Those are different time scales. If a patient enters a recovery state on day 20, a clock-forward model evaluates their transition risk using time since the study began; a clock-reset model evaluates it using time since day 20, their entry into recovery. When the scientific question concerns duration in the current state, resetting the clock is a natural way to represent that dependence.

A semi-Markov model allows a transition hazard to depend on the current state, the destination state, covariates, and elapsed time in the current state. This contrasts with a time-homogeneous Markov model, in which the current state is sufficient to describe the transition rate: once the current state is known, time already spent there does not further change the rate. A clock-reset analysis is not automatically appropriate merely because data can be reshaped this way; the choice of time scale should reflect the process being studied.

Definition: Let \(r\) be an origin state and \(s\) a possible destination. For an individual currently in state \(r\), let \(u\) denote elapsed time since entry into \(r\). The clock-reset transition hazard \(\lambda_{rs}(u\mid Z)\) is the instantaneous rate of transition from \(r\) to \(s\) at sojourn time \(u\), conditional on being in \(r\) and at risk immediately before \(u\), and on covariates \(Z\). A semi-Markov model permits this hazard to vary with \(u\).

For example, a transition-specific proportional-hazards model takes the form

\lambda_{rs}(u\mid Z)=\lambda_{0,rs}(u)\exp(\beta_{rs}^{\mathsf T}Z).

Here \(\lambda_{0,rs}(u)\) is the baseline hazard for transition \(r\to s\) on the sojourn-time scale, and \(\beta_{rs}\) is its transition-specific covariate coefficient vector. Separate transitions may have different baseline hazards and different coefficients. A Cox model leaves each baseline hazard unspecified; a parametric model specifies a distribution for it. In either case, the risk-set clock must match the intended time scale.

Constructing Clock-Reset Risk Sets

Represent each uninterrupted stay in an origin state as an episode. Record the time the episode starts, the time it ends, the origin state, the exit destination (if any), the covariates, and the individual identifier. For an episode beginning at calendar time \(L\) and ending at \(R\), its sojourn-time interval is from 0 to \(R-L\). If covariates change over the episode, split it into intervals at their change times, measuring each interval endpoint from the same state-entry time.

For a particular transition \(r\to s\), include episodes that begin in \(r\). An episode ending in a transition to \(s\) has an event indicator of 1 for this model. An episode ending in another destination is censored at that exit time for the cause-specific hazard of \(r\to s\): after leaving \(r\), the individual is no longer at risk of that transition. Administrative loss to follow-up is also a censoring event. Thus, for each destination-specific model, the end time is the episode’s sojourn duration, and the event indicator records whether the episode ended in the target destination.

This construction presumes that the relevant transition and censoring times are observed or handled by a valid observation model, that censoring is independent of the transition process conditional on the modeled information, and that the transition structure is correctly specified. For a proportional-hazards fit, the covariate effects must also satisfy proportional hazards on the chosen sojourn-time scale. If a person can contribute several episodes, those rows are not independent; use subject-level robust variance estimation when appropriate. Such variance estimation does not repair an incorrect risk set or informative censoring.

Worked Example: Turning State Histories into Sojourn Episodes

Suppose three hypothetical participants have these observed histories, with calendar time measured in days. A denotes the initial state, B a temporary state, and C an absorbing event state.

Person	History
1	A from day 0 to 4; enters B, then exits B to C at day 7
2	A from day 0 to 2; enters B, then is censored at day 8
3	A from day 0 to 5; exits A directly to C

For the B-to-C model, person 1’s B episode starts on day 4 and ends on day 7. The reset duration is \(7-4=3\) days, and the event indicator is 1. Person 2’s B episode starts on day 2 and is censored on day 8, giving duration \(8-2=6\) days and event indicator 0. Person 3 contributes no B episode because they never entered B.

Person 1’s B-to-C event is therefore recorded at sojourn time 3 days—not calendar day 7. Person 3’s direct A-to-C event belongs in the A-to-C model, at sojourn time \(5-0=5\) days. The episodes and endpoints are fully observed in this hypothetical example; in real data, delayed observation, missing state transitions, or dependent dropout would require additional handling rather than simply resetting the clock.

Fitting and Checking the Transition Models

With the episode data prepared, fit a separate Cox model for each transition, or fit a stratified model that gives transitions separate baseline hazards and allows the desired transition-specific effects. In software using start-stop survival records, set each episode’s start at zero and its stop at the reset duration; if an episode is split for time-varying covariates, use the corresponding reset-time interval endpoints. For example, a separate model for \(r\to s\) has the form \(\texttt{Surv(start, stop, event) \textasciitilde{} covariates}\), fit only to the relevant origin-state episodes. Use an individual-level cluster variance if repeated episodes from a person require it. The event indicator must refer to the target destination, not to any exit indiscriminately.

Worked Example: A Transition-Specific Cox Risk Set

At B-to-C sojourn time \(u=2\) days, suppose a hypothetical risk set contains three B episodes. All three remain in B just before day 2. One has covariate \(x=1\), and the other two have \(x=0\). The episode with \(x=1\) exits to C at day 2; the others leave B later or are censored later. Assume the proportional-hazards model \(\lambda_{BC}(u\mid x)=\lambda_{0,BC}(u)\exp(\beta x)\), with no ties at that event time.

The partial-likelihood contribution for the event is the event individual’s relative hazard divided by the sum of relative hazards in the risk set:

L(\beta)=\frac{\exp(\beta)}{\exp(\beta)+1+1} =\frac{\exp(\beta)}{\exp(\beta)+2}.

The two \(x=0\) episodes each contribute relative hazard 1; the \(x=1\) episode contributes \(\exp(\beta)\). If a hypothetical fitted model returned \(\hat{\beta}=0.4055\), the estimated hazard ratio for \(x=1\) versus \(x=0\) would be \(\exp(0.4055)=1.50\). In this example, that means the estimated instantaneous B-to-C transition rate is 1.50 times as high at a given time spent in B, conditional on still being in B and on the model’s assumptions. It does not mean that the probability of transition is 1.50 times as high over every follow-up interval.

This single risk set illustrates the likelihood contribution, not the complete model fit. A full fit uses all observed target transitions and their risk sets. Before interpreting it, verify that episode starts and ends are correct, that competing exits remove individuals from the risk set, and that the proportional-hazards assumption is tenable for the covariates.

Diagnostics should be performed on the reset scale. Check that event counts and time-at-risk totals by transition agree with the original histories; inspect whether any episode has a negative or implausible duration; and confirm that an individual is not still at risk in an origin state after leaving it. Examine proportional-hazards diagnostics and the adequacy of any parametric baseline on sojourn time. Sparse transitions can make the baseline and covariate effects unstable, especially in long-duration tails. A good numerical fit is not evidence that the reset clock is scientifically appropriate.

Predicting Occupancy When Duration Matters

A prediction based only on the current state is generally insufficient in a semi-Markov model. Two people in the same state can have different next-transition risks if they have spent different amounts of time there. For prediction from a known state-entry time, use the fitted sojourn-time hazards to obtain transition probabilities over duration, then account for the distribution of times at which individuals enter subsequent states. For more complicated networks, this calculation may require numerical integration or simulation over possible paths.

The following deliberately simple example shows the calculation. State 0 can transition either to state 1 or directly to absorbing state 2. State 1 can transition only to state 2. Assume the initial state is 0 at calendar time 0, there is no censoring, and covariates are fixed. The hazards out of state 0 are constant, \(\lambda_{01}=0.08\) per year and \(\lambda_{02}=0.02\) per year. In state 1, assume the hazard to state 2 depends on time \(u\) since entry and has survival function \(S_1(u)=\exp[-(u/4)^2]\). The assumed hazards and state structure fully specify this example’s prediction model.

Worked Example: Predicting State Probabilities at Five Years

The total rate of leaving state 0 is \(0.08+0.02=0.10\) per year. Therefore, the probability of still being in state 0 at five years is

P\{X(5)=0\}=\exp[-(0.10)(5)]=\exp(-0.5)=0.6065.

To be in state 1 at year 5, a person must enter state 1 at some time \(s\) between 0 and 5 and then remain there for \(5-s\) years. The density of entering state 1 at time \(s\), while still in state 0 immediately beforehand, is \(0.08\exp(-0.10s)\). Conditional on that entry, the probability of remaining in state 1 until year 5 is \(\exp[-((5-s)/4)^2]\). Thus,

P\{X(5)=1\} =\int_0^5 0.08\exp(-0.10s)\exp\left[-\left(\frac{5-s}{4}\right)^2\right]\,ds \approx 0.1943.

For example, Simpson’s rule with step size \(h=0.5\) gives an integral of approximately \(0.1943\). Since state 2 is absorbing and the three state probabilities sum to 1,

P\{X(5)=2\}=1-0.6065-0.1943=0.1992.

The estimated occupancy probabilities at five years are therefore 0.6065 in state 0, 0.1943 in state 1, and 0.1992 in state 2. The calculation explicitly weights each possible entry time into state 1 by its probability and applies the state-1 survival probability for the resulting sojourn duration. A calculation that treats every person in state 1 as having the same duration would not represent this model.

In applied work, replace the illustrative hazards with fitted transition models and propagate uncertainty in their estimated parameters. Compare predicted occupancy with observed state occupancy at relevant horizons, using appropriate methods for censoring, and assess sensitivity to alternative time scales or baseline-hazard specifications. Prediction also requires a clearly defined starting state and, where relevant, starting sojourn duration. For someone already in a state at the prediction origin, the elapsed time in that state is part of the conditioning information.

Common Pitfalls and Alternatives

Resetting at the wrong time. Using time since study entry rather than time since entry into the current origin state creates a clock-forward analysis, not the intended clock-reset analysis.
Keeping a person at risk after an exit. A subject who has left state \(r\) cannot remain in its transition risk sets. End the episode at the exit and record the appropriate destination.
Calling a cause-specific censoring event independent censoring without justification. A competing transition ends risk for the target transition, but censoring assumptions for estimating that cause-specific hazard still need to be considered in the modeled setting.
Confusing a hazard ratio with a probability ratio. A transition-specific hazard ratio compares instantaneous rates conditional on remaining in the origin state; it is not generally a ratio of cumulative transition probabilities.

A clock-forward Markov model is a useful alternative when transition rates depend on time since a common origin and the current state is sufficient for prediction under the model. Parametric transition models can also use a reset time scale, offering a specified baseline hazard and convenient calculations when the distributional assumptions are credible. The time scale and baseline form are separate choices: a parametric model is not necessarily clock-forward, and a clock-reset model need not be parametric. Semi-Markov models can be more realistic when duration in state changes risk, but their predictions require duration-aware calculations and can be difficult to estimate when few individuals remain in a state for long periods.

Key takeaway: A clock-reset model measures each transition episode from entry into its origin state. Its risk sets, fitted hazards, diagnostics, and predictions must all preserve that sojourn-time scale.

Check Your Understanding

Answer each question, then compare with the brief solutions.

A person enters state B at calendar day 12 and exits to C at day 19. What duration and event indicator belong in a B-to-C clock-reset record?
Why does an exit from B to a different destination end the B-to-C risk episode?
Under a fitted transition-specific coefficient of \(\log(0.8)\) for a one-unit increase in a covariate, what does the estimated hazard ratio mean?
What additional information is needed to predict transitions for someone already in a state at the prediction origin under a semi-Markov model?

Solutions. (1) The duration is \(19-12=7\) days, and the B-to-C event indicator is 1. (2) After leaving B, the person is no longer at risk of a B-to-C transition; the episode ends at that exit. (3) The estimated hazard ratio is \(\exp[\log(0.8)]=0.8\): at a given sojourn time, the instantaneous transition hazard is estimated to be 20% lower per one-unit covariate increase, conditional on the model. (4) The elapsed time already spent in the current state is needed, along with the state and modeled covariates, because risk can depend on sojourn duration.