Why the Memory Structure Matters
In an illness-death process, people may move from a healthy state to an illness state, or directly from healthy to death; people who become ill may later die. Choosing a model means deciding what information is needed to describe each transition rate. A state-only specification uses the current state and calendar time. A duration-dependent specification also uses time since entry into the current state. An event-history-dependent specification uses relevant features of the observed past, such as age at illness onset or cumulative exposure.
These choices concern the conditional dependence represented by the model, not simply which formula is most convenient. In particular, an apparent difference between people with different durations or recorded histories is not enough to establish duration or history dependence if they were observed at different calendar times. The comparison must control for calendar time as well as the other relevant information.
For a transition from state \(r\) to state \(s\), write \(\lambda_{rs}(t\mid\mathcal H(t-))\) for its intensity at calendar time \(t\), conditional on the observed history \(\mathcal H(t-)\) just before \(t\). Informally, over a short interval of length \(dt\), this is the conditional transition probability divided by \(dt\), to first order. The relevant intensity is defined only while the person is in the transition’s origin state.
The memory structure can differ by transition. For example, the rate of becoming ill may depend on calendar time alone, while the rate of death after illness may depend on time since illness onset. The transition labels identify the origin and destination; the choice of memory determines which aspects of time and history are allowed to affect that transition’s intensity.
Three Specifications and Their Distinguishing Questions
| Specification | Information in the transition intensity | Question the specification addresses |
|---|---|---|
| State-only | Current state and calendar time | Among people in the same state at the same calendar time, is additional dependence on duration or past history needed? |
| Duration-dependent | Current state, calendar time, and time since entering the state | At the same calendar time and state, does time already spent in the state help predict the transition? |
| Event-history-dependent | Current state, calendar time, duration, and specified past-history information | After accounting for state, calendar time, and duration, does recorded past history add predictive information? |
The state-only model is a time-inhomogeneous Markov specification: transition rates may change with calendar time, even when the current state is unchanged. The duration-dependent model distinguishes people in the same state and at the same calendar time if they entered that state at different times. The event-history-dependent model distinguishes people further when a relevant feature of their past differs, after the state, duration, and calendar time have been accounted for.
This ordering describes the information being considered, not a guarantee that every model is a simple nested parametric model. A duration specification may use a function of duration, and a history specification may use a chosen summary of past events. Their usefulness depends on whether those variables capture a real, adequately measured dependence pattern.
How to Select a Memory Structure
Start by stating the dependence pattern that the science suggests for each transition. Then ask whether a proposed predictor contributes information beyond the variables already included. For duration, compare people in the same origin state and at the same calendar time but with different times since entry. For event history, compare people in the same state, at the same calendar time, and with the same duration, but with different histories. If those comparisons are not possible in the data, or if calendar time is not otherwise controlled, the data do not isolate the proposed dependence.
Identify the origin and destination states and define the time origin for calendar time and for time in state. In an illness-death process, the duration clock for \(1\to2\) starts at entry into illness.
Decide whether the transition intensity can change over calendar time. Do not mistake a calendar-time pattern for a duration or history effect.
At comparable calendar times and within the same state, assess whether duration in the state adds predictive information. If it does, use a duration-dependent specification for that transition.
After accounting for state, calendar time, and duration, assess whether a prespecified past-history feature adds predictive information. Include it only when the comparison supports its relevance and the feature can be measured reliably.
Inspect whether the proposed dependence is supported across relevant time ranges and groups, and whether predictions remain adequate on data not used to choose the specification.
Worked Examples
Worked Example: State-Only Pattern for Death After Illness
Hypothetical data. Investigators compare two duration groups for the transition \(1\to2\) during the same six-month calendar window. They also compare people with different recorded pre-illness histories, but the history groups have been balanced within duration group for this illustration. Suppose the observed totals are:
| Duration and history comparison | Deaths | Person-months at risk | Crude rate |
|---|---|---|---|
| Shorter duration, history group A | 4 | 100 | 4/100 |
| Shorter duration, history group B | 4 | 100 | 4/100 |
| Longer duration, history group A | 4 | 100 | 4/100 |
| Longer duration, history group B | 4 | 100 | 4/100 |
Each crude rate is \(4/100=0.0400\) deaths per person-month. The comparisons are made within the same calendar window, and the rates show no pattern by duration or history in these hypothetical data. If broader, adequately supported comparisons likewise show no added duration or history pattern, a state-only specification for \(1\to2\) is a reasonable candidate. These small counts do not establish that the true rates are equal; they illustrate the pattern that would motivate the simpler specification, subject to uncertainty and further checking.
Worked Example: Duration Dependence After Controlling Calendar Time
Hypothetical data. In a common calendar quarter, suppose the \(1\to2\) risk sets have these totals, with comparisons also balanced for the recorded history under consideration:
| Duration since illness onset | Deaths | Person-months at risk |
|---|---|---|
| 0–3 months | 2 | 100 |
| More than 3 months | 8 | 100 |
The estimated rates are \(2/100=0.0200\) and \(8/100=0.0800\) deaths per person-month. Their rate ratio is
In this illustration, the longer-duration group has four times the observed death rate in that calendar quarter. Because the comparison is within the same calendar period and is balanced for the stated history, the pattern supports including duration for \(1\to2\), rather than attributing the difference to calendar time or that history. It does not by itself prove a population-level duration effect: sampling variation, unmeasured differences, and uncertainty still matter.
Worked Example: History Dependence Beyond State, Duration, and Calendar Time
Hypothetical data. Consider people currently in state \(1\), compared within the same calendar month and with the same two-to-four-month duration range. The past-history variable is whether the person had high cumulative exposure before illness onset.
| Pre-illness exposure history | Deaths | Person-months at risk | Observed rate |
|---|---|---|---|
| Lower | 1 | 100 | 1/100 |
| Higher | 6 | 100 | 6/100 |
The observed rates are \(0.0100\) and \(0.0600\) deaths per person-month. Their rate ratio is \(0.0600/0.0100=6.00\). Since the groups are compared at the same calendar time, in the same state, and at the same duration, this pattern supports considering an event-history term for pre-illness exposure in the \(1\to2\) intensity. The comparison addresses predictive dependence; it does not show that exposure caused the rate difference. With only one and six deaths, the estimate is imprecise, so the evidence would need to be assessed with suitable uncertainty measures and additional diagnostics before adopting the specification.
Why the Distinction Follows From the Intensity
The memory structure determines what differences the model permits between people who are currently in the same state. Under a state-only specification, once calendar time and current state are fixed, the intensity is the same regardless of how long the person has been in that state or what relevant past history they had. A duration-dependent intensity relaxes the first restriction. An event-history-dependent intensity relaxes the further restriction that past history adds nothing after duration and calendar time are considered.
For example, suppose two people are both ill at calendar time \(t\). If one entered illness at time \(t-u_1\) and the other at \(t-u_2\), their durations are \(u_1\) and \(u_2\). A duration model permits different death intensities when \(u_1\ne u_2\), even though the people share the same state and calendar time. This is the justification for keeping calendar time fixed when diagnosing duration dependence: otherwise a calendar-time change can produce the observed difference.
Here \(Z(t-)\) denotes specified information from the past, and the final form represents history dependence only if \(Z(t-)\) adds information after conditioning on state, calendar time, and duration. The display is schematic: an applied model must define its intensity functions and any history summaries precisely.
Assumptions, Diagnostics, and Limits
A defensible choice presumes that the three states and allowed transitions describe the process adequately; entry times and transition times are recorded with sufficient accuracy; and the duration clock is reset at the correct state-entry time. The analysis also needs an appropriate account of censoring and incomplete observation. In particular, observed risk sets must represent the people still at risk under the analysis assumptions. If censoring is informative after conditioning on recorded information, simple comparisons of observed rates may be misleading.
Useful diagnostics include plots or summaries of transition rates by calendar time and duration, comparisons of predicted and observed transition patterns within relevant groups, and checks for whether any proposed history effect persists when calendar time and duration are controlled. A specification can fail in several ways:
- Calendar-time confounding: rates differ because the groups were observed in different periods. Compare within calendar-time strata or otherwise account for calendar time before inferring duration or history dependence.
- Unstable duration patterns: apparent changes are driven by sparse risk sets or a few events. Report uncertainty and avoid treating a noisy rate ratio as a settled dependence pattern.
- Poorly chosen history summaries: a single summary may miss relevant features of the past, while an overly detailed history can be difficult to estimate and interpret.
- Incorrect state or clock definitions: misclassified illness onset or an incorrectly reset duration clock distorts duration comparisons.
No one memory structure is automatically best for every transition. State-only specifications are parsimonious but can miss real duration or history dependence. Duration-dependent models represent changing risk with time in state but may still omit informative past history. History-dependent models can represent richer patterns, but require defensible history variables, adequate data, and careful interpretation. A more elaborate specification is not automatically more accurate, particularly when its added terms are weakly supported.
Common Mistake
Check Your Understanding
Answer each question before reading the short solutions.
- In an illness-death process, name the allowed transitions when illness can occur before death or death can occur directly from the healthy state.
- Two groups in state \(1\) have different observed death rates, but one group was observed earlier in calendar time. Why is this insufficient evidence for duration dependence?
- At the same calendar time, people in state \(1\) with durations of one month and eight months show consistently different transition rates. Which memory structure should be considered, and what comparison condition matters?
- What additional condition must be met before a past-history variable supports an event-history-dependent specification?
Solutions. (1) The transitions are \(0\to1\), \(0\to2\), and \(1\to2\). (2) Calendar-time variation could explain the difference; compare at the same calendar time or account for it. (3) Consider a duration-dependent specification, ensuring the duration groups are compared at the same calendar time and in the same origin state. (4) The history variable must add predictive information after accounting for current state, duration, and calendar time.