Why a State Sometimes Needs More History
A state label is useful when it summarizes the information needed to describe what can happen next. But a scientifically convenient label can be too broad. Two people may both be recorded as being in remission, for example, even though one has never relapsed and the other has relapsed before. If their subsequent risks differ, the shared label hides information relevant to future transitions.
Refining states means replacing a coarse label with more specific labels that record a relevant feature of the observed history. The goal is not to preserve every past detail. It is to distinguish histories when that distinction matters for future transition behavior, while avoiding distinctions that do not add useful information.
This is a modeling criterion, not a guarantee supplied by relabeling. If \(H_t\) denotes the observed history up to time \(t\), the criterion can be expressed in terms of transition rates: for each possible destination \(j\), the instantaneous rate from refined state \(i\) must be determined by \(i\) and \(t\), not by additional details in \(H_t\). In a model with constant rates, it must be determined by \(i\) alone.
Here \(q_{ij}\) is the instantaneous transition rate from state \(i\) to state \(j\). Informally, over a short interval of length \(h\), the chance of moving from \(i\) to \(j\) is approximately \(q_{ij}h\), with the chance of two or more transitions negligible relative to \(h\). This rate-based check is a practical way to ask whether a proposed state label has retained enough history.
A Worked State-Splitting Calculation
Worked Example: Separating Remission Before and After Relapse
Consider a hypothetical disease model with an absorbing death state and a remission state. A coarse model uses one label, \(R\), for everyone in remission. Suppose clinical knowledge suggests that the next transition rates depend on whether a person has ever relapsed. Refine remission into \(R_0\), remission with no previous relapse, and \(R_1\), remission after at least one relapse. Assume that, within each refined state, rates are constant and do not depend on other details of the past:
- From \(R_0\), the relapse rate is \(0.08\) per month and the death rate is \(0.01\) per month.
- From \(R_1\), the relapse rate is \(0.20\) per month and the death rate is \(0.02\) per month.
These assumptions are part of the hypothetical model, not facts about a real disease. In each state, relapse and death are competing destinations. With constant rates \(\lambda_R\) and \(\lambda_D\), the probability that relapse occurs within \(u\) months before death is
This follows because the probability of remaining in the current state through time \(v\) is \(\exp[-(\lambda_R+\lambda_D)v]\); the probability density of relapse at \(v\) is that survival probability multiplied by \(\lambda_R\). Integrating this density from \(0\) to \(u\) gives the displayed formula.
For someone in \(R_0\), over six months, the total exit rate is \(0.08+0.01=0.09\) per month. Therefore,
Thus the model gives a six-month relapse-before-death probability of about \(0.371\) for a person in \(R_0\). For someone in \(R_1\), the total exit rate is \(0.20+0.02=0.22\) per month:
The two conditional probabilities differ substantially. Treating both groups as \(R\) would discard a recorded historical feature that, under the stated model, predicts different future transition behavior. Splitting \(R\) into \(R_0\) and \(R_1\) makes that feature part of the current state. The refinement is sufficient only if people within each of \(R_0\) and \(R_1\) have the same transition behavior after conditioning on the refined state, apart from any modeled calendar-time effects.
Why Aggregating the Refined States Can Lose the Property
A mixture of refined states generally does not behave like a single state with a fixed transition rate. Its composition changes as transitions occur. This can make the transition rate out of the coarse state depend on how people arrived there and how long they have remained.
Worked Example: A Coarse Outpatient State Has a Changing Rate
Suppose a hypothetical model has two outpatient states: \(O_0\), after a first hospital discharge, and \(O_1\), after a later discharge. From \(O_0\), the rates of readmission and death are \(0.10\) and \(0.01\) per month. From \(O_1\), the rates are \(0.30\) and \(0.02\) per month. Assume constant rates within each state. A proposed coarse label \(O\) merges both states.
At a particular time, suppose a cohort in \(O\) is 50% in \(O_0\) and 50% in \(O_1\). Its instantaneous readmission rate averaged over that cohort is
In the next month, people in \(O_1\) leave the outpatient states more quickly. Under the competing exponential exit rates, the probabilities of remaining in each state for one month are \(\exp(-0.11)\approx0.8958\) and \(\exp(-0.32)\approx0.7261\), respectively. Starting with 50 people in each state, the expected numbers remaining are \(50(0.8958)=44.79\) and \(50(0.7261)=36.31\). The proportion of survivors in \(O_1\) is therefore
The readmission rate averaged over these survivors is now
The rate changed from \(0.200\) to \(0.1895\) because the mixture changed, even though the rates within \(O_0\) and \(O_1\) stayed constant. More directly, a person known to be in \(O_0\) has an immediate readmission rate of \(0.10\), while a person known to be in \(O_1\) has a rate of \(0.30\). The single label \(O\) does not distinguish those histories. Keeping \(O_0\) and \(O_1\) separate preserves the information needed by the specified model.
The mixture calculation also explains why it is not generally valid to replace the two rates with one average and then use that average as though it remained constant. A rate averaged at one time depends on the current proportions in each refined state; those proportions can change over time. A fixed average rate is justified only under additional conditions, not merely because the groups share a coarse label.
How to Choose and Check a Refinement
A refinement should be tied to a specific scientific reason for expecting transition behavior to differ. First identify the coarse state and the earlier event or history feature that may be relevant. Then define labels that can be assigned consistently from each person’s observed history. Finally, specify the transitions out of each refined state and ask whether any remaining history could still alter those transitions.
For example, identify the shared clinical condition currently recorded, such as remission or outpatient status.
Choose a clearly defined event or category, such as a prior relapse or a first versus later discharge. State how it is determined from the history.
Define mutually exclusive categories that cover the relevant individuals. Check that every allowed transition into and out of each category is coherent with the event history.
Ask whether transition rates could still vary with other past events, time since entry, or time spent in the state. If so, the proposed labels may not be sufficient.
For example, “ever relapsed” must be updated when a relapse occurs, and the permitted state transitions must reflect that update. A state diagram that allows someone to move from “after relapse” back to “never relapsed” would contradict the definition. Similarly, if the historical feature cannot be reliably observed, the refined labels may not be available for the intended analysis.
In data, a possible diagnostic is to compare observed transition patterns across people with the same proposed refined label but different residual histories. If a remaining history feature continues to predict transitions, the refinement has not captured all relevant history. This check can be difficult with sparse transitions, so apparent similarity is not proof that the rates are equal. The scientific rationale, data support, and uncertainty all matter.
Worked Example: When Splitting by Prior Event Is Not Enough
Suppose, in a hypothetical study, the rate of a transition from “disease-free after treatment” rises with time since treatment ends. A modeler splits that state into \(F_0\), no previous relapse, and \(F_1\), previous relapse, but within \(F_0\) the transition rate is still \(0.02\) per month during the first month and \(0.08\) per month later. Consider two people both in \(F_0\), one in the first month and the other later.
Both people have the same proposed refined label, yet the stipulated rates are \(0.02\) and \(0.08\) per month. Since \(0.02\ne0.08\), the relapse-history split alone does not summarize the information needed for the rate. The model would need to represent time since treatment, use a time-dependent rate, or use another structure that accounts for duration. Merely adding the prior-relapse indicator cannot remove dependence on an omitted time feature.
A coarse duration category such as “first month” versus “later” may be useful if rates are reasonably constant within those categories and the transitions at the boundary are represented correctly. But if risk continues to change within “later,” that split is only an approximation. Its adequacy must be judged against the model’s purpose and the observed data.
Alternatives, Limitations, and a Common Mistake
State refinement is one way to represent history dependence, but it is not the only one. A model can retain a coarser state and allow rates to vary with time or observed covariates. A semi-Markov approach can represent dependence on time spent in a state. A model with additional history-dependent predictors can distinguish particular past events without making every combination a separate state. These alternatives answer different modeling needs; the choice depends on which information is available and how transition behavior is expected to vary.
Refining states also has costs. More labels mean more transition rates to estimate, and some transitions may be rare or absent in the data. Overly detailed labels can produce imprecise estimates and difficult-to-interpret diagrams. Conversely, a coarse label may hide important differences. The refinement should therefore be no more detailed than needed to represent the relevant history, but not so coarse that materially different transition patterns are combined.
- Common mistake: “The current clinical condition is the same, so the future transition rates must be the same.” This conclusion ignores the possibility that prior events alter future behavior.
- Correction: Check whether the histories grouped under that condition imply the same future transition behavior. If not, include a relevant historical feature in the state or represent its effect another way.
- Another mistake: Assuming that any split automatically restores the Markov property. A split only helps for the history it records; dependence on duration or other omitted history can remain.
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- A coarse “remission” state combines people who have and have not previously relapsed. What information does splitting it into \(R_0\) and \(R_1\) retain?
- Two refined states have relapse rates \(0.04\) and \(0.12\) per month. If each makes up half of a cohort, what is the cohort’s instantaneous average relapse rate at that time?
- Why can this average rate change as time passes, even if the rates within the refined states are constant?
- A proposed state split records prior relapse, but the rate also depends on time since treatment. Does the split alone establish a Markov description? Explain.
Solutions. (1) \(R_0\) records no prior relapse and \(R_1\) records at least one prior relapse, so the current label includes that historical distinction. (2) The average is \(0.50(0.04)+0.50(0.12)=0.080\) per month. (3) People in the two states can leave at different rates, changing their proportions in the cohort and therefore changing the mixture average. (4) No. People with the same refined label can still have different transition rates because their times since treatment differ; that dependence must also be represented or otherwise modeled.