Designing a State Space for a Study Question

Design the State Space Around the Question

A state space is not simply a list of diagnoses. It is a formal description of the clinical situations a person can occupy during follow-up and the changes between them. Its design determines what the resulting model can describe—and what it cannot. A useful design therefore starts with the study question, not with whichever variables happen to be available in a data set.

For example, “Does treatment improve outcomes?” is not yet specific enough to define a state space. The outcome might mean remission, time alive with a functioning graft, or the chance of hospital readmission before death. Each question may require different states, a different time origin, or a different treatment of later events. The earlier material on state diagrams and path feasibility provides the basic language; here the emphasis is on making the design choices systematically for a new question.

Definition: A state space is a set \(S\) of distinct states used to represent an individual’s clinical status over follow-up. The individual’s state at time \(t\) is denoted by \(X(t)\). A transition is a change from one distinct state to another. The set of allowed directed transitions is denoted by \(E\), where \(E\subseteq S\times S\). A state-space design also specifies the target population, the time origin, and which parts of prior history, if any, must be represented to answer the study question.

In a standard design, the states are defined so that an individual occupies one and only one state at any given time during the specified follow-up. The states should collectively cover the clinical statuses that the study intends to represent. “One and only one” rules out overlapping labels—for example, defining states as “has disease” and “has been hospitalized” if a person can satisfy both at once. Such a question may require separate dimensions or a combined set of states.

The target population is the group about whom the study question is asked, described using eligibility criteria that can be applied at a clearly specified time. The time origin is the point from which follow-up time is measured, such as treatment initiation or discharge from an index admission. Changing either can change the population represented and the meaning of the state transitions.

A Practical Design Sequence

A useful state space preserves distinctions that matter to the question while avoiding distinctions that add no useful interpretation. Work through the following decisions in order.

State the target question and population.
Specify the clinical outcome or process of interest and who is eligible. Make clear whether eligibility is defined at diagnosis, treatment initiation, discharge, or another landmark.

Choose time zero and the follow-up horizon.
Define when observation begins and when it ends. Decide whether the question concerns the first event only, repeated episodes, or the course after an event.

Write clinically meaningful state definitions.
Check that each person has a single applicable state at each time and that the states cover the statuses relevant to the question. State definitions should be operational enough to apply consistently to records.

List allowed transitions and their direction.
For every state, ask which distinct next states are clinically possible under the study’s definitions and follow-up. Distinguish an impossible transition from a transition that is merely uncommon.

Check whether the present state retains enough history.
Ask whether two people assigned the same state, but with different relevant prior histories, could have different future behavior. Retain historical detail when it is needed for the study question or for a defensible model assumption.

Historical Detail and the Markov Question

The current state may or may not summarize the parts of the past that matter for future transitions. The Markov property, introduced in the earlier material on the Markov property for state processes, says that conditional on the present state, the future state distribution does not additionally depend on the observed past. State design can make this assumption more plausible by retaining relevant history, but adding detail does not guarantee that the assumption holds.

One way to assess a proposed state definition is to consider the mapping from a person’s observed history \(H(t)\) to a state label \(g\{H(t)\}\). If two clinically different histories receive the same label, ask whether their future behavior needs to be treated as the same for the intended analysis. In a Markov model, a key diagnostic question is whether histories assigned to the same present state imply the same conditional future probabilities.

P\{X(t+u)=j\mid H(t)=h_1\} \quad\text{versus}\quad P\{X(t+u)=j\mid H(t)=h_2\}, \qquad g(h_1)=g(h_2).

If these probabilities differ in a way that matters, the design may need a more detailed state, a relevant time scale, or additional explanatory information. Conversely, splitting a state just because two subgroups can be named is not automatically useful. Extra states can make the clinical interpretation less direct and leave some transitions with little information. The design decision balances a meaningful representation of history against the question the study can realistically answer.

Worked Example: Remission and Relapse After Treatment

Hypothetical study question. Among adults beginning a specified treatment for an active disease, what clinical courses occur during the next two years, including remission, relapse, and death? Time zero is the date treatment begins. Suppose the study records whether disease is active or in remission at each assessment and the date of death, if it occurs.

Population and states. The target population is adults who meet the treatment eligibility criteria and begin treatment. A suitable initial state space is \(S=\{\text{active disease},\text{remission},\text{death}\}\). Before death, the disease-status definitions must assign each person to exactly one of the first two states. Death is treated as an absorbing state during this follow-up: after entering it, the person does not return to a living disease-status state.

Transitions. The allowed transitions are active disease to remission, active disease to death, remission to active disease (relapse), and remission to death. The allowed-transition set is therefore \(E=\{(\text{active},\text{remission}),(\text{active},\text{death}),(\text{remission},\text{active}),(\text{remission},\text{death})\}\). A direct remission-to-death transition is included even if the death is unrelated to the disease; the state space represents status changes, not only disease-caused events.

Check the historical detail. The proposed states do not distinguish a first remission from a later remission. That is reasonable if the question concerns the overall course and the study does not need remission number to interpret future changes. If relapse risk is specifically to be compared after first versus later remission, the state definition may need to retain that distinction, or the analysis may need another explicit way to represent it. The choice follows the question rather than a rule that every past event must become a new state.

First-Event Questions and Recurrent Courses

A first-event question and a question about the full clinical course need not use the same state space. If the target is only the first event after a defined landmark, the event states can be terminal for the analysis: later clinical changes are outside that estimand. If the target includes repeated events or outcomes after the first event, a state space that stops at the first event discards information needed for the question.

This is a design distinction, not a claim that one representation is universally superior. A first-event representation can give a clean answer about which event occurs first, but it cannot describe transitions after that event. A recurrent multi-state representation can describe returns and later episodes, but requires clear definitions for recurrent states and for how episodes are counted.

Worked Example: Readmission Before Death or the Full Hospital Course

Hypothetical study question A. Among adults discharged alive after an index hospital admission, what is the first event within 90 days: readmission or death before readmission? Time zero is the index discharge. Define \(S=\{\text{alive and not yet readmitted},\text{readmitted},\text{death before readmission}\}\). The two event states are terminal for this first-event question. Allowed transitions from “alive and not yet readmitted” are to either event state; there are no transitions out of the event states in this analysis.

Suppose a hypothetical participant is discharged on day 0, remains alive without readmission through day 18, and is readmitted on day 19. The state path is “alive and not yet readmitted” through day 18, followed by “readmitted” on day 19. A second participant who dies on day 12 without readmission moves from “alive and not yet readmitted” to “death before readmission.” Death after readmission is not represented as a subsequent transition here, because the question ends at the first event.

Hypothetical study question B. If the aim instead is to describe repeated admissions and discharges, the first-event space is insufficient. A broader space might distinguish “in hospital,” “alive outside hospital,” and “dead,” with hospital-to-outside-hospital, outside-hospital-to-hospital, and transitions to death as clinically allowed. This represents repeated admissions, but it does not by itself distinguish an index admission from a later admission. That distinction should be added only if it matters to the question or interpretation.

When to Refine a State

A state may be too coarse when it combines histories that need different interpretations. For instance, “functioning graft” could combine a functioning first graft with a functioning retransplanted graft. If the study asks about the course following a first transplant regardless of graft generation, that combination may be acceptable. If it asks whether outcomes differ by graft generation, the state space or accompanying model information must preserve that distinction.

A state can also need a time-related refinement. Suppose prognosis depends on how long a person has been in the current state. The label “remission” alone does not encode whether remission began one week or one year ago. Depending on the question and modeling approach, elapsed time since entry may need to be represented explicitly or handled through an appropriate time scale. Merely renaming the state does not solve a duration-dependence problem.

The design should also distinguish “not observed” from a clinical state. Loss to follow-up or administrative study end is usually an observation or censoring event, not a clinical status such as remission or hospitalization. Treating absence of later records as a state change can create transitions that did not occur clinically.

Worked Example: Functioning Grafts and Retransplantation

Hypothetical study question. Among adults receiving a first transplant, describe transitions among a functioning first graft, graft failure, a functioning subsequent graft, and death during follow-up. Suppose the study wants to distinguish whether the currently functioning graft is the first one or a later one, but does not aim to compare second and third grafts separately.

Define \(S=\{\text{functioning first graft},\text{no functioning graft after failure},\text{functioning subsequent graft},\text{death}\}\). A clinically coherent set of allowed transitions is functioning first graft to no functioning graft or death; no functioning graft to functioning subsequent graft or death; functioning subsequent graft to no functioning graft or death. Death is absorbing. Under these definitions, the return from “no functioning graft” to “functioning subsequent graft” represents retransplantation.

Now consider a hypothetical participant whose first graft functions from day 0 to day 120, fails on day 120, and is replaced on day 175. The assigned path is functioning first graft, then no functioning graft, then functioning subsequent graft. If the subsequent graft fails later, the participant returns to no functioning graft. The state “functioning subsequent graft” deliberately groups later graft generations: that is consistent with the stated question, but it would not support a comparison of second versus third graft outcomes. If that comparison were the target, the states would need to retain generation-specific information.

Design Checks, Alternatives, and Limitations

Before analysis, test the design against clinical histories rather than relying only on the diagram. For each proposed state, ask whether two reviewers would assign the same label from the same record. For each proposed transition, ask whether the event order and clinical definitions make it possible. Then walk through representative histories that include recovery, relapse, repeat events, death, and incomplete observation where relevant.

Design choice	Useful when	What it leaves out or risks
First-event states	The question concerns which of a specified set of events occurs first after a landmark.	Does not describe later states or recurrent episodes.
Recurrent multi-state states	The question concerns movement among clinical statuses over time, including returns or repeat episodes.	Requires explicit definitions for repeated transitions and may create sparse or complex pathways.
Refined states retaining history	A past event or episode changes the interpretation needed for the question or the future behavior assumed by the model.	Can fragment the data and make state-specific results difficult to estimate or communicate.

Common warning signs include a state definition that combines statuses a person can occupy simultaneously, a permitted transition that violates the protocol’s event order, and a “death” state that is not treated as absorbing despite death being terminal. Another warning sign is a very detailed state space with many clinically implausible or scarcely observed paths. These problems can often be found during design review, before fitting any model.

No state-space design can repair inaccurate event dates, inconsistent clinical definitions, or an unsuitable target population. Nor does a plausible diagram establish that the Markov assumption holds. The assumption concerns the conditional behavior of the process given the present state; the state definitions help specify what that assumption means. The final design should therefore be judged both by whether it answers the clinical question and by whether its assumptions can be defended.

Key takeaway: Define the population and time origin first; then choose states, transitions, and historical detail to match the question. A good state space is clinically interpretable, operationally assignable, and no more detailed than the study’s purpose requires.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

A study asks about first relapse or death after remission. What information must be specified before proposing states?
Why might “alive” and “hospitalized” fail as two mutually exclusive states?
A study of readmission stops follow-up at the first readmission or death. Why should it not use that state space to describe deaths occurring after readmission?
When could “functioning graft” need to be split into first-graft and subsequent-graft states?
Give one reason to avoid adding a separate state for every recorded episode number.

Solutions. (1) Specify the eligible population, the point at which remission is established and follow-up begins, the follow-up horizon, and the operational definitions of relapse and death. (2) A person can be alive and hospitalized at the same time, so the labels overlap; the design needs mutually exclusive combined statuses or a different representation. (3) The analysis ends at the first event, so post-readmission death is outside its target question and cannot be described by its terminal readmission state. (4) Split the state when graft generation is part of the question or when histories by generation must be distinguished for the intended interpretation or model. (5) Extra episode-specific states can fragment the data and complicate interpretation when episode number is not relevant to the research question.