Multistate Intermediate Outcomes

Why Represent Intermediate Outcomes as States?

An intermediate outcome can be more than a single response indicator or event time. A participant may respond, later progress, and eventually die; another may progress without responding. At an interim look, a summary that records only whether response ever occurred can hide these different paths. A multistate representation preserves the participant’s current clinical status and makes the possible transitions explicit.

The goal here is to construct arm-level summaries from those states for interim comparison and selection. These summaries describe what has been observed by a specified landmark. They do not, by themselves, predict the final endpoint or determine whether an arm should be retained.

Define the States Before Estimating Them

Consider a trial in which participants begin without documented response or progression. One useful state definition is:

• State 0: alive, without progression, and not currently in response.
• State 1: alive, without progression, and currently in response.
• State 2: alive after progression, whether or not response occurred earlier.
• State 3: dead.

These states are mutually exclusive and cover the possible statuses under this definition. Response may be gained or lost before progression, so transitions \(0\to1\) and \(1\to0\) are allowed. Progression is treated as irreversible, and death is absorbing: once in state 3, a participant remains there. If a protocol defines response as “ever responded,” response loss would not lead from state 1 to state 0; state definitions must match the endpoint and protocol.

For treatment arm \(a\), let \(\lambda_{ij}^{(a)}(t)\) denote the instantaneous transition rate from state \(i\) to state \(j\) at time \(t\), among participants in state \(i\) just before \(t\). For example, \(\lambda_{12}^{(a)}(t)\) describes progression among participants currently responding without progression. The possible transitions in this illustration are \(0\to1\), \(0\to2\), \(0\to3\), \(1\to0\), \(1\to2\), \(1\to3\), and \(2\to3\).

This transition-rate matrix has nonnegative off-diagonal entries, and each row sums to zero. A zero entry represents a transition excluded by the state definitions, not evidence that the transition is merely rare. If a clinically possible path is omitted, the states or allowed transitions need reconsideration.

From Transition Rates to Arm-Level Summaries

Write \(\mathbf p^{(a)}(t)\) for the row vector of probabilities of occupying states 0 through 3 in arm \(a\) at time \(t\). In a continuous-time Markov model, the future transition distribution depends on the current state, time, and any specified covariates, rather than on additional unrecorded history. The forward equation is

If all participants start in state 0, then \(\mathbf p^{(a)}(0)=(1,0,0,0)\). Solving the forward equation gives the probabilities of occupying each state at a later time. These probabilities sum to one. For an interim look at time \(t^*\), the principal summaries are \(p_1^{(a)}(t^*)\), the probability of currently being alive, unprogressed, and responding; \(p_2^{(a)}(t^*)\), the probability of being alive after progression; and \(p_3^{(a)}(t^*)\), the probability of death by that time.

With observed transition histories and right-censoring, a standard nonparametric estimator is the Aalen–Johansen estimator. At each observed transition time, estimate the transition increment from the number of observed transitions \(i\to j\) divided by the number at risk in state \(i\). Assemble these increments in a matrix \(d\widehat A(t)\), placing the negative sum of outgoing increments on each diagonal. The product integral accumulates the increments over time:

The product is ordered in time. This construction extends the competing-risks cumulative-incidence approach to multiple states and transitions. It estimates state-occupation probabilities; the entries of the transition-probability matrix can also summarize movement between states over a chosen interval.

Worked Example: Occupancy Probabilities From Two Observation Intervals

Compare Arms Without Collapsing the State Structure

At a common interim landmark, report arm-specific state probabilities and, where useful, their differences. The contrast is descriptive of the probability of occupying a state at that landmark. It is not automatically a causal effect or a forecast of the final endpoint; interpretation depends on randomization, follow-up, and the precise estimand.

A related summary is restricted time spent in a state up to a landmark \(\tau\). For state \(j\), define \(R_j(\tau)=\int_0^\tau p_j(t)\,dt\). This is the expected time in that state during the interval, in units of time. It summarizes the whole occupancy trajectory rather than only status at one instant. If probabilities are estimated at discrete times, numerical integration is required and its approximation should be reported.

Derivation: Why Occupancy Probabilities Add Up

The forward equation follows by considering a short interval of length \(h\). Conditional on being in state \(i\) at time \(t\), the probability of a transition \(i\to j\) during that interval is approximately \(\lambda_{ij}(t)h\), while the probability of remaining in \(i\) is approximately one minus the sum of its outgoing transition probabilities. Thus, to first order, the probability vector changes by \(\mathbf p(t)Q(t)h\). Dividing this change by \(h\) and taking \(h\) toward zero gives \(d\mathbf p(t)/dt=\mathbf p(t)Q(t)\).

Because every row of \(Q(t)\) sums to zero, the derivative of the total occupancy probability is

where \(\mathbf 1\) is a column vector of ones. If the initial probabilities sum to one, they continue to sum to one. Nonnegative transition rates also ensure that the model moves probability only through the specified transitions. This provides a basic numerical check when calculating or implementing the model.

Diagnostics, Alternatives, and Limitations

Check the state definitions, data, and estimated trajectories before using a summary for interim decisions. Death occupancy is nondecreasing when death is absorbing: participants may enter the death state, but cannot leave it. An apparent decrease in estimated death-state probability is a warning sign for coding, estimation, or presentation errors. By contrast, the probability of being alive in the progression state can decrease as participants die, and the response-state probability can decrease after response loss or progression. A decreasing occupancy probability in either of those states is not, by itself, evidence of a problem.

Other useful checks include verifying that state probabilities are between zero and one and sum to one at each time, inspecting whether observed paths include transitions forbidden by the protocol, and checking the numbers at risk for each transition. Sparse transitions can produce unstable estimates, especially for uncommon paths such as death directly from response. Also examine whether censoring differs in ways related to prognosis: if participants with worsening status are more likely to leave observation, standard estimates may be biased unless the censoring mechanism is appropriately addressed.

A competing-risks analysis is suitable when the question concerns first occurrence of mutually exclusive event types, such as progression before death versus death before progression. It does not, by itself, describe later transitions after a first event. A binary response summary is simpler and may be sufficient when only response at a prespecified time matters, but it discards progression and death paths. Multistate summaries retain these paths at the cost of more detailed data requirements, more possible transitions, and potentially less precision. They also rely on the chosen states being clinically meaningful; an omitted history that affects transition patterns can make a simple Markov interpretation misleading.

A common mistake is to treat response, progression, and death as three independent binary outcomes and compare their percentages as though they were unrelated. They are linked statuses in a participant’s trajectory, and at a fixed time the defined states are mutually exclusive. The correction is to specify the states and permitted transitions first, estimate the joint state-occupation probabilities, and interpret arm contrasts as comparisons of those probabilities at the stated landmark.