How State Aggregation Creates Non-Markov Behavior

Why Combining States Can Change the Process

A multi-state model may distinguish states that are meaningful biologically or clinically but indistinguishable in a particular data set. For example, two unobserved disease phases might both be recorded as “ill.” Combining them simplifies the observed state space, but it can also remove information that predicts what happens next. Even if the detailed process has constant transition rates and is Markov, the process recorded after states are combined need not be Markov.

The mechanism is selection among hidden states. If the constituent states have different exit risks, people who remain in the combined state for longer become increasingly concentrated in the lower-risk constituent state. The exit risk for the combined state can therefore change with time spent there. Alternatively, distinct observed histories can imply different probabilities of currently occupying each hidden state, producing different future transition risks despite the same recorded state.

Definition: Let $X_t$ be a continuous-time Markov chain on a finite set of detailed states, with generator $Q=(q_{ij})$. An aggregation maps each detailed state $i$ to an observed block $g(i)$; the recorded process is $Y_t=g(X_t)$. The aggregated process is Markov if, conditional on its observed state at time $t$, its future distribution does not additionally depend on its earlier observed history.

The detailed chain has transition rate $q_{ij}$ from $i$ to $j$, for $i\ne j$, and $q_{ii}=-\sum_{j\ne i}q_{ij}$. The rates describe the detailed process, not necessarily the aggregated one. The central question is whether the hidden distinctions matter for transitions between observed blocks.

A Mixture of Exit Risks Changes With Time

Consider two detailed states, 1 and 2, that are both recorded as aggregate state $A$. From either detailed state, the only possible transition is to an absorbing state $B$. Suppose the rates are $\lambda_1$ from state 1 and $\lambda_2$ from state 2. At entry to $A$, let the probabilities of being in states 1 and 2 be $\pi_1$ and $\pi_2$, where $\pi_1+\pi_2=1$.

Conditional on starting in detailed state $i$, the probability of still being in $A$ after duration $t$ is $e^{-\lambda_i t}$. Thus the aggregate survival probability and the probability of occupying each hidden state while still in $A$ are

S_A(t)=\pi_1e^{-\lambda_1t}+\pi_2e^{-\lambda_2t}, \qquad w_i(t)=\frac{\pi_i e^{-\lambda_i t}}{S_A(t)}.

The weights $w_i(t)$ are conditional probabilities given that the process has not yet left $A$. The aggregate exit hazard is the weighted mean of the detailed exit rates:

h_A(t)=w_1(t)\lambda_1+w_2(t)\lambda_2 =\frac{\pi_1\lambda_1e^{-\lambda_1t}+\pi_2\lambda_2e^{-\lambda_2t}} {\pi_1e^{-\lambda_1t}+\pi_2e^{-\lambda_2t}}.

This formula follows by differentiating the aggregate survival function: $h_A(t)=-S_A'(t)/S_A(t)$. If both initial weights are positive and $\lambda_1\ne\lambda_2$, the hazard is generally not constant. As duration increases, the higher-rate state is depleted more quickly among those still in $A$, so the lower-rate state receives greater conditional weight.

Key result: Aggregating detailed states with different exit rates can turn an exponential holding time into a mixture of exponential holding times. The mixture generally has a duration-dependent hazard, even though the detailed process is a time-homogeneous Markov chain.

Worked Example: Survivor Selection Changes the Aggregate Hazard

Suppose state 1 exits to $B$ at $0.20$ per month and state 2 exits at $0.80$ per month. At entry to $A$, each detailed state has probability $0.5$. Find the aggregate hazard after two months, and compare it with the hazard at entry.

The probability of still occupying detailed state 1 after two months, before conditioning on remaining in $A$, is $0.5e^{-0.20(2)}$. For state 2 it is $0.5e^{-0.80(2)}$. Therefore

S_A(2)=0.5e^{-0.4}+0.5e^{-1.6} =0.335160+0.100948 =0.436108.

Using the unrounded terms, the conditional weights among survivors are $w_1(2)=0.5e^{-0.4}/S_A(2)\approx0.7685$ and $w_2(2)=0.5e^{-1.6}/S_A(2)\approx0.2315$. Hence

h_A(2)= \frac{0.20(0.5e^{-0.4})+0.80(0.5e^{-1.6})} {0.5e^{-0.4}+0.5e^{-1.6}} \approx 0.3389\ \text{per month}.

At entry, the weights are both $0.5$, so $h_A(0)=0.5(0.20)+0.5(0.80)=0.50$ per month. The aggregate hazard has fallen to about $0.3389$ per month after two months because survivors are more likely to be in the lower-rate state. Using rounded weights in intermediate arithmetic can slightly distort the final result; the displayed hazard is calculated from the unrounded survivor terms.

Different Histories Can Imply Different Risks

Duration dependence is one way aggregation loses the Markov property. A second is history dependence: two individuals can have the same current aggregate state but different hidden-state mixtures because they reached it by different routes. Their next transition distributions can then differ.

Worked Example: Same Recorded State, Different Histories

Suppose the rates from hidden state 1 and hidden state 2 to $B$ are $0.20$ and $0.80$ per month, respectively. Consider two histories that both end in aggregate state $A$. Given history $H_1$, suppose the current hidden-state probabilities are $0.80$ for state 1 and $0.20$ for state 2. Given history $H_2$, suppose they are $0.20$ and $0.80$.

The instantaneous exit rates conditional on these histories are

h_A(0\mid H_1)=0.80(0.20)+0.20(0.80)=0.32\ \text{per month},

h_A(0\mid H_2)=0.20(0.20)+0.80(0.80)=0.68\ \text{per month}.

The difference is also visible in a finite interval. Over the next $0.1$ month, the conditional exit probability after $H_1$ is

0.80(1-e^{-0.20(0.1)})+0.20(1-e^{-0.80(0.1)}) \approx 0.03122.

After $H_2$, it is

0.20(1-e^{-0.20(0.1)})+0.80(1-e^{-0.80(0.1)}) \approx 0.06547.

Both histories give the same current recorded state $A$, but different probabilities of leaving $A$ soon. Therefore the future distribution cannot be determined from $Y_t=A$ alone; the aggregated process is not Markov. The numbers here are hypothetical and illustrate the effect of differing hidden-state mixtures.

When Does Aggregation Preserve the Markov Property?

For a finite-state continuous-time Markov chain, a standard exact condition is strong lumpability. It requires that, for any two detailed states $i$ and $i'$ in the same observed block $A$, the total rate from each state into every observed block $C$ be equal. Rates within the current block are not included as exits from that block.

Definition: The partition into observed blocks is strongly lumpable if, for every pair $i,i'\in A$ and every block $C\ne A$, $$ \sum_{j\in C}q_{ij}=\sum_{j\in C}q_{i'j}. $$ When this holds, the aggregate transition rate from block $A$ to block $C$ is the common value of these sums, independent of the detailed state occupied within $A$.

The condition explains why matching only the total rate of leaving a block is insufficient when there are multiple destination blocks. The rates into each destination block must match separately. Otherwise the overall holding-time distribution may look the same while the identity of the next observed state still carries information about the hidden state.

Worked Example: An Aggregation That Is Exactly Markov

Let detailed states 1 and 2 form aggregate state $A$, and let $B$ and $C$ be absorbing blocks. From either state 1 or state 2, the rate to $B$ is $0.20$ per month and the rate to $C$ is $0.30$ per month. Assume these are the only exits from states 1 and 2.

For both detailed states, the rate into block $B$ is $0.20$, and the rate into block $C$ is $0.30$. The lumpability condition holds for each destination block. Regardless of the hidden mixture within $A$, the total exit rate is $0.50$ per month and the aggregate transition rates are $q_{AB}=0.20$ and $q_{AC}=0.30$.

For example, over an interval of length $\Delta$ months, the aggregate probabilities are

P(Y_{t+\Delta}=A\mid Y_t=A)=e^{-0.50\Delta},

P(Y_{t+\Delta}=B\mid Y_t=A)=\frac{0.20}{0.50}(1-e^{-0.50\Delta}), \qquad P(Y_{t+\Delta}=C\mid Y_t=A)=\frac{0.30}{0.50}(1-e^{-0.50\Delta}).

These probabilities do not depend on which detailed state in $A$ is occupied or on the history before time $t$. Aggregation therefore produces a Markov process on the observed blocks.

By contrast, suppose state 1 has rates $0.10$ to $B$ and $0.40$ to $C$, while state 2 has rates $0.40$ to $B$ and $0.10$ to $C$. Both states have the same total exit rate, $0.50$ per month. But the rates into $B$ and $C$ differ across the hidden states, so strong lumpability fails. A history that changes the hidden-state mixture can change which destination is more likely, even though it does not change the total instantaneous exit rate.

Derivation of the Changing Mixture Hazard

The dependence of the hazard on duration can be shown directly. For two hidden states, write $w_1(t)$ for the conditional probability of state 1 among those still in $A$, and let $w_2(t)=1-w_1(t)$. Differentiating the survivor weight gives

w_1'(t)=w_1(t)\{h_A(t)-\lambda_1\}.

Since $h_A(t)=w_1(t)\lambda_1+w_2(t)\lambda_2$, differentiating the hazard yields

h_A'(t) =w_1'(t)(\lambda_1-\lambda_2) =-w_1(t)w_2(t)(\lambda_1-\lambda_2)^2 \le 0.

Thus, in this two-state model with only exits from $A$, the aggregate hazard is nonincreasing. It decreases strictly at times when both hidden states have positive survivor weight and their exit rates differ. It is constant if the rates are equal or if only one hidden state is possible. This result is specific to the stated mixture model; it is not a claim that every aggregated multi-state transition intensity must decrease.

Modeling Choices, Diagnostics, and Limitations

Aggregation is useful when only coarse states are observed or when a simpler summary is required. Its cost is that an aggregate state may no longer summarize all information relevant to the future. Whether this matters depends on the target: a coarse description may be adequate for some summaries but inadequate for predicting destination-specific transitions or state occupancy over time.

Retain the detailed states. If the constituent states can be observed or credibly represented, keeping them avoids hiding transition-relevant distinctions. Exact and approximate state expansions provide ways to represent such distinctions.
Represent duration explicitly. If the main predictive information is time spent in the current aggregate state, a semi-Markov specification can model duration-dependent holding behavior rather than asserting a constant aggregate rate.
Represent relevant history. If routes, recent transitions, or other observed history alter the hidden-state mixture, a history-dependent model may be more appropriate. Adding a history variable only helps if it captures information relevant to future transitions.

Useful warning signs include an aggregate holding-time distribution that is not exponential, transition risks that differ among histories ending in the same recorded state, and destination patterns that vary despite a common aggregate state. These patterns do not by themselves identify the hidden mechanism: unmeasured covariates, model misspecification, or observation processes can also produce discrepancies. Conversely, failure to detect a difference does not establish exact lumpability, especially with limited data.

Approximate equality of rates into the destination blocks can make an aggregation practically useful, but it does not make the process exactly Markov. Small rate discrepancies may accumulate over repeated transitions or long prediction horizons. The appropriate simplification depends on the intended use, the observable histories, and the consequences of prediction error.

Common Mistake

A frequent mistake is to check only whether all detailed states in an aggregate block have the same total exit rate, then conclude that the aggregate process is Markov. That checks only the holding-time rate. If the constituent states send different rates to different destination blocks, the next destination can still depend on the hidden state and on the history that predicts it. The correction is to compare the rates into each destination block separately, as required by strong lumpability.

Key takeaway: A Markov detailed process does not guarantee a Markov aggregated process. Hidden-state mixtures can change through survival or differ across histories; exact Markov aggregation requires equal rates into every observed destination block from all detailed states within a given block.

Check Your Understanding

Answer each question, then compare your reasoning with the short solutions below.

Two hidden states have exit rates $0.10$ and $0.50$ per week, with equal probabilities at entry to $A$. After one week, calculate the conditional weights among survivors and the aggregate hazard.
State the strong lumpability condition in words. Why must it be checked separately for each destination block?
Suppose the hidden-state weights at time $t$ are $0.25$ and $0.75$, and the exit rates are $0.20$ and $0.80$ per month. Calculate the aggregate instantaneous exit hazard.
Can equal total exit rates guarantee a Markov aggregate if the rates to two destination blocks differ by hidden state? Explain.

Solutions. (1) The unnormalized survivor weights are $0.5e^{-0.10}=0.452419$ and $0.5e^{-0.50}=0.303265$, with sum $0.755684$. Dividing by that sum gives weights approximately $0.5987$ and $0.4013$. The hazard is $(0.10(0.5e^{-0.10})+0.50(0.5e^{-0.50}))/0.755684\approx0.2605$ per week. (2) For any two detailed states in the same observed block, their total rates into each other observed block must match. Separate equality is needed so both the time to exit and the destination distribution are independent of hidden state. (3) $0.25(0.20)+0.75(0.80)=0.6500$ per month. (4) No. Equal total rates control only the overall exit rate; unequal rates into a particular destination block can make the next observed state depend on the hidden-state mixture and hence on history.