Continuous-Time Markov Likelihoods for Panel Data

From Continuous-Time Rates to Panel Observations

In panel data, a subject’s state is observed at scheduled visits, but changes between visits are not usually recorded. A subject observed in state 0 at one visit and state 1 at the next may have changed state at any point during the gap. A continuous-time Markov model handles this by assigning transition probabilities to the observed state pairs over the actual elapsed times. The likelihood contribution is therefore a probability of the observed endpoints, not a probability for one particular unobserved path.

The central calculation is the matrix exponential of a generator. Its entries give the probabilities of moving between states over a specified time interval. When gaps differ across subjects or visits, the same generator is used to calculate a different transition matrix for each gap.

Definition: For a finite-state, time-homogeneous continuous-time Markov model, the generator \(Q=(q_{rs})\) has off-diagonal entries \(q_{rs}\geq 0\), where \(q_{rs}\) is the instantaneous transition intensity from state \(r\) to state \(s\). Each diagonal entry is \(q_{rr}=-\sum_{s\ne r}q_{rs}\), so every row sums to zero. Under the row-oriented convention, the transition-probability matrix over elapsed time \(t\) is \(P(t)=\exp(Qt)\), and \(P_{rs}(t)\) is the probability of being in state \(s\) after time \(t\), conditional on starting in state \(r\).

The entries of \(Q\) have units of inverse time, such as per year, while \(P(t)\) is dimensionless. The time \(t\) must use the corresponding unit: if intensities are per year, an interval of six months is \(t=0.5\) years. Time homogeneity means that the generator does not change during the interval. The Markov assumption means that, conditional on the current state, the future evolution does not additionally depend on the earlier path.

A valid transition matrix has nonnegative entries and rows that sum to one. Its diagonal entries represent remaining in the same state at the end of the interval; they do not mean that no transitions occurred, since a subject can leave and return to that state between visits in models that permit return.

Why the Matrix Exponential Appears

Let \(P(t)\) denote the transition matrix over an interval of length \(t\). Over a short increment \(h\), the probability of one transition from \(r\) to \(s\ne r\) is \(q_{rs}h+o(h)\), and the probability of remaining in \(r\) is \(1+q_{rr}h+o(h)\). The Markov property gives \(P(t+h)=P(t)P(h)\). Subtracting \(P(t)\), dividing by \(h\), and taking the limit yields the forward equation and its initial condition:

\frac{dP(t)}{dt}=P(t)Q,\qquad P(0)=I.

The matrix exponential \(\exp(Qt)=I+Qt+(Qt)^2/2!+\cdots\) solves this system. For example, the \(Qt\) term describes contributions from one transition over a short interval, while higher-order terms include sequences of transitions. This explains why an endpoint probability can include paths that contain unobserved intermediate changes.

Worked Example: A Two-State Transition Matrix

Suppose a hypothetical process has states 0 and 1, with intensity \(0.2\) per year from 0 to 1 and intensity \(0.1\) per year from 1 to 0. Its generator is

Q= \begin{pmatrix} -0.2 & 0.2\\ 0.1 & -0.1 \end{pmatrix}.

The total rate of leaving state 0 or 1 is \(0.2\) or \(0.1\) per year, respectively. Solving the two-state forward equations gives, for \(t\) years,

P_{01}(t)=\frac{0.2}{0.3}(1-e^{-0.3t}),\quad P_{10}(t)=\frac{0.1}{0.3}(1-e^{-0.3t}), $$ $$ P_{00}(t)=1-P_{01}(t),\qquad P_{11}(t)=1-P_{10}(t).

At \(t=1\) year, \(e^{-0.3}\approx0.740818\), so

P_{01}(1)=\frac{0.2}{0.3}(1-0.740818) =0.666667(0.259182)\approx0.1728.

Thus \(P_{00}(1)\approx1-0.172788=0.8272\). The two probabilities in the row for starting state 0 sum to one, as required. This model allows transitions in both directions; the endpoint probability \(P_{00}(1)\) includes any paths that start and end in state 0, including paths with an intermediate visit to state 1.

Likelihood Contributions at Unequal Visit Gaps

Suppose subject \(i\) is observed at times \(t_{i0},t_{i1},\ldots,t_{im_i}\), with observed states \(y_{i0},y_{i1},\ldots,y_{im_i}\). Let \(\Delta_{ij}=t_{ij}-t_{i,j-1}>0\) be the elapsed time between consecutive visits. If the subject follows the same time-homogeneous generator \(Q\) throughout, then the conditional probability of the next observed state, given the current one, is

P_{y_{i,j-1},y_{ij}}(\Delta_{ij}) =\left[\exp(Q\Delta_{ij})\right]_{y_{i,j-1},y_{ij}}.

Conditioning on each subject’s first observed state, the likelihood for independent subjects is the product of these transition probabilities over all consecutive observed pairs. This construction sums over the possible unobserved paths within each gap through the matrix exponential; it does not require assigning a time or sequence to an unseen transition.

L(Q)=\prod_i\prod_{j=1}^{m_i} \left[\exp(Q\Delta_{ij})\right]_{y_{i,j-1},y_{ij}}, \qquad \ell(Q)=\log L(Q) =\sum_i\sum_{j=1}^{m_i} \log\left[\exp(Q\Delta_{ij})\right]_{y_{i,j-1},y_{ij}}.

These expressions are conditional on the starting states. If the initial states are instead modeled as random, their probabilities also contribute to the likelihood. The product assumes independent subjects and, within a subject, the Markov transition structure specified by \(Q\). It also assumes that the observation process does not add unmodeled information about the latent state transitions; for example, the visit schedule should be fixed or appropriately noninformative for the likelihood being used.

Worked Example: A Likelihood With Five Unequal-Gap Observations

Consider the hypothetical two-state model from the previous example. Suppose five independent subjects each contribute one observed pair: \(0\to1\) over 1 year, \(0\to0\) over 2 years, \(1\to0\) over 1 year, \(1\to1\) over 3 years, and \(0\to1\) over 2 years. The likelihood conditional on their observed starting states is

L(Q)=P_{01}(1)P_{00}(2)P_{10}(1)P_{11}(3)P_{01}(2).

Each factor uses the interval for that pair. Using \(e^{-0.3}\approx0.740818\), \(e^{-0.6}\approx0.548812\), and \(e^{-0.9}\approx0.406570\), the probabilities are

P_{01}(1)=\frac{2}{3}(1-e^{-0.3})\approx0.172788, \qquad P_{00}(2)=1-\frac{2}{3}(1-e^{-0.6})\approx0.699208, $$ $$ P_{10}(1)=\frac{1}{3}(1-e^{-0.3})\approx0.086394, \qquad P_{11}(3)=1-\frac{1}{3}(1-e^{-0.9})\approx0.802190, $$ $$ P_{01}(2)=\frac{2}{3}(1-e^{-0.6})\approx0.300792.

Substitution gives

L(Q)\approx (0.172788)(0.699208)(0.086394)(0.802190)(0.300792) \approx0.002519.

This is the probability of these five observed transitions conditional on the specified starting states, under the stated model and rates. It is not the probability that the rates are correct. In actual inference, the likelihood is evaluated over candidate parameter values; this example only evaluates it at the specified intensities.

An Illness-Death Model as a Matrix Exponential

A common structure has state 0 for being well, state 1 for illness, and absorbing state 2 for death. Suppose the only possible transitions are \(0\to1\), \(0\to2\), and \(1\to2\), with intensities \(0.2\), \(0.1\), and \(0.1\) per year. The generator is

Q= \begin{pmatrix} -0.3 & 0.2 & 0.1\\ 0 & -0.1 & 0.1\\ 0 & 0 & 0 \end{pmatrix}.

The diagonal entry \(-0.3\) in the first row is the negative sum of the two possible exit intensities from state 0. For this triangular generator, solving the forward equations gives \(P_{00}(t)=e^{-0.3t}\), \(P_{11}(t)=e^{-0.1t}\), and \(P_{01}(t)=e^{-0.1t}-e^{-0.3t}\). The other probabilities follow from the row sums and the absorbing state.

Worked Example: Computing Panel Probabilities in an Illness-Death Model

For a 2-year gap, the probabilities starting in state 0 are

P_{00}(2)=e^{-0.6}\approx0.548812, \qquad P_{01}(2)=e^{-0.2}-e^{-0.6} \approx0.818731-0.548812=0.269919, $$ $$ P_{02}(2)=1-P_{00}(2)-P_{01}(2) =1-e^{-0.2}\approx0.181269.

They sum to \(0.548812+0.269919+0.181269=1.000000\), up to rounding. Starting in state 1, death is the only possible change, so \(P_{11}(2)=e^{-0.2}\approx0.818731\) and \(P_{12}(2)=1-e^{-0.2}\approx0.181269\). State 2 is absorbing, so \(P_{22}(2)=1\). These are endpoint probabilities over two years, not probabilities that the observed state remained unchanged throughout the gap.

If one hypothetical subject is observed in state 0 and then state 1 after 2 years, this interval contributes \(P_{01}(2)\approx0.2699\) to the likelihood. Another subject observed in state 1 and then state 2 after 2 years contributes \(P_{12}(2)\approx0.1813\). Together, these independent contributions are approximately \((0.269919)(0.181269)=0.04893\), conditional on their starting states.

Assumptions, Checks, and Common Pitfalls

Before using these likelihood contributions, check that the state space and allowed transitions match the scientific model, the generator has nonnegative off-diagonal entries and zero row sums, and time units are consistent. Confirm that observed states are correctly classified and that each positive gap is paired with the correct ordered start and end states. Under a proposed generator, calculated transition matrices should have nonnegative entries and rows summing to one, aside from negligible numerical rounding.

A further model check is whether a single homogeneous generator is plausible over each interval. If transition rates vary with time, or depend on covariates that change between visits, the simple expression \(\exp(Q\Delta)\) for one fixed \(Q\) may not represent the interval. The observation schedule also matters: if visits are prompted by emerging symptoms or other information about the unobserved state, treating the visits as noninformative may be inappropriate without modeling that process.

Common mistake: treating an endpoint pair as a fully observed path. Observing \(0\) and later \(0\) does not prove that no change occurred. Use \(P_{00}(\Delta)\), which includes all paths ending in 0.
Common mistake: using one transition matrix for every gap. The matrix is \(\exp(Q\Delta)\), so its value depends on the elapsed time. Use each pair’s own \(\Delta\).
Common mistake: reversing the state indices. With the row-oriented convention, the row is the starting state and the column is the ending state. Thus \(P_{01}\) means start in 0, end in 1.
Common mistake: confusing a rate with a probability. An intensity such as \(0.2\) per year is not a 20% probability of transition in one year. The relevant interval probability is an entry of \(\exp(Q\Delta)\).

A discrete-time transition model specifies transition probabilities directly at a chosen time step. The continuous-time model instead specifies intensities and obtains endpoint probabilities for any interval length from the generator. This is useful when visit gaps are unequal, but it imposes a continuous-time Markov structure. In particular, having panel observations does not by itself reveal exactly when transitions occurred or guarantee that every intensity can be learned precisely from the available gaps.

Key takeaway: For a time-homogeneous continuous-time Markov model, an observed panel transition from \(r\) to \(s\) over a gap of \(\Delta\) contributes \(\left[\exp(Q\Delta)\right]_{rs}\) to the likelihood. Use the actual interval for every observed pair, and interpret the contribution as an endpoint probability that includes unobserved paths.

Check Your Understanding

Try each question, then compare your reasoning with the solutions.

Under the row-oriented convention, what does \(P_{21}(t)\) represent?
A model has two states with rates \(0.4\) per month from 0 to 1 and \(0.2\) per month from 1 to 0. Write its generator.
Why is \(P_{00}(\Delta)\) generally not the probability of making no transitions during the interval?
For the two-state model in the first worked example, calculate \(P_{00}(2)\) to four significant figures.
Write the likelihood contribution for an observed transition from state 1 to state 2 over a 3-year gap in a three-state model.

Solutions. (1) It is the probability of being in state 1 after time \(t\), conditional on starting in state 2. (2) The generator is \(Q=\begin{pmatrix}-0.4&0.4\\0.2&-0.2\end{pmatrix}\), with rates in inverse months and rows summing to zero. (3) The subject may leave state 0 and return before the next visit, so the endpoint event includes paths with intermediate transitions. (4) \(P_{00}(2)=1-\frac{2}{3}(1-e^{-0.6})=1-\frac{2}{3}(1-0.548812)\approx0.6992\). (5) The contribution is \(\left[\exp(3Q)\right]_{12}\), with the time unit and intensity units matched; it is the probability of the observed endpoint given the starting state under that model.