Likelihood Construction for Panel-Observed Illness-Death Data

From panel records to likelihood contributions

In illness-death data, a participant may be observed only at scheduled visits, while illness or death occurs between them. The likelihood must reflect what was actually recorded: a state at each visit and, sometimes, that death occurred before the next visit. It must not treat an unobserved illness onset as though its date were known.

Use state 0 for alive without the illness, state 1 for alive with the illness, and state 2 for dead. The illness-death structure allows transitions from 0 to 1, from 0 to 2, and from 1 to 2; state 2 is absorbing, and recovery from state 1 to state 0 is excluded. A panel record describes the state at its observation times, not necessarily every transition between them.

Definition: Let \(X(t)\in\{0,1,2\}\) denote a participant’s state at time \(t\). Write \(P_{ab}(s,t;\theta)=P\{X(t)=b\mid X(s)=a;\theta\}\) for the probability of being in state \(b\) at time \(t\), given state \(a\) at time \(s\), under model parameter \(\theta\). For visits \(v_0<\cdots<v_m\), the observed panel record is the sequence of states recorded at those visits, together with any available death-interval information.

The matrix with entries \(P_{ab}(s,t;\theta)\) has rows indexed by the state at the earlier time and columns by the state at the later time. Each entry is a probability, every row sums to one, and the illness-death structure gives structural zeros for impossible transitions. In particular, a participant observed in state 2 remains in state 2. The matrix for a visit interval can differ from the matrix for another interval if the model allows transition probabilities to vary with time or interval length.

The basic likelihood construction uses three assumptions. First, the state process follows the specified illness-death structure. Second, conditional on the current state and model parameters, the probability of the next observed state is given by the appropriate transition matrix; this is the Markov assumption used for the transition-probability factorization. Third, subjects are independent, and the visit schedule and censoring can be conditioned on without adding a state-process likelihood factor—for example, because the schedule is fixed by design or is non-informative given the model.

The observed-history likelihood

Suppose a participant is observed in states \(x_0,\ldots,x_m\) at visits \(v_0,\ldots,v_m\). Given the initial state, the contribution to the likelihood is the product of the transition probabilities for consecutive observed intervals. The justification is the chain rule followed by the Markov assumption:

\begin{aligned} P\{X(v_1)=x_1,\ldots,X(v_m)=x_m\mid X(v_0)=x_0\} &=\prod_{j=1}^{m}P\{X(v_j)=x_j\mid X(v_0)=x_0,\ldots,X(v_{j-1})=x_{j-1}\}\\ &=\prod_{j=1}^{m}P_{x_{j-1},x_j}(v_{j-1},v_j;\theta). \end{aligned}

If all participants begin in state 0, that initial state is conditioned on and contributes no factor. For independent participants \(i=1,\ldots,n\), multiply their contributions:

L(\theta)=\prod_{i=1}^{n}\prod_{j=1}^{m_i} P_{x_{i,j-1},x_{ij}}(v_{i,j-1},v_{ij};\theta).

The index \(m_i\) allows participants to have different numbers of observed intervals. If a participant is alive at their last observed visit and then lost to follow-up, the record contributes through that visit; no transition factor is added for time after the last known state. If death is known to have occurred after a last visit in state \(a\) and by a specified interval endpoint \(u\), the terminal contribution is \(P_{a2}(s,u;\theta)\), where \(s\) is the last visit time. The endpoint represents the information “dead by \(u\),” not a claim that death occurred exactly at \(u\).

Worked Example: Repeated Visits With No Observed Illness

Consider a hypothetical participant observed in state 0 at times 0, 1, and 2 years. Suppose the fitted transition matrices for the two one-year intervals are

P(0,1)= \begin{pmatrix} 0.70&0.20&0.10\\ 0&0.75&0.25\\ 0&0&1 \end{pmatrix}, \qquad P(1,2)= \begin{pmatrix} 0.60&0.25&0.15\\ 0&0.65&0.35\\ 0&0&1 \end{pmatrix}.

Each row sums to one and the zero entries respect the illness-death structure. The observed history is \(0\to0\to0\), so its likelihood contribution is the product of the state-0-to-state-0 entries for the two intervals:

L_i=P_{00}(0,1)P_{00}(1,2)=0.70\times0.60=0.42.

Thus this history has probability 0.42 under the specified matrices. Under the no-recovery illness-death structure, observing state 0 at both later visits is consistent with remaining free of illness through those visits. It does not establish what happened after the final visit, and the contribution does not include any post-visit follow-up.

Worked Example: Illness Observed, Then Death Between Visits

Now suppose a hypothetical participant is observed in state 0 at year 0, state 1 at year 1, and is known to have died by year 2. Use the same matrices. The first interval contributes the probability of moving from state 0 to state 1:

P_{01}(0,1)=0.20.

The second interval starts with the participant observed ill and ends with death known by year 2. Its contribution is \(P_{12}(1,2)=0.35\). Therefore the complete contribution is

L_i=P_{01}(0,1)P_{12}(1,2)=0.20\times0.35=0.070.

The result is a probability of 0.070 for this observed sequence under the specified model. It does not assert that illness began exactly at year 1: the visit establishes that the participant was ill then, while the onset may have occurred earlier. Nor does “dead by year 2” assert that death occurred at year 2.

Worked Example: Death by the Next Visit With No Illness Recorded

A different hypothetical participant is observed in state 0 at year 0 and is known to be dead by year 1. No visit recorded state 1. The contribution is the state-0-to-state-2 entry of the first interval’s matrix:

L_i=P_{02}(0,1)=0.10.

This is the probability of being in the dead state by year 1, conditional on being healthy at year 0, under the model. The record contains no observed illness. It does not, by itself, establish that illness never occurred between visits: the panel did not observe the intervening history. The likelihood uses the endpoint-state probability, not an invented illness-onset date. If the exact death time, rather than only the death interval, is to be used, this transition-probability contribution alone is not a density for that exact time.

Combining records and checking the model

For a sample, each person contributes the factors corresponding to their observed intervals, and the full likelihood is their product. For example, if the three hypothetical histories above were three independent participants, with the same matrices applying to the stated intervals, their joint likelihood contribution would be

L=0.42\times0.070\times0.10=0.00294.

This product is not the probability that the model is true. It is the likelihood of the observed data as a function of the model parameters. Estimation chooses parameter values that make the observed records relatively likely, subject to the model. If a single transition matrix applies to all participants over the same interval, records can also be grouped by observed transition: the likelihood contains one factor \(P_{ab}\) for each observed \(a\)-to-\(b\) transition. With different interval lengths or time-specific matrices, factors must remain associated with the appropriate intervals.

A practical construction proceeds in a fixed order:

Encode the observed states.
For each subject, list the state at each visit and mark any terminal death interval. Do not add an illness state that was not observed.

Choose the interval matrix.
Match each observed transition to the matrix for its start and end times, including the final death interval when applicable.

Select the observed transition entry.
For an interval from state \(a\) to state \(b\), use \(P_{ab}\). Multiply the entries within each record, then multiply across independent subjects.

Check assumptions and coding.
Confirm that each matrix is a valid transition matrix, structural zeros match the model, and censoring or visit timing is handled as assumed.

Several checks can reveal problems before estimation. Every matrix entry must lie between zero and one, each row must sum to one, and impossible transitions such as 1-to-0 or 2-to-0 must have probability zero under this model. An observed 1-to-0 panel transition is therefore a warning: it may indicate a coding or state-definition problem, a recovery process omitted from the model, or a violation of the assumed state structure. Likewise, a record that shows a participant alive at a visit after a recorded death is inconsistent unless the dates or records have been reconciled.

The observation process also matters. If visits are more likely when symptoms worsen, conditioning on the visit schedule as if it were non-informative may be unjustified. A likelihood based only on the state-transition factors may then fail to account for how records were generated. The same concern applies if loss to follow-up depends on unobserved state history. These are not repaired by multiplying the observed transition entries more carefully; they require an observation or censoring model, or a defensible design-based argument for conditioning on the schedule.

What the likelihood does and does not say

The earlier material on panel-data likelihoods from transition matrices provides the general framework: observed states select entries from matrices over observation intervals. In this illness-death application, the structural state restrictions clarify how to interpret those entries. A contribution \(P_{00}\) describes a healthy-to-healthy observation; \(P_{01}\) describes healthy at one observation and ill at the next; \(P_{12}\) describes ill followed by death; and \(P_{02}\) describes healthy followed by death before the next observation.

A key distinction is between no illness observed and no illness occurred. For a participant repeatedly observed in state 0, the no-recovery illness-death model implies no illness transition before those observations. But when a participant is observed in state 0 and then dead by the next visit, the endpoint record does not reveal whether illness occurred in between. The \(P_{02}\) entry is the appropriate likelihood contribution for that observed record; it should not be replaced by a probability for a known direct transition unless the model and data justify that interpretation.

This approach also differs from analyzing only time to first illness with death as a competing event. A first-event analysis can target illness before death, but it does not by itself represent the observed sequence of illness followed by later death. The illness-death likelihood uses both observed transitions and can therefore use post-illness follow-up. Conversely, replacing an interval-censored illness onset by the date it was first detected changes the data representation and generally assigns more timing information than the visits supplied.

Observed record over an interval	Likelihood factor	What it records
0 at both endpoints	\(P_{00}\)	Healthy at the two observations
0, then 1	\(P_{01}\)	Healthy at the first observation and ill at the second
1, then dead by the endpoint	\(P_{12}\)	Ill at the first observation and dead by the endpoint
0, then dead by the endpoint	\(P_{02}\)	Healthy at the first observation and dead by the endpoint, with intermediate history unobserved

The transition-matrix likelihood is only as useful as its state definitions, transition model, and observation assumptions. It does not recover exact onset times from visit dates, and a death-interval probability is not an exact death-time density. If the scientific question requires the exact timing of death, or if recovery is possible, the likelihood must be modified to match that information or process. Those extensions should not be smuggled into the present likelihood by reinterpreting a panel-state probability.

Common mistake: A participant is healthy at one visit and dead by the next, so the analyst writes a “direct death from health” contribution and concludes that illness did not occur. The correction is to use \(P_{02}\), the probability of the observed endpoint state under the specified model. The panel record does not reveal the intervening path, so it does not establish absence of an unobserved illness episode.

Check Your Understanding

For each question, identify the likelihood factor or explain the relevant interpretation. Solutions are included after the questions.

A participant is observed in state 1 at time \(s\) and in state 1 at time \(t\). Which transition probability is the interval contribution?
A participant is in state 0 at the first visit and known dead by the second. What does the corresponding matrix entry represent, and what does the record not establish?
For the two-interval history \(0\to1\to2\), write the likelihood contribution using interval-specific transition probabilities.
Why should no transition factor be added after a participant’s last observed visit if they are alive but lost to follow-up?

Solutions. (1) The contribution is \(P_{11}(s,t;\theta)\), the probability of being in state 1 at \(t\), given state 1 at \(s\). (2) The factor is \(P_{02}(s,t;\theta)\), the probability of being dead by the second observation given state 0 at the first. The record does not establish whether illness occurred between visits. (3) The contribution is \(P_{01}(v_0,v_1;\theta)P_{12}(v_1,v_2;\theta)\), the product of the observed transition entries for the two intervals. (4) After the last visit, the participant’s state history is unobserved; adding a transition factor would assert information about an unobserved endpoint. The likelihood conditions on the observed record and ends at the last known state under the stated censoring assumption.