Unknown Event Causes Under Intermittent Observation

Why a Missing Cause Changes the Likelihood

With intermittent observation, a participant may be known to be event-free at one visit and found to have experienced an absorbing event at the next. The event time is then known only to lie in an interval. If its cause is also unobserved, the data identify neither the exact time nor which cause-specific event occurred. The likelihood contribution must account for both kinds of missing information.

The central rule is simple: when several mutually exclusive causes could produce the recorded observation, add the probabilities of those possible causes. Do not assign the observation to one cause without evidence, and do not count it once for each possible cause as if it were several events. This distinction determines whether the data inform the overall event risk, cause-specific risks, or both.

Definition: Let \(D_1,\ldots,D_K\) be distinct absorbing event states, with cause \(k\) corresponding to entry into \(D_k\). If an observation records that an event occurred in an interval but does not record its cause, the likelihood contribution is the probability of the union of the compatible cause-specific outcomes. Because the causes are mutually exclusive, this is the sum of their probabilities.

Continuous-Time Likelihood Contributions

First consider a participant known to be event-free at time \(L\), who is known to have experienced one of \(K\) competing events by time \(R\), where \(L<R\). Let \(\lambda_k(t)\) be the cause-specific hazard at time \(t\): conditional on being event-free just before \(t\), \(\lambda_k(t)\,dt\) is approximately the probability of cause \(k\) occurring in the next short interval of length \(dt\). Define the total hazard as \(\lambda(t)=\sum_{k=1}^K\lambda_k(t)\), and let \(S(t)\) be the probability of remaining event-free through time \(t\) from the time origin.

Under the usual continuous-time competing-risks model, the cause-\(k\) event density at time \(t\) is \(S(t)\lambda_k(t)\). Thus the probability that cause \(k\) occurs in \((L,R]\) is the integral of that density over the interval. If the cause is missing, sum these probabilities. The resulting contribution is also the probability of any event in the interval, \(S(L)-S(R)\), because the causes exhaust the possible events and are mutually exclusive.

L_{\mathrm{unknown\ cause}}(L,R) =\sum_{k=1}^K\int_L^R S(t)\lambda_k(t)\,dt =\int_L^R S(t)\lambda(t)\,dt =S(L)-S(R).

This expression assumes that the event-free state and event definition are correctly specified, that the cause-specific hazards describe the event process, and that the observation mechanism does not add unmodelled information about the event or its cause. In particular, if visit timing or cause recording depends on the unobserved cause after conditioning on the modeled history, that observation process may need to be included in the likelihood. For panel data with intermediate states or possible paths, the endpoint probability must account for those paths rather than using a one-state survival expression automatically.

Worked Example: Unknown Cause Under Constant Hazards

Suppose a hypothetical event-free group has two constant cause-specific hazards over a two-year interval: \(\lambda_1=0.08\) per year and \(\lambda_2=0.12\) per year. All participants are event-free at the interval start, and the observation at year 2 establishes that an event occurred, but does not identify its cause. Assume these hazards apply throughout the interval and there are no other event types.

The total hazard is \(\lambda=0.08+0.12=0.20\) per year. Under constant hazards, \(S(t)=\exp(-0.20t)\), with \(t\) measured in years from the interval start. The probability of an event by year 2 is

1-S(2)=1-\exp[-(0.20)(2)] =1-\exp(-0.40) \approx 0.3297.

This is the likelihood contribution for the observed event when its cause is unknown. For comparison, the cause-specific pieces are

\begin{aligned} P(\text{cause 1 in }(0,2]) &=\frac{0.08}{0.20}\{1-\exp(-0.40)\} \approx 0.1319,\\ P(\text{cause 2 in }(0,2]) &=\frac{0.12}{0.20}\{1-\exp(-0.40)\} \approx 0.1978. \end{aligned}

Their sum, \(0.1319+0.1978=0.3297\) up to rounding, is the unknown-cause contribution. The assumptions are satisfied by the stipulated constant hazards and exhaustive two-cause event definition; the calculation would not be justified if a third event type were possible but omitted from the model.

Panel Transition Probabilities and Unobserved Paths

A panel-data model can describe more than a single event-free state followed by an absorbing event. Let \(P_{ab}(\Delta;\theta)\) denote the probability of being in state \(b\) after elapsed time \(\Delta\), given state \(a\) at the start, under model parameter \(\theta\). If observation at the next visit records a terminal event but not which of the possible absorbing states \(D_k\) was reached, the contribution is

\sum_{k=1}^K P_{aD_k}(\Delta;\theta).

For a time-homogeneous continuous-time Markov model with generator matrix \(Q\), the transition matrix over an interval of length \(\Delta\) is \(\exp(Q\Delta)\). The unknown-cause contribution is the sum of the corresponding entries in that matrix. These probabilities include all paths compatible with the endpoints, including paths through intermediate states that were not observed during the interval. A likelihood that instead uses only direct transitions can be wrong when such paths are possible.

Worked Example: Summing Over Causes After Two Panel Intervals

Consider a hypothetical discrete-time model with states \(A\), \(B\), and two absorbing event states \(D_1\) and \(D_2\). In each visit interval the transition probabilities are: from \(A\), to \(A\) with probability 0.6, to \(B\) with probability 0.2, to \(D_1\) with probability 0.1, and to \(D_2\) with probability 0.1. From \(B\), to \(B\) with probability 0.5, to \(D_1\) with probability 0.2, and to \(D_2\) with probability 0.3. Suppose participants are observed in \(A\) initially and, two intervals later, are known to be in an absorbing event state, but the event cause is unavailable. Assume the stated transition probabilities apply in both intervals and the absorbing states remain absorbing.

The observation is compatible with several paths. An event may occur in the first interval, contributing \(0.1+0.1=0.2\). Alternatively, the participant may remain in \(A\) in the first interval and experience an event in the second, contributing \(0.6(0.1+0.1)=0.12\). Or the participant may move to \(B\) and then experience an event, contributing \(0.2(0.2+0.3)=0.10\). Therefore,

P(\text{absorbing event by interval 2}\mid A\text{ at start}) =0.20+0.12+0.10=0.42.

The likelihood contribution is 0.42. As a check, the cause-specific probabilities are \(0.10+0.6(0.10)+0.2(0.20)=0.20\) for \(D_1\) and \(0.10+0.6(0.10)+0.2(0.30)=0.22\) for \(D_2\); their sum is 0.42. The transition probabilities are valid because each row sums to 1, and the calculation includes every path that reaches an absorbing state by the second interval.

What the Data Can Identify

Summing over causes is necessary for a correct likelihood, but it does not create information about which cause occurred. For example, in a one-interval discrete-time model, let \(q_1\) and \(q_2\) be the probabilities of causes 1 and 2, with \(1-q_1-q_2\) the probability of no event. If every recorded event has an unknown cause, the event observation contributes \(q_1+q_2\). Data consisting only of known event status can identify the total event probability, but generally cannot distinguish \(q_1\) from \(q_2\).

Additional information can make cause-specific risks estimable. Examples include reliable cause labels for some or all events; covariates or different observation settings that provide distinct, justified information about the causes; or external information or structural assumptions that constrain the cause-specific risks. Such assumptions must be explicit and scientifically defensible. A fitted model may return separate estimates because its parameterization imposes structure, but that is not the same as the data identifying those estimates without the structure.

Worked Example: Why Unknown Labels Do Not Separate Risks

In a hypothetical discrete-time study, suppose there are 100 participants at risk for one interval. The observed data show 30 events and 70 participants with no event, but none of the 30 event causes is recorded. Under a model with constant interval probabilities \(q_1\), \(q_2\), and \(1-q_1-q_2\), the likelihood, apart from the combinatorial factor for the event count, is

L(q_1,q_2)\propto(q_1+q_2)^{30}(1-q_1-q_2)^{70}.

This likelihood depends on \(q_1\) and \(q_2\) only through their sum. For instance, \((q_1,q_2)=(0.10,0.20)\) and \((q_1,q_2)=(0.15,0.15)\) both imply total event probability 0.30 and give the same likelihood. The observed event frequency informs \(q_1+q_2\), not the separate probabilities. To distinguish them, the study would need additional information, such as cause labels, or a justified model restriction that supplies information beyond these event/no-event observations.

Likelihood Construction, Alternatives, and Pitfalls

For independent participants, the likelihood is the product of each participant's observation contribution. A known cause contributes its cause-specific probability; an unknown cause contributes the sum over compatible causes; a participant with no event through the last observation contributes the probability of the observed non-event history. If a participant has multiple observed visits, use the probability of the full observed state sequence under the model, summing over any unobserved states or causes compatible with that sequence. This is the same likelihood principle applied to a partially observed outcome, not a reason to discard the participant.

The main alternatives answer different questions. A model that combines all causes into one event type can estimate overall event risk, but cannot describe separate cause-specific risks. A cause-labeled analysis uses more information when labels are available, while restricting analysis to labeled events may be biased if cause recording is selective. Assigning unknown causes by a guessed proportion is not a likelihood solution unless that allocation is part of an explicit probabilistic model. The cause-specific hazards themselves are defined relative to remaining free of all event types; they should not be interpreted as risks in a hypothetical setting where competing events have been removed.

A common mistake is to treat an event with missing cause as if it were a cause-1 event, or to multiply the likelihood by both cause-specific probabilities. The first inserts unsupported information; the second treats mutually exclusive outcomes as simultaneous observations. The correction is to sum the probabilities of the mutually exclusive possibilities exactly once. Another warning sign is reporting separately estimated cause-specific risks when the likelihood only depends on their sum: inspect the likelihood's dependence on the parameters and state which information or constraints support separation.

Key takeaway: An unobserved cause is handled by marginalizing over the possible causes: add their probabilities in the likelihood. This produces a valid contribution for the observed data, but separate cause-specific risks require enough cause-specific information or explicit, defensible assumptions.

Check Your Understanding

Answer each question, then compare with the worked solutions below.

Two causes have constant hazards 0.03 and 0.07 per year. What is the probability of an event of unknown cause within one year?
In a discrete-time interval, the probabilities of cause 1, cause 2, and no event are 0.08, 0.12, and 0.80. What contribution does an event with unknown cause make?
Why can event/no-event observations alone fail to identify two cause-specific probabilities?
What likelihood contribution should be used when an event's cause is known to be cause 2?

Solutions. 1. The total hazard is \(0.03+0.07=0.10\) per year, so the contribution is \(1-\exp(-0.10\times1)\approx0.09516\), the probability of any event in one year. 2. Add the mutually exclusive cause probabilities: \(0.08+0.12=0.20\). 3. The event likelihood depends on the probabilities through their sum; different cause-specific pairs with the same sum give the same event/no-event likelihood. 4. Use the probability of cause 2 alone, \(0.12\), rather than summing over causes, because the cause is observed.