Deriving the Stratified Partial Likelihood in Competing Risks

Why the Denominator Must Follow the Transition

In multi-state data, an individual can be eligible for one transition but not another. For example, a patient in one state may be at risk of moving to a second state, while a patient in a different state may be at risk of a different transition. A Cox partial likelihood must compare an observed transition with individuals who could have made that same transition at that time. It must not compare them with everyone in the study indiscriminately.

The key step in deriving the likelihood is to condition on both the observed transition type and its event time. The transition-specific baseline hazard then cancels, while the risk-set denominator remains specific to that transition. The result is a product of conditional contributions, one for each observed transition event.

Model, Notation, and Assumptions

Definition: Let $r$ index a modeled transition and $i$ index individuals or individual risk intervals. Let $Y_{ir}(t)=1$ when individual $i$ is observed and eligible to make transition $r$ just before time $t$, and $Y_{ir}(t)=0$ otherwise. Write $N_{ir}(t)$ for the counting process of observed transitions of type $r$, and $X_i(t)$ for the covariate vector known just before $t$. A transition-stratified Cox model with a common coefficient vector $\beta$ specifies the transition-specific intensity $$ \lambda_{ir}(t \mid \mathcal{F}_{t-}) = Y_{ir}(t)\lambda_{0r}(t)\exp\{\beta^\mathsf{T}X_i(t)\}. $$ Here $\lambda_{0r}(t)$ is an unspecified baseline hazard for transition $r$, and $\mathcal{F}_{t-}$ denotes the observed history immediately before $t$.

This model allows the baseline hazard to vary freely across transitions; it does not require, for example, a move from one state to have the same baseline rate as a move from another. The shared coefficient vector says that the modeled covariate association is common across transitions. The algebra below concerns the stratified likelihood; changing the coefficient parameterization does not change the need for a separate denominator in each transition stratum.

The derivation uses correctly defined risk sets, covariates measured or updated before the event time, and the specified proportional-hazards form within each transition. Observation and censoring must be handled so that individuals are included in a risk set only while observed and eligible. Each transition event must be classified correctly. For the simple product formula below, assume event times are distinct within each transition stratum. Events of different transition types at the same clock time belong to different strata; ties within one transition require a tie method.

Deriving a Single Conditional Contribution

Fix a transition $r$ and an event time $t$ at which exactly one eligible individual makes that transition. The risk set is $R_r(t)=\{j:Y_{jr}(t)=1\}$. Over a sufficiently short interval of length $dt$, the probability that eligible individual $j$ makes transition $r$ is approximately $\lambda_{0r}(t)\exp\{\beta^\mathsf{T}X_j(t)\}dt$. Conditional on one transition-$r$ event occurring at that time, the probability that its individual is $i$ is therefore proportional to that individual’s intensity:

\begin{aligned} P(i\text{ makes transition }r\mid \text{one transition-}r\text{ event at }t,\mathcal{F}_{t-}) &= \frac{\lambda_{0r}(t)\exp\{\beta^\mathsf{T}X_i(t)\}dt} {\sum_{j\in R_r(t)}\lambda_{0r}(t)\exp\{\beta^\mathsf{T}X_j(t)\}dt}\\ &= \frac{\exp\{\beta^\mathsf{T}X_i(t)\}} {\sum_{j\in R_r(t)}\exp\{\beta^\mathsf{T}X_j(t)\}}. \end{aligned}

The common baseline hazard and $dt$ cancel. This cancellation is why the baseline hazard need not be specified parametrically to estimate $\beta$ by partial likelihood. It does not remove the risk-set restriction: the sum is over those eligible for transition $r$, not over the risk set for another transition or over the entire sample.

Worked Example: One Event and Its Transition-Specific Denominator

Suppose a hypothetical study has a transition $A$. At an event time, the eligible risk set contains three individuals with covariate values $X=0,1,2$, and the individual with $X=1$ makes transition $A$. For illustration, evaluate the likelihood contribution at $\beta=\log(2)$. Assume all three individuals are observed and eligible for $A$, and that there is no tie at this time.

Their relative intensity weights are $\exp(\beta X)=1,2,4$. The event individual’s conditional contribution is

\frac{\exp\{(\log 2)(1)\}} {\exp\{(\log 2)(0)\}+\exp\{(\log 2)(1)\}+\exp\{(\log 2)(2)\}} = \frac{2}{1+2+4} = \frac{2}{7} \approx 0.2857.

The contribution is a conditional likelihood value, not the probability that this individual will ever make transition $A$. It describes the event identity conditional on a transition-$A$ event at this time and on the specified risk set.

From Event Contributions to the Stratified Partial Likelihood

For each transition $r$, let $\mathcal{D}_r$ denote the observed event records for that transition. A record identifies an individual $i$ and event time $t$. Under distinct event times within each transition, multiplying the conditional contributions gives the partial likelihood:

L_{\mathrm{partial}}(\beta) = \prod_r\ \prod_{(i,t)\in\mathcal{D}_r} \frac{\exp\{\beta^\mathsf{T}X_i(t)\}} {\displaystyle\sum_{j\in R_r(t)} \exp\{\beta^\mathsf{T}X_j(t)\}}.

Every event contributes a numerator for the individual who made that transition and a denominator formed from that transition’s risk set at that time. The product combines information across the transition strata, while the separate baseline hazards have already canceled within their respective conditional contributions.

Taking logarithms turns the product into a sum. Define $S_r^{(0)}(\beta,t)=\sum_{j\in R_r(t)}\exp\{\beta^\mathsf{T}X_j(t)\}$. Then

\ell_{\mathrm{partial}}(\beta) = \sum_r\ \sum_{(i,t)\in\mathcal{D}_r} \left[ \beta^\mathsf{T}X_i(t)-\log S_r^{(0)}(\beta,t) \right].

Differentiating each event term gives the event individual’s covariate vector minus a risk-set weighted mean. With $S_r^{(1)}(\beta,t)=\sum_{j\in R_r(t)}\exp\{\beta^\mathsf{T}X_j(t)\}X_j(t)$, the score is

U(\beta)= \sum_r\ \sum_{(i,t)\in\mathcal{D}_r} \left[ X_i(t)- \frac{S_r^{(1)}(\beta,t)}{S_r^{(0)}(\beta,t)} \right].

To see the derivative directly, the derivative of $\log S_r^{(0)}(\beta,t)$ is $S_r^{(1)}(\beta,t)/S_r^{(0)}(\beta,t)$. Thus each event compares the observed covariates with the weighted average covariates of those eligible for that transition. The information calculation likewise uses transition-specific weighted risk-set variation; the partial likelihood is not obtained by pooling the denominators.

Worked Example: Multiplying Contributions Across Transitions

Consider hypothetical event records at three distinct times. At the first time, an $A$-event occurs for an individual with $X=1$, among $A$-eligible values $0,1,2$. At the second time, a $B$-event occurs for an individual with $X=0$, among $B$-eligible values $0,1$. At the third time, an $A$-event occurs for an individual with $X=2$, among then-eligible $A$ values $0,2$. Suppose all listed individuals are observed and eligible at their stated times, and evaluate at $\beta=\log(2)$. The risk sets are specified separately at each time; in particular, the second $A$ risk set reflects eligibility at the later time.

For the first event the contribution is $2/(1+2+4)=2/7$. For the $B$-event it is $1/(1+2)=1/3$, using only the $B$ risk set. For the later $A$-event it is $4/(1+4)=4/5$. Therefore,

L_{\mathrm{partial}}(\log 2) = \frac{2}{7}\times\frac{1}{3}\times\frac{4}{5} = \frac{8}{105} \approx 0.07619.

The small product is not itself a failure or a probability that the fitted model is correct. It is the value of the partial likelihood at the specified coefficient for these three conditional event-identity contributions. Estimation compares such values over candidate coefficient values.

Worked Example: A Numerical Score Contribution

At a hypothetical transition-$A$ event, suppose the event individual has $X=2$, while the risk-set covariates are $0,1,2$. Evaluate that event’s score contribution at $\beta=\log(2)$. The weights are $1,2,4$, so the weighted risk-set mean is

\frac{0(1)+1(2)+2(4)}{1+2+4} = \frac{10}{7} \approx 1.429.

Thus the score contribution is $2-10/7=4/7\approx0.5714$. It is positive, so this event’s contribution to the log partial likelihood has positive slope at the stated coefficient. This one contribution does not determine the fitted coefficient: the full score sums contributions over all observed events and transitions.

Ties, Diagnostics, and Common Mistakes

If multiple individuals make the same transition at exactly the same recorded time, the single-event conditioning argument does not directly yield the distinct-time product for that time. Software commonly uses a specified tie approximation, such as the Breslow contribution

\frac{\displaystyle\prod_{i\in E_r(t)} \exp\{\beta^\mathsf{T}X_i(t)\}} {\left[ \displaystyle\sum_{j\in R_r(t)} \exp\{\beta^\mathsf{T}X_j(t)\} \right]^{d_r(t)}},

where $E_r(t)$ is the set of individuals with a transition-$r$ event at $t$, and $d_r(t)$ is the number of those events. This is an approximation, not the distinct-time formula. With substantial ties, the tie handling method should be stated and chosen appropriately; an exact tied partial likelihood is another option.

A practical diagnostic begins with the event and risk-set records, not with the coefficient output. Check that each event is assigned to the correct transition, that each risk set contains only individuals eligible immediately before the event, and that an individual leaves or enters eligibility at the correct time after changing state. Confirm that covariates used at an event time were available just before that event. Empty or implausible risk sets and unexpected event counts by transition often signal a data-construction error.

Wrong reasoning: “All individuals are in the study, so every event denominator can include everyone.”
Correction: Include only individuals in $R_r(t)$, the risk set for the observed transition $r$ at time $t$.
Wrong reasoning: “Transition stratification means using one pooled denominator and adding transition labels afterward.”
Correction: The transition label determines the denominator before the event contribution is formed. A pooled denominator changes the conditional comparison and is not this stratified partial likelihood.
Wrong reasoning: “A competing event can simply be ignored when building any risk set.”
Correction: A competing event may end eligibility for a transition from the original state. Update eligibility according to the observed state history; do not keep someone at risk for a transition they can no longer make.

The partial likelihood addresses the regression parameters under the stated model; it does not by itself establish that proportional hazards is adequate or that the state-history and censoring records are correct. It also does not estimate the transition-specific baseline hazards through the conditional event-identity calculation. Compared with fitting separate cause-specific Cox models, the stratified likelihood makes explicit that the event terms are combined while each transition keeps its own risk-set denominator. A single pooled Cox model that ignores transition strata imposes a different comparison and may also impose inappropriate baseline-hazard restrictions. Neither approach repairs incorrect transition eligibility or an incorrect model.

Key takeaway: For every observed transition event, condition on its transition type and time. The baseline hazard cancels, but the denominator remains the sum of relative hazards over that transition’s eligible risk set at that time.

Check Your Understanding

Use the stated transition-specific risk sets in each question. Solutions follow each problem.

At a transition-$C$ event, the event individual has $X=0$, and the risk-set covariates are $0,1$. Find the contribution at $\beta=\log(3)$. Solution: The weights are $1,3$, so the contribution is $1/(1+3)=1/4=0.2500$.
Why does the transition-specific baseline hazard not appear in the conditional contribution? Solution: Conditional on one event of transition $r$ at time $t$, each eligible individual’s intensity contains the same factor $\lambda_{0r}(t)dt$. It cancels between the event individual’s intensity and the sum over $R_r(t)$.
A person experiences a transition out of state 1 and becomes eligible for a transition from state 2. Should they remain in the old transition’s risk set? Solution: No, not if they are no longer eligible for that transition from state 1. Their eligibility must be updated using the observed state history; they may enter a different transition risk set if appropriate.
At a transition-$A$ event, the risk-set covariates are $0,1,2$, and the event individual has $X=1$. Calculate the contribution at $\beta=0$. Solution: All weights are $\exp(0X)=1$. The contribution is $1/(1+1+1)=1/3\approx0.3333$.