Delayed Entry And Left-Truncated Validation Samples

Why Delayed Entry Changes the Denominator

A validation sample may begin observing people only after their multi-state process has already been underway. A person entering a registry at process time 5 is not known to have been in the validation data at time 2. Counting that person in a time-2 risk set would add information that was not observed and could change the estimated transition rate. Conversely, ignoring entry times and treating everyone as observed from the process origin can bias estimates, especially when entry depends on prior survival or state history.

The essential rule is temporal: at a transition time, a subject contributes to a risk set only if they have entered observation before that time, have not yet had an exit or censoring event, and are in the transition’s origin state immediately before it. Delayed entry changes the risk set at each event time; it does not justify assigning unobserved pre-entry history to a subject.

Definition: Subject \(i\) has entry time \(L_i\), observed exit time \(T_i\), and state process \(X_i(u)\) while under observation. Under the convention used here, entry is after any updates at \(L_i\). At an event time \(u\), the subject is eligible for a transition risk set only when \(L_i<u\leq T_i\). Thus an event at \(L_i\) is not counted as an observed post-entry transition. \(X_i(u-)\) denotes the state immediately before time \(u\).

For a transition from state \(j\) to a different state \(k\), define \(N_{i,jk}(u)\) as the number of observed \(j\)-to-\(k\) transitions by time \(u\), after entry. The risk-set size just before \(u\) and the transition count at \(u\) are

Y_j(u)=\sum_i I\{L_i<u\leq T_i,\ X_i(u-)=j\}, \qquad dN_{jk}(u)=\sum_i dN_{i,jk}(u).

Here \(I\{\cdot\}\) is 1 when its condition holds and 0 otherwise. If a subject has a competing transition out of \(j\), that subject leaves the risk set for transitions out of \(j\). A subject censored at \(u\) contributes just before \(u\), but not after it; for tied event and censoring times, the observation protocol must specify which event occurred first. The displayed convention treats an observed transition at \(u\) as occurring before removal from the risk set at \(u\).

Worked Example: Who Is at Risk at a Transition Time?

Suppose a hypothetical validation sample has five subjects in state 0 at process time 2. Their entry times are 0, 0.5, 1, 1, and 1.5; all are still under observation and in state 0 immediately before time 2. Two further subjects enter at time 2.5 in state 0. At time 4, one of the original five has transitioned to state A, one to state B, and the other three remain under observation in state 0. Both later entrants are still in state 0 at time 4.

At time 2, \(Y_0(2)=5\). The two subjects who enter at 2.5 are not included: \(2.5<2\) is false, so they were not observed before time 2. At time 4, the two earlier transitions have removed their subjects from state 0, leaving three original subjects in that state; both later entrants are now eligible. Therefore \(Y_0(4)=3+2=5\).

If there is one observed \(0\to A\) transition at time 4, its transition-specific increment is \(1/5=0.20\), not \(1/7\). The denominator is the five people actually in state 0 and observed just before time 4, not everyone who has ever entered the sample. This calculation assumes that both later entrants’ states and observation status are known immediately before time 4.

Transition Increments and the Aalen–Johansen Estimate

For each observed transition time \(u\), the estimated transition-hazard increment from state \(j\) to state \(k\) is \(dN_{jk}(u)/Y_j(u)\). It is defined only when \(Y_j(u)>0\). The estimate of the increment in the diagonal entry for state \(j\) is the negative sum of its outgoing increments, so each row of the increment matrix sums to zero. Combining these increments over time gives an estimated transition-probability matrix through the Aalen–Johansen product integral.

d\widehat A_{jk}(u)=\frac{dN_{jk}(u)}{Y_j(u)} \quad (j\ne k), \qquad d\widehat A_{jj}(u)=-\sum_{k\ne j}d\widehat A_{jk}(u), \qquad \widehat P(s,t)=\prod_{s<u\leq t}\{I+d\widehat A(u)\}.

The product is taken in increasing time order, with each increment matrix multiplying the current state-probability row vector from the right. The interval \((s,t]\) matters: a prediction made at \(s\) after updates at \(s\) uses transitions strictly after \(s\), and includes transitions at \(t\). A transition at \(s\) is not an outcome from that post-update prediction origin.

Worked Example: A Left-Truncated Aalen–Johansen Calculation

Consider a hypothetical process with state 0 and two absorbing destinations, A and B. The risk set and event counts are as follows. At time 2, five subjects are observed in state 0 immediately before the event time; one moves to A and one to B. At time 4, three of those five remain in state 0, and two subjects who entered between times 2 and 4 are also observed in state 0. One of these five moves to A at time 4. Assume no other transitions occur by time 4, all relevant censoring is independent, and the transition process is adequately represented by the same state- and process-time-specific transition intensities for the target being estimated.

At time 2, the two nonzero outgoing increments from state 0 are \(d\widehat A_{0A}(2)=1/5=0.20\) and \(d\widehat A_{0B}(2)=1/5=0.20\). The diagonal increment is \(d\widehat A_{00}(2)=-(0.20+0.20)=-0.40\). Starting in state 0, the state-probability row after time 2 is therefore \((0.60,0.20,0.20)\), in the order \(0,A,B\).

At time 4, \(d\widehat A_{0A}(4)=1/5=0.20\), \(d\widehat A_{0B}(4)=0\), and \(d\widehat A_{00}(4)=-0.20\). The four subjects in A or B stay in their absorbing states. The probability of remaining in state 0 is updated to \(0.60(1-0.20)=0.48\). The probability of being in A is its previous value plus the probability still in state 0 times the new increment: \(0.20+0.60(0.20)=0.32\). The probability in B remains \(0.20\). Thus the estimated state-occupation probabilities at time 4 for a subject starting in state 0 at time 0 are \(0.48\) in state 0, \(0.32\) in A, and \(0.20\) in B; they sum to 1.

The entrants between times 2 and 4 contribute to the time-4 risk set, but their unobserved histories before entry are not treated as observed transitions. Their contribution helps estimate the time-4 increment under the stated assumptions; it does not make them members of the time-2 risk set.

Why the Increment Has This Form

The risk-set denominator follows from the conditional transition rate. Among subjects observed in state \(j\) immediately before \(u\), the fraction making a \(j\to k\) transition at \(u\) estimates the transition probability increment at that time. If \(Y_j(u)\) subjects are eligible and \(dN_{jk}(u)\) make that transition, the empirical fraction is \(dN_{jk}(u)/Y_j(u)\). Adding these increments across destination states and subtracting their sum from the diagonal preserves the total probability in each row.

The product-integral update follows by applying each event-time transition matrix in sequence. If \(p(u-)\) is the row of state probabilities immediately before \(u\), then after the update

p(u)=p(u-)\{I+d\widehat A(u)\}.

Repeated multiplication over event times in \((s,t]\) gives \(\widehat P(s,t)\). This argument also shows why the origin convention must be consistent: including an update at \(s\) would describe a pre-update origin, not predictions made after the updates at \(s\).

Choosing the Validation Origin

There are two distinct targets, and delayed entry does not make them interchangeable. For validation from a common process time \(s\), subjects must be observed, eligible, and in the required state at \(s\). Subjects whose entry time is later than \(s\) cannot contribute outcomes from \(s\), because their state and transitions between \(s\) and entry are unobserved. A common-time analysis is conditional on being observed and eligible at \(s\), unless a separate design or weighting argument supports a broader target.

Alternatively, a prediction may be made for each subject at that subject’s entry time \(L_i\), using information available then. The outcome interval is \((L_i,t_i]\), where \(t_i\) is the subject-specific horizon endpoint. Pooling such records evaluates predictions across the entry-time distribution; it is not automatically an estimate of performance at one fixed process time. If entry times differ, the prediction horizon and time scale must also be specified: a fixed duration after entry and a fixed calendar process time answer different questions.

Worked Example: A Post-Entry Prediction Origin

Suppose six hypothetical subjects are observed in state 0 just after all updates at process time 5. Predictions are generated at that time for the next two time units. All six have complete observed follow-up through time 7. Their states at time 7 show two transitions to A, one to B, and three who remain in state 0. The mean predicted probabilities across these six subjects are 0.30 for A, 0.20 for B, and 0.50 for remaining in state 0.

The observed transition proportions over \((5,7]\) are \(2/6=0.333\) for A, \(1/6=0.167\) for B, and \(3/6=0.500\) for remaining in state 0. The observed-minus-mean-predicted differences are \(0.333-0.30=0.0333\) for A, \(0.167-0.20=-0.0333\) for B, and \(0.500-0.50=0\) for state 0. These are descriptive comparisons for this hypothetical entry sample, not evidence by themselves of a causal effect or a general population result.

If a different subject had transitioned at exactly time 5 before entering the validation sample, that transition would not be counted as an outcome from the post-update prediction made at time 5. It would have occurred before the subject was eligible for this validation origin. Likewise, a transition exactly at time 5 cannot be used as an outcome for a prediction defined to start after the updates at 5. This example has complete follow-up; with censoring before time 7, the simple proportions would not generally estimate the horizon probabilities, and censoring adjustment such as the established IPCW approach would be needed under its assumptions.

Assumptions, Diagnostics, and Alternatives

A delayed-entry estimate needs more than correctly counted denominators. The event and state history after entry must be recorded sufficiently well to determine each subject’s state and eligibility. Entry must be independent of future transition experience, conditional on variables or history used to define the target and analysis, in the sense required for left-truncation methods. Independent censoring is also required, possibly conditional on measured predictors when censoring weights or conditional models are used. There must be adequate positivity: at the transition times of interest, relevant subjects must have a nonzero chance of having entered and remaining observable. For an Aalen–Johansen transition matrix, the transition intensities must be estimable for the target process; a Markov interpretation based only on current state and process time requires the Markov assumption, or a model that appropriately accounts for additional history.

When subjects enter at different process times, check the number at risk by state and time, event counts by transition, entry-time patterns, and whether the observation system captured state at entry. A transition increment based on a very small risk set is unstable; a zero risk-set size means that increment is not estimable there. Large changes in entry rates or long periods with no entrants can reveal weak positivity. Compare entry distributions across measured prognosis-related variables and examine whether entry depends on prior state history or survival. Such checks can reveal problems but cannot establish independence of entry from unobserved future prognosis.

A common-time landmark analysis and an entry-time analysis answer different questions. Restricting to subjects already observed at the common origin avoids imputing pre-entry information but may narrow the target to those observed then. Using each person’s entry as their prediction origin makes use of later entrants, but estimates performance averaged over their entry circumstances unless standardized to another target. Neither approach corrects informative entry automatically. If selection into observation depends on unmeasured prognosis, risk-set fractions may not represent the transition experience in the intended population.

Common pitfall: Treating every subject in the eventual validation sample as at risk from process time zero, or counting an event at entry as an outcome after an entry-time prediction, uses time the subject was not under observation for that target. Use \(L_i<u\) in the risk-set definition and state explicitly whether the prediction origin is before or after updates at the origin time.

Check Your Understanding

Answer each question, then compare with the solutions.

At time 3, four subjects are observed in state 1 immediately before an event time. One makes a \(1\to 2\) transition. A fifth subject enters at time 3.2. What is the transition increment at time 3, and does the fifth subject belong to that risk set?
A prediction is made at time 6 after all updates at 6. Which event interval should be used to evaluate its outcome through time 9, and should a transition at exactly time 6 count?
At a transition time, \(Y_0(u)=8\), with two \(0\to A\) events and one \(0\to B\) event. What are the outgoing increments and the diagonal increment for state 0?
Why does an entry-time validation sample not automatically estimate prediction performance at a fixed common process time?

Solutions. (1) The increment is \(1/4=0.25\). The fifth subject entered after time 3 and is not in the risk set. (2) Use \((6,9]\). A transition exactly at 6 is excluded because the prediction origin is post-update at 6. (3) The increments are \(2/8=0.25\) for \(0\to A\) and \(1/8=0.125\) for \(0\to B\); the diagonal increment is \(-(0.25+0.125)=-0.375\). (4) Subjects enter at different times and may differ in observed state, history, and prognosis. Their outcomes therefore correspond to different origins; pooling them targets performance across the entry-time distribution unless the analysis is restricted or standardized to the chosen common-time target.