Tutorials › Exam ASTAM - Advanced Short-Term Actuarial Mathematics › Deriving the Product-Limit Formula from Conditional Survival

Exam ASTAM · Kaplan-Meier and Nelson-Aalen · Tutorial 232 of 979

Deriving the Product-Limit Formula from Conditional Survival

Learn how conditional survival probabilities lead to the Kaplan-Meier product, including how to define its starting factor and use risk sets when observations are censored.

Intermediate 10 min read

What You'll Learn

  • Factor survival into successive conditional survival probabilities without assuming there is no probability mass at time zero.
  • Define the event-time risk set and estimate a conditional failure probability from its observed failures.
  • Derive the Kaplan-Meier product from those risk-set conditional survival estimates.
  • Explain why independent censoring allows censored observations to contribute information up to their censoring times.
  • Distinguish failures, which reduce the survival estimate, from censorings, which only affect later risk sets.
  • Check a product-limit calculation using event times, risk-set sizes, and a final survival estimate.

Introduction

The Kaplan-Meier estimator is a product of conditional survival estimates. That product form is not merely a computational shortcut: it follows from breaking survival through successive event times into the chance of surviving each step, given that the subject reached it. The risk set at each event time supplies the data for estimating that conditional chance.

The starting point needs care. If event time \(X\) can equal zero with positive probability, it is not generally valid to start with survival conditional on \(X>0\) and treat that as the full survival probability. We will state the factorization so that the initial survival probability is included explicitly.

Definition: Let \(X\) be a nonnegative event time and let \(S(t)=P(X>t)\) be its survival function. At a distinct observed failure time \(t_j\), let \(r_j\) be the number of subjects at risk immediately before \(t_j\), and let \(d_j\) be the number who fail at \(t_j\). The Kaplan-Meier estimate of survival just after \(t_k\) is $$ \widehat S(t_k)=\prod_{j=1}^{k}\left(1-\frac{d_j}{r_j}\right), $$ provided the initial cohort is observed from time zero and the event times are ordered. Between observed failure times, the estimate is constant.

Here \(r_j\) counts subjects whose observed time is at least \(t_j\): they have not failed or been censored before \(t_j\). A subject censored before \(t_j\) is not in that risk set. A censoring is not a failure and does not itself produce a factor below one.

Conditional Survival and the Product

For a discrete event-time distribution with ordered possible event times \(t_1<t_2<\cdots<t_k\), define the conditional failure probability at \(t_j\), among those who have reached that time, by $$ h_j=P(X=t_j\mid X\ge t_j). $$ The conditional probability of surviving past \(t_j\), given that the subject was still at risk immediately before it, is \(1-h_j\). Because no other event-time support points occur between successive times, the survival probabilities multiply:

$$ S(t_k)=\prod_{j=1}^{k}P(X>t_j\mid X\ge t_j) =\prod_{j=1}^{k}(1-h_j). $$

This is a telescoping factorization. At the first event time, the factor is the proportion of the original population that survives past \(t_1\). At each later event time, the conditional factor converts survival through the preceding event time into survival through the current one. The product therefore gives survival through all the event times.

In a sample, \(d_j/r_j\) estimates the conditional probability of failure at \(t_j\) among those at risk there. Thus \(1-d_j/r_j\) estimates the conditional survival factor. Multiplying these estimates gives the Kaplan-Meier product. With censoring, interpreting the risk-set proportion as estimating the event probability requires censoring to be noninformative: conditional on the information used to define the analysis, censoring must not preferentially remove subjects with different subsequent event prospects. The estimate is not justified by this argument if censoring depends on unobserved event time in a way that is not accounted for.

Important qualification: The conditional product is exact for a discrete distribution on the stated event-time support and for the empirical step distribution used to construct the estimator. In general event-time models, the Kaplan-Meier product is the nonparametric product-limit estimator; its risk-set factors estimate successive conditional survival steps under independent censoring. Do not interpret each finite-sample factor as an exact population probability for an arbitrary continuous distribution.

Why the Initial Factor Must Be Correct

An equivalent general conditional-probability identity can be written using thresholds \(a_0<a_1<\cdots<a_k\), where \(P(X>a_0)=1\). Then $$ P(X>a_k)=\prod_{j=1}^{k}P(X>a_j\mid X>a_{j-1}). $$ The first factor is \(P(X>a_1)\), because the conditioning event \(X>a_0\) has probability one. This identity follows directly by cancelling successive survival probabilities. It does not require assuming \(P(X>0)=1\), provided \(a_0\) is chosen below the support of \(X\).

If instead \(a_0=0\), the product begins with \(P(X>a_1\mid X>0)\). That product equals \(P(X>a_k\mid X>0)\), not necessarily \(P(X>a_k)\). A probability mass at zero must be handled in the starting factor. This distinction is useful when deriving a product, and it prevents a seemingly harmless choice of time origin from changing the quantity being estimated.

Worked Example: Factoring Survival When Some Events Occur at Time Zero

Suppose a hypothetical event-time distribution has \(P(X=0)=0.20\), \(P(X=1)=0.30\), and \(P(X=2)=0.50\). Find \(S(1.5)\) by factoring through the threshold \(0.5\), and show why conditioning on \(X>0\) alone does not give the unconditional survival probability.

Step 1: Calculate survival at the first threshold. The event \(X>0.5\) includes the outcomes \(X=1\) and \(X=2\), so $$ P(X>0.5)=0.30+0.50=0.80. $$

Step 2: Calculate the next conditional factor. Given \(X>0.5\), survival past \(1.5\) occurs only when \(X=2\). Therefore $$ P(X>1.5\mid X>0.5)=\frac{P(X=2)}{P(X>0.5)} =\frac{0.50}{0.80}=0.625. $$

Step 3: Multiply the factors. $$ P(X>1.5)=0.80(0.625)=0.50. $$ This agrees with the distribution directly, since only the mass at \(X=2\) is greater than \(1.5\).

Check the starting condition. Since \(P(X>0)=0.80\), conditioning on \(X>0\) excludes the mass at zero. In this example \(P(X>0.5\mid X>0)=1\), which is not \(P(X>0.5)=0.80\). The initial unconditional factor is essential.

Method and trap avoided: Use the conditional-probability chain with a starting threshold below the support, or include the initial survival probability explicitly. Do not assume that a nonnegative event time is strictly positive.

From Risk Sets to the Kaplan-Meier Product

At an observed failure time \(t_j\), \(r_j\) subjects have supplied information by remaining under observation and event-free until just before that time. Of those subjects, \(d_j\) fail at \(t_j\). The observed conditional failure proportion is therefore \(d_j/r_j\), and the corresponding conditional survival estimate is $$ 1-\frac{d_j}{r_j}=\frac{r_j-d_j}{r_j}. $$ Applying the conditional multiplication rule successively gives the product-limit estimator.

A censoring between two failure times removes that subject from subsequent risk sets, but does not multiply the survival estimate by a failure factor. Under independent censoring, the people still observed at a later time can represent the event experience of those who would otherwise have been at risk. The censoring record contributes follow-up up to its censoring time; it does not tell us that the event occurred then.

1
Order the distinct failure times.
List the times at which failures are observed. Censoring times affect who remains at risk but are not failure times.
2
Construct each risk set.
For each failure time \(t_j\), count the subjects still observed and event-free immediately before that time to obtain \(r_j\).
3
Estimate conditional survival.
Use \(1-d_j/r_j\) for the estimated conditional survival factor at that failure time.
4
Multiply through the time of interest.
The product of the factors through \(t_k\) is \(\widehat S(t_k)\); censorings after \(t_k\) do not enter that product.

Worked Example: Building the Product From Successive Risk Sets

In a hypothetical follow-up of 10 components, two fail at time 2, one is censored at time 3, one fails at time 5, two are censored at time 6, and one fails at time 8. The remaining three are censored after time 8. Find the Kaplan-Meier estimate of survival just after time 8.

At time 2: All 10 components are initially at risk, so \(r_1=10\) and \(d_1=2\). The estimated conditional survival factor is $$ 1-\frac{2}{10}=\frac{8}{10}. $$

At time 5: The two failures at time 2 and the censoring at time 3 are no longer in the risk set. Thus \(r_2=10-2-1=7\). One component fails, so $$ 1-\frac{1}{7}=\frac{6}{7}. $$

At time 8: After the failure at time 5, six components remain. The two censorings at time 6 reduce the risk set to \(r_3=6-2=4\). One fails at time 8, giving $$ 1-\frac{1}{4}=\frac{3}{4}. $$

Multiply the conditional factors: $$ \widehat S(8)=\frac{8}{10}\cdot\frac{6}{7}\cdot\frac{3}{4} =\frac{144}{280}=\frac{18}{35}=0.5143. $$ The estimate of the proportion surviving beyond time 8 is \(0.5143\), rounded to four decimal places. As a check, the cohort accounts for four failures and six censorings: \(2+1+1=4\) failures and \(1+2+3=6\) censorings, for 10 components total.

Method and trap avoided: Reconstruct each risk set from all earlier failures and censorings. The censorings reduce later denominators but do not add factors to the survival product.

Worked Example: Censoring Changes Later Risk Sets, Not Survival at Its Own Time

A hypothetical cohort contains nine devices. One fails at time 1, one is censored at time 2, one fails at time 4, one is censored at time 5, and one fails at time 7. The four remaining devices are censored after time 7. Calculate the estimated survival just after time 7.

At time 1: The risk set contains all nine devices, and one fails: $$ r_1=9,\qquad d_1=1,\qquad 1-\frac{d_1}{r_1}=\frac{8}{9}. $$

At time 4: The failure at time 1 and censoring at time 2 have left the risk set. Thus \(r_2=9-1-1=7\). One failure gives the factor $$ 1-\frac{1}{7}=\frac{6}{7}. $$

At time 7: The failure at time 4 and censoring at time 5 also leave the risk set. Hence \(r_3=7-1-1=5\), and the failure gives $$ 1-\frac{1}{5}=\frac{4}{5}. $$

Multiply and check: $$ \widehat S(7)=\frac{8}{9}\cdot\frac{6}{7}\cdot\frac{4}{5} =\frac{192}{315}=\frac{64}{105}=0.6095. $$ The final estimate is \(0.6095\), rounded to four decimal places. The cohort count is consistent: three failures plus six censorings (at times 2, 5, and the four later censorings) equals nine devices.

Method and trap avoided: A censored device contributes to the risk set until its censoring time, then leaves it. Treating the censoring at time 2 as a failure would incorrectly insert a survival factor such as \(1-1/8\). The correct action is to make no survival decrement at time 2 and use a smaller risk set at time 4.

Assumptions, Interpretation, and Exam Checks

The product formula is straightforward, but its interpretation depends on both correct bookkeeping and a defensible censoring assumption. The estimator describes survival in the population represented by the risk sets, not merely among subjects whose event times happened to be observed exactly. If subjects with poor prospects are systematically censored earlier, the observed later risk sets can be unrepresentative and the product can be biased.

The Kaplan-Meier estimate also has a precise time interpretation: \(\widehat S(t)\) estimates the probability of an event time strictly greater than \(t\). At an observed failure time, the factor applies to survival just after that failure time. Between observed failures, the estimate has no additional failure factor. These conventions help avoid confusing a failure count with a censoring count or using a survival estimate from before a failure when the question asks for survival after it.

  • Do not multiply censoring factors. A censoring is not an observed event.
  • Do not retain censored subjects in later risk sets. They contribute only through their censoring time.
  • Do not omit the initial survival condition in a theoretical factorization. Choose a starting threshold below the support or include the first unconditional factor.
  • Check every denominator. The risk set is the number at risk immediately before the failure time, after earlier removals.
Key takeaway: Kaplan-Meier survival is built by estimating conditional survival at successive failure times and multiplying those factors. The factor \(1-d_j/r_j\) comes from the failures among those at risk; independent censoring justifies using the risk sets, and the initial factor must preserve any probability mass at time zero.

Check Your Understanding

For each problem, identify the relevant conditional survival factors before multiplying them. Round probability answers to four decimal places where needed.

  1. A hypothetical cohort starts with 20 subjects. Two fail at time 2, four are censored at time 3, and one fails at time 5. Find the Kaplan-Meier estimate just after time 5. (Answer: \(0.8357\))
  2. A cohort of 15 units has three failures at the first observed failure time. Two are censored before the next failure time, when two more fail. Find the estimate just after the second failure time. (Answer: \(0.6400\))
  3. A hypothetical event-time distribution has \(P(X=0)=0.10\), \(P(X=2)=0.30\), and \(P(X=5)=0.60\). Factor \(P(X>3)\) using the initial factor \(P(X>0.5)\) and the conditional probability given \(X>0.5\). (Answer: \(0.6000\))
  4. There are 12 subjects. Two fail at time 1, three are censored at time 2, and one fails at time 4. Find the estimate just after time 4. (Answer: \(0.7143\))
  5. A cohort of 18 has three failures at the first failure time, four censorings before the next failure time, and two failures at that next time. Find the estimate just after the second failure time. (Answer: \(0.6818\))
Clinical Biostats