Tutorials › Econometrics (Graduate) › Quantile Estimation Under Conditional Independent Censoring

Quantile and Distribution Regression · Tutorial 606 of 952

Quantile Estimation Under Conditional Independent Censoring

Estimate outcome quantiles from right-censored data by weighting observed events for their probability of being observed, and assess when those estimates are credible.

Advanced 11 min read

What You'll Learn

  • Define the event time, censoring time, observed time, and event indicator.
  • State the conditional independent censoring assumption and its role in identification.
  • Construct a normalized inverse-probability-of-censoring weighted distribution estimate.
  • Recover an outcome quantile from the weighted distribution.
  • Check positivity over the outcome range needed for the target quantile.
  • Diagnose how small censoring survival probabilities create unstable estimates.

Why Censoring Changes Quantile Estimation

With right-censored data, some outcome times are only known to exceed a recorded time. Treating those recorded times as if they were event times changes the outcome distribution; discarding censored observations can also select a nonrepresentative subset. The central task is to use observed events while accounting for how likely an event at that time was to be observed.

This tutorial estimates marginal quantiles of an outcome time in a target population represented by the sample. Covariates may affect both the outcome and censoring, so the method conditions on measured covariates when modeling the censoring process, then averages the weighted contributions across the population. The resulting target is not automatically a conditional quantile for any particular covariate value.

Definition: Let \(T\) be the outcome time of interest, \(C\) the censoring time, and \(X\) observed covariates. We observe \(Y=\min(T,C)\) and \(\Delta=1(T\le C)\), where \(\Delta=1\) indicates an observed event and \(\Delta=0\) indicates right censoring. Define the conditional censoring survival probability \(G(t\mid X)=P(C\ge t\mid X)\). For a target probability \(\tau\), the marginal outcome quantile is \(q_\tau=\inf\{t:F_T(t)\ge\tau\}\), where \(F_T(t)=P(T\le t)\).

The event indicator distinguishes an observed event time from a censoring time. If \(\Delta=1\), then \(Y=T\), so the outcome is observed. If \(\Delta=0\), then \(Y=C<T\) under the stated convention, and the data establish only that \(T>Y\). We assume no event-censoring ties, or that ties are handled consistently with the definition \(G(t\mid X)=P(C\ge t\mid X)\).

Identification by Inverse-Probability Weighting

The key assumption is conditional independent censoring: given the observed covariates \(X\), the outcome time and censoring time are independent. Thus \(T\perp C\mid X\). Censoring need not be independent of the outcome in the population without conditioning; it may depend on \(X\), provided the relevant covariates are observed and the conditional independence assumption is credible.

An event at time \(T\) is observed only if censoring has not occurred by that time. Given \(X\), this has probability \(G(T\mid X)\). Consequently, an observed event receives weight \(1/G(T\mid X)\): events that were less likely to be observed represent more individuals in the target population. Censored observations do not contribute as if their censoring times were event times.

Identification result: Under conditional independent censoring and positivity, for any time \(a\), $$ E\left[\frac{\Delta\,1(Y\le a)}{G(Y\mid X)}\right]=P(T\le a)=F_T(a), $$ where the fraction is evaluated only for observations with \(\Delta=1\). Positivity requires \(G(T\mid X)>0\) on the outcome support needed to identify the target distribution.

The result can be checked conditional on \(T\) and \(X\). If \(T\le a\), then an event is observed with probability \(G(T\mid X)\); when it is observed, \(Y=T\). The conditional expected weighted contribution is therefore one. If \(T>a\), the indicator is zero whenever the event is observed, and a censored observation cannot satisfy \(Y\le a\) while also contributing as an event. The conditional expected contribution is then zero.

$$ E\left[\left.\frac{\Delta\,1(Y\le a)}{G(Y\mid X)}\right|T,X\right] = 1(T\le a)\frac{P(C\ge T\mid T,X)}{G(T\mid X)} = 1(T\le a). $$

The second equality uses conditional independent censoring, which gives \(P(C\ge T\mid T,X)=P(C\ge T\mid X)=G(T\mid X)\). Taking expectations over \(T\) and \(X\) gives \(F_T(a)\). This identity explains why weighting recovers the marginal distribution even when censoring depends on measured covariates.

Constructing the Weighted Distribution and Quantile

In practice, \(G(t\mid X)\) is usually unknown and is replaced by an estimate \(\widehat G(t\mid X)\), obtained from a censoring model. For example, one may estimate the conditional probability of remaining uncensored up to time \(t\) using an appropriate survival model. The model must be for the censoring distribution, not the outcome distribution. The calculations below take the censoring probabilities as known to keep the weighting arithmetic transparent.

A normalized inverse-probability-of-censoring weighted empirical distribution is

$$ \widehat F_T(t) = \frac{\displaystyle\sum_{i=1}^n \frac{\Delta_i\,1(Y_i\le t)}{\widehat G(Y_i\mid X_i)}} {\displaystyle\sum_{i=1}^n \frac{\Delta_i}{\widehat G(Y_i\mid X_i)}}. $$

Only observed events enter these sums. Normalizing by the sum of event weights produces a weighted empirical distribution whose total mass is one. The estimated quantile is its generalized inverse:

$$ \widehat q_\tau=\inf\{t:\widehat F_T(t)\ge\tau\}. $$

This definition also specifies how to handle ties: the estimated quantile is the first event time at which the weighted cumulative share reaches or exceeds \(\tau\). If the censoring probabilities are estimated, consistency of the weighted distribution additionally requires a sufficiently well-estimated censoring model and appropriate regularity conditions. The simple formula does not by itself supply a standard error; uncertainty in estimating \(G\) must be accounted for in inference.

Worked Example: A Weighted Median From Right-Censored Times

Suppose a hypothetical sample has five individuals. The following table gives the observed time \(Y\), event indicator \(\Delta\), and the known conditional censoring survival probability at the event time when an event is observed. The covariates \(X\) used in those probabilities are understood to be included in the censoring model.

Individual\(Y\) (time units)\(\Delta\)\(G(Y\mid X)\), if event observed
1210.90
230—
3410.80
4510.50
570—

The censored records contribute no event mass. The three observed events receive weights \(1/0.90=1.111\), \(1/0.80=1.250\), and \(1/0.50=2.000\), respectively. The total event weight is \(1.111+1.250+2.000=4.361\).

At time 2, the weighted cumulative share is \(1.111/4.361=0.2548\). At time 4 it is \((1.111+1.250)/4.361=2.361/4.361=0.5414\). Thus the cumulative share first reaches 0.50 at time 4, and the estimated median is \(\widehat q_{0.50}=4\) time units. At time 5 the share is \(4.361/4.361=1\), so the estimated 0.75 quantile is 5 time units.

These are estimates of the outcome-time quantiles under the weighting assumptions, not claims that the two censored individuals had events at times 3 and 7. Their event times remain unknown.

Positivity and the Targetable Quantile Range

Positivity means that individuals in the covariate and outcome regions relevant to the target have a nonzero chance of remaining uncensored long enough for an event to be observed. If \(G(t\mid X)=0\), an event at time \(t\) for those covariates can never appear in the observed event data. No inverse weight can recover information that is never observed. For a target quantile \(q_\tau\), censoring survival must be positive over the outcome times and covariate values needed to identify \(F_T(t)\) through that quantile.

A stronger, often useful stability condition is that \(G(t\mid X)\) be bounded away from zero in the relevant region. Positivity in the strict sense only says the probability is greater than zero; it does not guarantee precise estimates. When \(G\) is small, its inverse is large, so a small number of observed events can dominate the weighted distribution. This issue is especially important for upper-tail quantiles, which require information about later outcome times.

Worked Example: Diagnosing an Unstable Tail Estimate

Consider a hypothetical set of four observed events at times 2, 4, 6, and 8 time units. Their estimated censoring survival probabilities are 0.80, 0.50, 0.10, and 0.05. The corresponding weights are \(1.25\), \(2.00\), \(10.0\), and \(20.0\), for a total weight of \(33.25\).

The weighted cumulative shares are \(1.25/33.25=0.03759\) at time 2, \(3.25/33.25=0.09774\) at time 4, and \(13.25/33.25=0.3985\) at time 6. At time 8 the share reaches \(33.25/33.25=1\). The estimated median is therefore 8 time units, because the weighted share remains below 0.50 through time 6.

The event at time 8 contributes \(20/33.25=0.6015\), or about 60.2% of all event weight. The calculation is valid only if the estimated censoring probability of 0.05 is credible and positive. Even then, a single event controls much of the result, signaling high sampling variability and sensitivity to the censoring model. If that probability were zero, the event-time distribution in that region would not be identified by this weighting method.

Practical Checks Before Reporting Quantiles

A weighted estimate is not automatically reliable because it has been computed correctly. The assumptions concern the data-generating process, while the estimated weights also depend on how well the censoring model describes that process. Before interpreting a quantile, check the target population, the covariates in the censoring model, the range of estimated censoring survival probabilities, and the amount of weighted information near the target quantile.

1
Specify the outcome and target.
Define \(T\), the population whose marginal distribution is sought, and the quantile level \(\tau\). Make clear that \(Y\) is the observed minimum, not necessarily the event time.
2
Assess conditional independent censoring.
Ask whether the recorded covariates \(X\) are sufficient to make censoring independent of the outcome time within covariate strata. This is a substantive assumption, not something established by the weight formula.
3
Estimate and inspect censoring survival.
Estimate \(G(t\mid X)\) using a model suited to the covariates and censoring process. Examine whether probabilities become very small over times needed for the requested quantile.
4
Compute and qualify the quantile.
Form the weighted distribution and its generalized inverse. Report the time units, the target population, and any weak-positivity or model concerns that make the estimate unstable.

Common Mistakes and Interpretation

  • Using censored times as event times. A recorded censoring time says the event has not yet occurred by that time; it does not reveal the event time. Only observed events contribute event mass in the stated estimator.
  • Ignoring covariate-dependent censoring. Marginal independence of \(T\) and \(C\) is not required, but conditional independence given adequate observed covariates is. Omitting important common predictors of outcome and censoring can invalidate the weights.
  • Assuming a large inverse weight is automatically better correction. A large weight compensates for a small observation probability, but it also makes the estimate more sensitive to a few records and to errors in \(\widehat G\).
  • Treating positivity as a computational detail. Replacing a zero estimated probability with an arbitrary small number does not restore information where events are never observable. Truncating large weights can stabilize a calculation, but generally changes the estimator and can introduce bias; it should be justified and reported.
  • Interpreting the result causally. The weighted quantile describes an outcome-time distribution under the stated assumptions. Weighting for censoring alone does not establish a causal effect.

The method matters because it converts the observation process into an explicit part of distribution estimation. A credible quantile estimate requires not only a correct weighted calculation, but also a defensible conditional independence assumption and adequate censoring survival through the part of the distribution being estimated.

Check Your Understanding

Use the definitions and assumptions above to answer the following questions.

  1. What information about \(T\) is observed when \(\Delta=0\), and why should \(Y\) not be treated as an event time?
  2. State conditional independent censoring using \(T\), \(C\), and \(X\). Does it require \(T\) and \(C\) to be marginally independent?
  3. Why is an observed event at a time with \(G(t\mid X)=0.20\) assigned weight 5?
  4. How is the estimated quantile obtained from the normalized weighted empirical distribution?
  5. Explain why positivity is especially important for estimating upper-tail quantiles, and what a very large inverse weight signals.
Clinical Biostats