Why IPCW Recovers the Uncensored History Distribution

Why Weighting Can Recover Missing Histories

When dropout depends on a person’s observed history, the histories available at the end of follow-up need not resemble those of the original cohort. For example, people with worsening health may be more likely to leave a study. An average computed only among those still observed can therefore describe a selected group rather than the full cohort. Inverse probability of censoring weighting (IPCW) corrects this selection by giving more weight to histories that had a lower probability of remaining observed.

The claim is not that weighting reconstructs each person’s unobserved history. Rather, under specified assumptions, a weighted average across observed people has the same population expectation as the corresponding average across everyone’s full history. This is a statement about distributions and expectations, not individual-level recovery.

Histories, Retention, and Weighting

Definition: Let $H_t$ denote the full history through time $t$ that would be available if a participant were not censored. Let $R_t=1$ indicate that the participant remains uncensored and observed through time $t$, and let $R_0=1$. For a participant at risk of censoring at time $j$, define $q_j(H_{j-1})=P(R_j=1\mid R_{j-1}=1,H_{j-1})$. The cumulative probability of remaining uncensored through time $t$, conditional on the evolving history, is $G_t=\prod_{j=1}^{t}q_j(H_{j-1})$. The unstabilized IPCW at time $t$ for an observed participant is $1/G_t$.

Here, $H_{j-1}$ is the history available before the retention decision at time $j$. It can include measured covariates, treatment, and outcomes recorded up to that point. The same time ordering must be used in the censoring model and in defining the history. A participant censored before time $t$ has $R_t=0$, so their history does not contribute to an observed-data weighted average at $t$.

Two conditions are central. Sequential independent censoring means that, among participants still observed at time $j-1$, the retention decision at time $j$ is independent of the remaining full-data history given $H_{j-1}$. In a precise form, for full histories compatible with the observed past, $P(R_j=1\mid R_{j-1}=1,H_t)=q_j(H_{j-1})$, for $j\leq t$. Censoring positivity requires $q_j(H_{j-1})>0$ for histories that occur in the target population. Without it, some histories have no chance of being observed, even after conditioning on the recorded past.

The Recovery Identity

Theorem: Suppose the full history $H_t$ is well-defined for every participant; sequential independent censoring holds through time $t$; and $G_t>0$ almost surely in the target population. Then, for every bounded measurable function $g$, $$ E\left[\frac{R_t}{G_t}g(H_t)\right]=E[g(H_t)]. $$ In particular, for every measurable set $B$, the expected IPCW contribution from observed histories in $B$ equals the full-population probability of $B$: $$ E\left[\frac{R_t}{G_t}\mathbf{1}\{H_t\in B\}\right]=P(H_t\in B). $$

The theorem concerns the population probability model. In a finite sample, the corresponding weighted empirical average is an estimate of the full-data quantity; it will not generally equal that quantity exactly. The result also does not say that the observed histories have the same unweighted distribution as the full histories.

Proof. By the chain rule for conditional probabilities and sequential independent censoring, conditional on the full history $H_t$, the probability of remaining uncensored at every time through $t$ is the product of the conditional retention probabilities:

P(R_t=1\mid H_t) =\prod_{j=1}^{t}P(R_j=1\mid R_{j-1}=1,H_t) =\prod_{j=1}^{t}q_j(H_{j-1}) =G_t.

Because $R_t$ is an indicator, $E[R_t\mid H_t]=P(R_t=1\mid H_t)=G_t$. Since $g(H_t)/G_t$ is a function of $H_t$ and $G_t>0$, iterated expectation gives

E\left[\frac{R_t}{G_t}g(H_t)\right] =E\left[ \frac{g(H_t)}{G_t}E(R_t\mid H_t) \right] =E\left[\frac{g(H_t)}{G_t}G_t\right] =E[g(H_t)].

Boundedness of $g$ ensures the expectation on the right is finite; the identity then establishes the result. Taking $g(H_t)=\mathbf{1}\{H_t\in B\}$ gives the distributional statement. $\square$

Worked Examples

Worked Example: Correcting a One-Period History Distribution

Suppose a binary history $U$ has $P(U=1)=0.4$ and $P(U=0)=0.6$. The probability of remaining observed is $0.9$ when $U=1$ and $0.5$ when $U=0$. Find the observed probabilities and show how weighting recovers the full distribution.

The probability of observing a participant with $U=1$ is $0.4(0.9)=0.36$, while the probability of observing one with $U=0$ is $0.6(0.5)=0.30$. Thus, 0.66 of the population is observed. Among observed participants, the proportion with $U=1$ is $0.36/0.66\approx0.545$, not the full-population proportion 0.4.

For an observed participant with $U=1$, the inverse retention weight is $1/0.9$. The weighted population contribution for that history is $0.36/0.9=0.4$. For $U=0$, the weight is $1/0.5$, giving $0.30/0.5=0.6$. The recovered weighted contributions are therefore 0.4 and 0.6, which sum to one and match the full history distribution.

Worked Example: Recovering a Two-Time History Distribution

Let the full history be $(X_0,X_1)$, with probabilities $0.30,0.20,0.20,0.30$ for $(0,0),(0,1),(1,0),(1,1)$, respectively. Retention probabilities at time 1 are $0.8$ for $X_0=0$ and $0.5$ for $X_0=1$. At time 2, the retention probabilities for those four histories are $0.9,0.6,0.7,0.4$, respectively. Calculate each observed and weighted history contribution.

For each cell, the probability of observing the full history is its full-data probability multiplied by both retention probabilities. The total weight is the reciprocal of their product.

History	Full probability	Retention product	Observed probability	Weighted contribution
(0,0)	0.30	0.8 × 0.9 = 0.72	0.30 × 0.72 = 0.216	0.216 ÷ 0.72 = 0.30
(0,1)	0.20	0.8 × 0.6 = 0.48	0.20 × 0.48 = 0.096	0.096 ÷ 0.48 = 0.20
(1,0)	0.20	0.5 × 0.7 = 0.35	0.20 × 0.35 = 0.070	0.070 ÷ 0.35 = 0.20
(1,1)	0.30	0.5 × 0.4 = 0.20	0.30 × 0.20 = 0.060	0.060 ÷ 0.20 = 0.30

The observed probability of having $X_1=1$ is $0.096+0.060=0.156$, and the total observed probability is $0.442$. Among observed participants, $P(X_1=1\mid R_2=1)=0.156/0.442\approx0.353$. After weighting, the contributions for $X_1=1$ are $0.20+0.30=0.50$, and all four weighted contributions sum to 1. The full-data probability $P(X_1=1)=0.50$ is recovered.

Worked Example: Recovering a Full-Population Outcome Mean

At one follow-up time, suppose half the population has outcome $Y=0$ and half has $Y=10$. Participants with $Y=0$ remain observed with probability $0.8$, while those with $Y=10$ remain observed with probability $0.2$. Calculate the unweighted observed mean and the IPCW mean.

The observed probabilities of the two outcome values are $0.5(0.8)=0.4$ and $0.5(0.2)=0.1$. The total observed probability is $0.5$. Consequently, the unweighted observed mean is $ (0.4(0)+0.1(10))/0.5=2$, despite the full-population mean being $0.5(0)+0.5(10)=5$.

The inverse weights are $1/0.8=1.25$ for $Y=0$ and $1/0.2=5$ for $Y=10$. The weighted contributions to the population total are $0.4(1.25)=0.5$ and $0.1(5)=0.5$. The weighted mean is therefore $(0.5(0)+0.5(10))/(0.5+0.5)=5$. The correction works because the weights restore both the full-population outcome mass and the total population mass.

When the Identity Does and Does Not Apply

The assumptions explain both the appeal and the limits of IPCW. Sequential independent censoring says that, after conditioning on the observed history used to calculate the retention probabilities, censoring does not select participants according to their remaining full-data histories. Positivity says that every relevant history has some chance of continued observation. The identity is exact for the true conditional retention probabilities; estimates from a censoring model introduce additional estimation error.

A practical consequence of the theorem is that, for a bounded outcome $Y$ recorded by time $t$, choosing $g(H_t)=Y$ recovers $E[Y]$. Choosing $g(H_t)=\mathbf{1}\{Y\leq y\}$ recovers the cumulative distribution at $y$. Thus the same identity applies to means and to distributional summaries, provided the chosen function is defined on the full history and has a finite expectation.

Common mistake: “The observed group is only half the population, so multiplying everyone’s contribution by two must recover the full distribution.” This is wrong when retention probabilities differ across histories. A single constant changes the total weight but cannot undo history-dependent selection. The correction is to use the conditional retention probabilities given the relevant history, as the theorem requires.

The one-period outcome example makes the problem visible: the observed probabilities of $Y=0$ and $Y=10$ are 0.4 and 0.1. Multiplying both by the same constant preserves their ratio, so the weighted mean remains 2 after normalization. The history-specific weights $1/0.8$ and $1/0.2$, by contrast, restore equal outcome masses and the mean of 5.

Diagnostics can flag problems but cannot prove the assumptions. Examine fitted retention probabilities and the resulting weights for very small probabilities or a long right tail; these suggest weak positivity and can make estimates unstable. Check whether weighted distributions of measured histories among retained participants resemble the corresponding distributions in the original cohort at each follow-up time. Large discrepancies can reveal a poorly specified censoring model. However, measured-history balance does not rule out dependence on unmeasured future information, so it cannot establish sequential independent censoring.

Complete-case analysis is a simpler alternative, but it generally targets the observed complete cases rather than the full cohort when selection depends on history. Imputation and IPCW address missingness in different ways: imputation models unobserved values, whereas IPCW reweights observed contributions to represent the target population. Either approach depends on assumptions and model choices; neither can recover information about histories with zero probability of observation. IPCW can also be variable when some retention probabilities are small, even if they are strictly positive.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

If every participant has the same retention probability at a given time, does censoring change the distribution of full histories among those retained under the stated assumptions? Explain.
In a one-period study, $P(U=1)=0.25$, with retention probabilities 0.8 for $U=1$ and 0.4 for $U=0$. What are the observed probabilities of each value, and what weighted contributions recover the full distribution?
Why does a positive overall retention rate not guarantee censoring positivity?
Can IPCW recover an outcome mean if retention depends on an unmeasured future outcome even after conditioning on the recorded history? Explain.

Solutions. 1. Yes. If retention probability is constant across histories, then the probability of each history among retained participants is proportional to its full-population probability by the same factor. Normalizing cancels that factor, so the distributions agree. 2. The observed probabilities are $0.25(0.8)=0.20$ for $U=1$ and $0.75(0.4)=0.30$ for $U=0$. Weighting gives $0.20/0.8=0.25$ and $0.30/0.4=0.75$, the full distribution. 3. Positivity is required for each relevant history, not merely for the population overall. Some history may have zero retention probability even when many other participants remain observed. 4. Not generally. That dependence violates sequential independent censoring given the recorded history, so the factorization used in the recovery proof need not hold.