Tutorials › Marginal Structural Models › Hand-Calculating Longitudinal Censoring Weights

Censoring and Informative Dropout · Tutorial 223 of 501

Hand-Calculating Longitudinal Censoring Weights

Work through longitudinal censoring probabilities and individual weights by hand, and learn how to check each step for errors or unstable values.

Advanced 12 min read

What You'll Learn

  • Distinguish a censoring hazard from the conditional probability of remaining observed.
  • Multiply conditional retention probabilities to obtain a cumulative observation probability.
  • Calculate an individual’s inverse-probability weight at each observed period.
  • Stop a person’s weight calculation at the last period with an observed outcome.
  • Check calculations for incorrect denominators, rounding errors, and near-positivity violations.
  • Compare unstabilized weights with stabilized weights and complete-case analysis.

From Conditional Retention to an Individual Weight

A longitudinal censoring weight is built one follow-up period at a time. At each period, the calculation uses the probability of remaining observed, conditional on the history available when the censoring decision is made. Those conditional probabilities are then multiplied along the individual’s observed history. This hand calculation makes the role of each model prediction explicit: a small probability of remaining observed at any period can substantially increase the weight.

Fix \(K\) follow-up periods and let \(R_t=1\) mean that an individual is still uncensored and has the outcome observed through period \(t\). Set \(R_0=1\). Let \(H_t\) denote the observed history available just before the censoring decision for period \(t\). For someone at risk of censoring at that period, define the conditional retention probability \(q_t(H_t)\) and censoring hazard \(h_t(H_t)\) as follows.

Definition: The conditional retention probability is \(q_t(H_t)=P(R_t=1\mid R_{t-1}=1,H_t)\). The conditional censoring hazard is \(h_t(H_t)=P(R_t=0\mid R_{t-1}=1,H_t)=1-q_t(H_t)\). For an individual observed through period \(t\), the cumulative observation probability is \(G_t=\prod_{j=1}^{t}q_j(H_j)\), and the unstabilized inverse-probability-of-censoring weight is \(W_{C,t}=1/G_t\).

The two probabilities \(q_t\) and \(h_t\) refer to complementary outcomes among people still observed through period \(t-1\). A model may predict the censoring hazard, in which case calculate \(q_t=1-h_t\) before forming the product. Do not multiply censoring hazards to obtain a retention probability.

The calculation assumes that the conditional probabilities are defined among those at risk, that the histories used to evaluate them are available before each censoring decision, and that retention has positive probability along the histories being weighted: \(q_t(H_t)>0\). For an IPCW interpretation, the sequential independent censoring assumption also matters: conditional on the recorded history, censoring at a period does not further select on the future outcomes of interest. Correct specification of the conditional probability model is needed when its fitted probabilities are used as estimates of these probabilities.

Why the Product Uses Each Period’s Conditional Probability

The cumulative probability is not a sum of period-specific probabilities. Remaining observed through period \(t\) requires remaining observed at every earlier period. Along an individual’s history, the chain rule gives the following result.

Result: Under the additional full-history condition stated in the derivation, the probability of remaining observed through period \(t\) along a history path is the product of \(q_1(H_1),\ldots,q_t(H_t)\). The inverse weight is the reciprocal of that product.

Derivation. Conditional on a complete history path \(\bar H_t=\bar h_t\), the chain rule gives \(P(R_1=\cdots=R_t=1\mid\bar H_t=\bar h_t)=P(R_1=1\mid\bar H_t=\bar h_t)\prod_{j=2}^{t}P(R_j=1\mid R_1=\cdots=R_{j-1}=1,\bar H_t=\bar h_t)\). For this full-path factorization, assume the stronger condition that for every \(j\), \(P(R_j=1\mid R_1=\cdots=R_{j-1}=1,\bar H_t=\bar h_t)=P(R_j=1\mid R_{j-1}=1,H_j=h_j)=q_j(h_j)\), with the prior-retention condition empty for \(j=1\). The usual outcome-specific sequential independent censoring assumption alone does not imply this equality. Under this stronger condition, \(G_t(\bar h_t)=P(R_1=\cdots=R_t=1\mid\bar H_t=\bar h_t)=\prod_{j=1}^{t}q_j(h_j)\). Taking the reciprocal gives \(W_{C,t}=1/G_t(\bar h_t)\). \(\square\)

The product is individual-specific when the model uses histories that vary across people. Even when two people have the same baseline characteristics, their predicted retention probabilities can diverge as their observed histories diverge. A person censored before period \(t\) has no observed period-\(t\) outcome to weight: calculate the weight only through that person’s last observed period.

Worked Example: Three Periods With Given Retention Probabilities

Suppose an individual remains observed through periods 1 and 2, and the conditional retention probabilities along the individual’s history are \(q_1=0.80\), \(q_2=0.75\), and \(q_3=0.50\). Calculate the cumulative observation probability and weight at each period.

At period 1, \(G_1=0.80\), so \(W_{C,1}=1/0.80=1.25\).

At period 2, \(G_2=0.80(0.75)=0.60\), so \(W_{C,2}=1/0.60=1.6666\ldots\).

At period 3, \(G_3=0.80(0.75)(0.50)=0.30\), so \(W_{C,3}=1/0.30=3.3333\ldots\). Equivalently, using the unrounded period-2 weight, \(W_{C,3}=(1/0.60)(1/0.50)=3.3333\ldots\). The product of the displayed rounded value \(1.667\) and 2 is \(3.334\), so use unrounded values for intermediate calculations when accuracy matters.

The period-3 weight is larger because only 30% of individuals with this sequence of histories are predicted to remain observed through period 3. This example does not mean that every individual has the same weight: another history can yield different conditional probabilities.

Reading Censoring Probabilities From a Small Dataset

When a table gives a censoring hazard rather than a retention probability, first take its complement. Then multiply only the retention probabilities for periods in which the individual was still at risk and observed through the period. The following table gives model-based retention probabilities along each person’s history.

Person\(q_1\)\(q_2\)\(q_3\)Observed through
A0.800.750.50Period 3
B0.800.500.50Period 3
C0.800.75—Period 1; censored before period 2

For a person censored before period 2, \(q_2\) is not included in the weight for an observed period-2 outcome because that outcome is not observed. The probability \(q_2\) describes the chance of remaining observed into period 2; it is not an additional weight factor for an outcome that is unavailable.

Worked Example: Comparing Weights Across Observed Histories

Use the table above to calculate each person’s weight at their last observed period.

Person A. A is observed through period 3. The cumulative probability is \(G_3=0.80(0.75)(0.50)=0.30\), so the weight is \(1/0.30=3.3333\ldots\).

Person B. B is observed through period 3. The cumulative probability is \(G_3=0.80(0.50)(0.50)=0.20\), so the weight is \(1/0.20=5\).

Person C. C has an observed period-1 outcome but is censored before period 2. At the last observed period, \(G_1=0.80\), so the weight is \(1/0.80=1.25\). Do not multiply by \(0.75\): C did not remain observed through period 2 and has no observed period-2 outcome to reweight.

B’s larger period-3 weight reflects the smaller cumulative probability of being observed along B’s history. It does not indicate that B’s outcome is intrinsically more important; the weight compensates for the lower modeled chance of observing an outcome from a person with that history.

Calculating From a Fitted Censoring Model

In practice, a fitted discrete-time model may return predicted censoring hazards. For each period in the person’s observed risk history, convert the predicted hazard to a retention probability and then accumulate the product. This is arithmetic on model predictions, not a separate fitting procedure.

Worked Example: Converting Predicted Hazards to a Weight

For one person, suppose a fitted model gives censoring hazards \(h_1=0.20\), \(h_2=0.30\), and \(h_3=0.50\). The person remains observed through all three periods. Calculate the weight at each period.

The retention probabilities are \(q_1=1-0.20=0.80\), \(q_2=1-0.30=0.70\), and \(q_3=1-0.50=0.50\).

At period 1, \(G_1=0.80\) and \(W_{C,1}=1/0.80=1.25\).

At period 2, \(G_2=0.80(0.70)=0.56\) and \(W_{C,2}=1/0.56=1.785714\ldots\).

At period 3, \(G_3=0.80(0.70)(0.50)=0.28\) and \(W_{C,3}=1/0.28=3.571428\ldots\). Multiplying sequentially gives the same answer when unrounded values are retained: \(1.25(1/0.70)(1/0.50)=3.571428\ldots\).

A quick check is that each retention probability lies between 0 and 1, each cumulative probability is no larger than the previous one, and each unstabilized weight is no smaller than the previous one. Here \(0.80\), \(0.56\), and \(0.28\) decrease, while \(1.25\), \(1.785714\ldots\), and \(3.571428\ldots\) increase.

What the Weight Corrects and How to Check It

The following identity explains why the inverse probability has this form. In a finite or discrete history space, let \(\bar H_t\) denote a complete history path through period \(t\), let \(f(\bar H_t)\) be any function of that path, and let \(G_t(\bar H_t)=P(R_t=1\mid\bar H_t)\) be the cumulative retention probability along it. Under the stronger full-history condition stated in the derivation and positivity, inverse weighting recovers the full-history expectation:

Result: If \(G_t(\bar H_t)>0\) on every history path with positive probability, then \(E[R_t f(\bar H_t)/G_t(\bar H_t)]=E[f(\bar H_t)]\), provided the expectations exist.

Proof. Sum over the possible history paths. Conditional on a path, \(R_t\) is an indicator with conditional mean \(G_t(\bar H_t)\). Therefore,

$$ \begin{aligned} E\left[\frac{R_t f(\bar H_t)}{G_t(\bar H_t)}\right] &=\sum_{\bar h_t}P(\bar H_t=\bar h_t) E\left[\left.\frac{R_t f(\bar h_t)}{G_t(\bar h_t)}\right|\bar H_t=\bar h_t\right]\\ &=\sum_{\bar h_t}P(\bar H_t=\bar h_t) \frac{f(\bar h_t)P(R_t=1\mid\bar H_t=\bar h_t)}{G_t(\bar h_t)}\\ &=\sum_{\bar h_t}P(\bar H_t=\bar h_t)f(\bar h_t) =E[f(\bar H_t)]. \end{aligned} $$

The cancellation in the second line is valid because \(G_t(\bar h_t)\) is the conditional observation probability for that same history path, and positivity makes the denominator nonzero. In applications the probabilities are estimated, so the identity motivates the weighting but does not guarantee that estimated weights will exactly balance a finite sample. \(\square\)

Useful checks on hand calculations include verifying that \(h_t+q_t=1\), that all probabilities are in \([0,1]\), and that the cumulative probability is the product of the conditional retention probabilities rather than a product of hazards. A probability near zero is a warning: its reciprocal is large, and multiplying several such reciprocals can produce extreme weights. Such values can signal weak positivity, sparse histories, or a poorly supported model prediction.

A common mistake is to use a period’s hazard as though it were the probability of remaining observed. If \(h_t=0.30\), the probability used in the retention product is \(q_t=0.70\), not \(0.30\). Another mistake is to carry rounded intermediate weights into later products. Keep full precision, or multiply the original probabilities and invert once. Finally, only calculate a period-\(t\) weight for a person with an observed period-\(t\) outcome.

Alternatives and Limits of the Calculation

Complete-case analysis simply discards people censored before the analysis endpoint. It does not use the history-specific retention probabilities and can be biased when continued observation selects on prognostic history. IPCW instead gives more weight to observed people whose histories predict lower retention, subject to the assumptions above. Stabilized weights use a numerator probability as well as the denominator; they can reduce variability, but their calculation and interpretation depend on the chosen numerator and are distinct from the unstabilized inverse weights calculated here.

Hand calculation is useful for tracing a few people and validating software output, but it is not a substitute for checking the censoring model. Estimated probabilities can be inaccurate, and weights can become highly variable even when every probability is technically positive. Inspect the distribution of predicted retention probabilities and resulting weights, identify which histories produce the largest values, and compare hand calculations against the software’s period-specific predictions. Weight truncation may be considered in an analysis, but it changes the weights and can introduce bias; it does not repair a failure of positivity or an incorrect model.

These weights address selection due to censoring under the stated assumptions. They do not, by themselves, establish that the causal estimand is identified or specify how to estimate it. Those are separate parts of an analysis.

Check Your Understanding

Try each calculation, then compare it with the solutions.

  1. A person remains observed through two periods with \(q_1=0.90\) and \(q_2=0.80\). What are \(G_2\) and \(W_{C,2}\)?
  2. A model predicts a period-2 censoring hazard of \(0.25\). What retention probability belongs in the product?
  3. A person has \(q_1=0.80\), is censored before period 2, and has an observed period-1 outcome. What weight applies to that period-1 outcome?
  4. Why can a small cumulative observation probability produce an unstable weight?

Solutions. (1) \(G_2=0.90(0.80)=0.72\), so \(W_{C,2}=1/0.72=1.388888\ldots\). (2) \(q_2=1-0.25=0.75\). (3) The last observed period is period 1, so the weight is \(1/0.80=1.25\); do not include a period-2 factor. (4) The weight is the reciprocal of the cumulative probability, so as \(G_t\) approaches zero, \(1/G_t\) grows without bound; a few such weights can dominate weighted calculations.

Clinical Biostats