Assessing Weight Balance and Extreme Influence

Why Inspect the Weights After Constructing Them?

Inverse-probability-of-censoring weighting aims to compensate for the loss of observed follow-up associated with measured covariates and history. A censoring model can produce a weight for every observed record, but the existence of those weights does not show that they work as intended. The weighted records might still have a different covariate distribution from the target population, or a small number of records might carry nearly all the weighted information.

This tutorial focuses on two related questions: does weighting improve balance on measured predictors of censoring and relevant prognostic factors? And are the resulting summaries dominated by a few observations? These are diagnostics, not proofs that censoring is conditionally independent or that an estimate is unbiased.

For a compact notation, let \(O_i(t)=1\) indicate that person \(i\)'s outcome is observed at time \(t\), and let \(w_i(t)\) be the applicable inverse-probability-of-censoring weight. For a baseline covariate \(X_i\), the weighted mean among records observed at \(t\) is

For an unstabilized weight \(w_i(t)=1/\widehat{G}_i(t)\), where \(\widehat{G}_i(t)\) is the estimated probability of remaining uncensored through \(t\) given the modeled history, this weighted mean is intended to represent the corresponding target mean under the censoring assumptions. Stabilized weights may target a different reference distribution because their numerator is not generally constant across covariate values. In that case, use a reference distribution consistent with the numerator rather than assuming the weighted mean must equal the unweighted baseline mean.

Assumptions Behind the Diagnostic

A balance check assesses how the fitted weights behave for measured variables; its interpretation rests on the same substantive conditions as the weighting procedure. The censoring model must represent the relevant predictors and history adequately, and the probability of remaining uncensored must be positive for histories in the target population. Conditional independent censoring requires that, given the modeled observed history, censoring does not carry further information about the event process relevant to the estimand. The covariates used in the comparison must also be measured and coded appropriately.

The target comparison must respect event-risk-set membership. For example, an event removes a person from a later event-free risk set for a reason other than censoring. Weighting for censoring does not reverse that event-related change. At a chosen time, compare like with like: define the target risk set and observed records using the event-history design, then diagnose whether censoring weighting addresses the selection from loss of observation within that setup.

A good balance result is not a test of conditional independent censoring. Balance on observed covariates cannot reveal imbalance in unmeasured predictors, and balance at a few selected times cannot guarantee balance at every time. Conversely, poor balance on a key measured predictor is a warning that the model, the weights, the reference distribution, or the data construction needs investigation.

How to Measure Balance

For a continuous covariate, a standardized mean difference (SMD) expresses the weighted-versus-target mean difference in units of a standard deviation. A common choice is to divide by the target-population standard deviation \(s_X\); state the choice and keep it consistent across comparisons. For a binary covariate, the target standard deviation is \(\sqrt{p(1-p)}\), where \(p\) is the target prevalence. SMDs are descriptive measures of discrepancy, not hypothesis tests; there is no universal threshold that establishes adequate balance.

Check more than a single mean. For continuous variables, compare distributions using quantiles, spread, and plots where appropriate; two distributions can have the same mean but differ in their tails or shape. For categorical variables, inspect each relevant level. Include variables used in the censoring model and important predictors of the event or estimand, along with meaningful nonlinear terms, categories, and time interactions. When covariate history changes over time, examine balance at multiple times or intervals rather than relying on baseline checks alone.

The following short justification explains why inverse weighting can recover a target mean in a simple setting. Suppose a baseline covariate \(X\) is observed for the full cohort, \(O\) indicates observation at a fixed time, and \(G(X)=P(O=1\mid X)\) is the true observation probability. Assume \(G(X)>0\) and that the observation model is correctly specified. Then

Taking expectations over \(X\) gives \(E[OX/G(X)]=E[X]\). In a finite sample, the normalized weighted mean estimates this target mean. In event-history data, the same reasoning must be applied to the relevant risk-set and time-specific observation process; the simple identity does not remove event-related selection or excuse a misspecified model.

Extreme Weights and Concentrated Influence

Balance alone is insufficient. A weighted mean can match the target mean because a few records receive very large weights. A useful summary of concentration is the effective sample size (ESS):

ESS equals the number of records when all included weights are equal. It falls as weight variability increases; it is a measure of weight concentration, not a replacement for the actual sample size and not, by itself, a variance estimate. Also inspect the maximum weight, upper quantiles, and the share of total weight held by the largest observations. For a particular weighted estimate, leave-one-out changes or each record's contribution to the numerator and denominator can help reveal influence. Such checks are screening tools, not formal inference.

Truncation, Comparisons, and Failure Modes

Truncating or capping weights can reduce their maximum and increase ESS, but it changes the weighted pseudo-population and can reintroduce measured imbalance. Therefore, compare the same balance summaries, weight distribution, and estimate under the original and truncated weights. Do not choose a truncation rule solely because it makes an outcome estimate more appealing. The earlier material on Positivity Diagnostics for Censoring Weights addresses limited support; balance and influence checks complement, rather than replace, those diagnostics.

Weight diagnostics should also be distinguished from model diagnostics. A censoring model can predict who is censored well yet yield poor weighted balance, or have modest predictive discrimination while giving acceptable balance for variables relevant to the estimand. Calibration of predicted censoring probabilities and observed-versus-predicted censoring patterns can provide additional evidence about model fit. Neither predictive performance nor balance alone verifies the conditional independence assumption.

Common warning signs include a covariate still imbalanced after weighting, a very large maximum weight, low ESS relative to the number of observed records, major changes when a single record is omitted, and large discrepancies between full and truncated analyses. Investigate coding, time alignment, missing histories, model functional form, and the chosen reference population. Avoid interpreting a small SMD as proof of no selection: balance is limited to the measured variables and summaries examined.