When a Weighted Estimate Depends on Sparse Histories
Weighting can reweight observed people to represent a target treatment strategy. The difficulty is that the target distribution may place substantial mass on histories that appear only rarely in the observed data. A weighted estimate can then be driven by a small number of observations, even when its formula is straightforward. This is an extrapolation signal: the estimate depends on information from thinly represented parts of the data, rather than on broad empirical support.
The signal needs careful interpretation. An inverse probability weight is not itself an outcome prediction, and large weights do not prove that an outcome model has extrapolated. Rather, they indicate that the target distribution is asking the observed sample to represent histories that were uncommon under the observed treatment process. Whether the resulting estimate is useful depends on how much relevant outcome information those histories provide, and on the assumptions needed to interpret the weighted data causally.
Weights as a Change of Distribution
Let \(t=0,\ldots,K\) index treatment decisions. Let \(H_t\) be the observed history just before treatment \(A_t\), including the covariates available at that time and previous treatments. Write \(g_t(a\mid h)=P(A_t=a\mid H_t=h)\) for the observed treatment mechanism, and \(q_t(a\mid h)\) for the treatment distribution under a target strategy. For simplicity, suppose these are discrete treatment distributions; the same ideas apply to densities. The cumulative treatment weight through time \(K\) is
The numerator may specify a deterministic intervention or a stochastic target strategy. When \(q_t\) assigns positive probability to an action that has very small \(g_t\) for the observed history, that factor is large. Products across time can magnify this discrepancy. The weight is therefore a measure of how much the target strategy asks an observed treatment history to represent relative to its frequency under the observed process.
Why Sparse Support Can Make Weights Signal Extrapolation
The key identity is a change of distribution. Let \(P\) denote the observed longitudinal law and let \(Q\) denote the law obtained by replacing the observed treatment mechanisms \(g_t\) with \(q_t\), while leaving the covariate transition mechanisms unchanged. Suppose the relevant conditional probabilities are well defined and \(q_t(a\mid h)>0\) only where \(g_t(a\mid h)>0\), for histories encountered under \(Q\). Then, for any integrable function \(f\) of the full record,
This identity describes the target of weighting. For a causal interpretation as an intervention outcome, additional assumptions are required: consistency, sequential exchangeability given the recorded histories, and positivity. The identity does not establish those assumptions. It also does not guarantee that a finite sample represents \(Q\) well.
Proof. If \(Q\ll P\), its Radon–Nikodym derivative \(dQ/dP\) exists. Under the stated construction, the treatment part of this derivative is the product of the conditional likelihood ratios:
The covariate transition factors cancel because \(P\) and \(Q\) use the same covariate mechanisms. The definition of the Radon–Nikodym derivative gives \(E_Q[f]=E_P[(dQ/dP)f]=E_P[W_Kf]\) for integrable \(f\). Conversely, if a set \(B\) has \(P(B)=0\) and \(Q(B)>0\), then any weighting of \(P\) still assigns zero mass to \(B\), whereas \(Q\) assigns positive mass to it. Thus such weighting cannot reproduce \(Q\) on \(B\). \(\square\)
In a sample, the population identity can be a poor guide to precision if only a few observations carry most of the weighted mass. For a region \(R\), the normalized empirical weighted mass is
where \(R_i\) indicates that person \(i\)'s record lies in the region. A region can have few observed records but a substantial \(\widehat Q_w(R)\). The weighted estimate then uses those records to represent a larger share of the target population. That representation may be mathematically appropriate under the assumptions, yet empirically fragile: one unusual outcome or a small change in the treatment model can substantially alter the result.
Worked Examples
Worked Example: A Rare History Represents Half the Target
Suppose \(R\) is a particular treatment history. In the observed data, \(P(R)=0.02\), while the target strategy assigns \(Q(R)=0.50\). In a sample of \(n=100\), the expected number of records in \(R\) is \(100(0.02)=2\).
The region-level likelihood ratio is \(0.50/0.02=25\). The complement has observed probability \(0.98\) and target probability \(0.50\), so its ratio is \(0.50/0.98\approx0.5102\). In expectation, the weighted contribution from \(R\) is \(0.02(25)=0.50\), and that from the complement is \(0.98(0.5102)\approx0.50\). Thus the rare region supplies half of the target mass despite containing only about two records in a sample of 100.
This is a clear warning of practical sparsity: the target relies heavily on a region represented by very few observations. It is not proof that the target is unsupported, since \(P(R)>0\). But estimates of outcomes within \(R\) will be sensitive to which two people happened to be observed and to their outcomes.
Worked Example: A Zero-Probability History Cannot Be Reweighted Into Existence
Suppose a binary baseline history \(H\) has two values, 0 and 1. In the observed data, people with \(H=1\) never receive treatment \(A=1\), so \(P(A=1\mid H=1)=0\). A target strategy assigns treatment \(A=1\) with probability \(0.40\) when \(H=1\).
For records with \(H=1,A=1\), the required ratio would have denominator zero and positive numerator \(0.40\). There are no such records to receive this weight. Multiplying observed records by finite weights cannot create treated outcomes for \(H=1\); their contribution remains zero, although the target assigns positive probability to that treatment-history combination.
Consequently the target outcome mean is not identified by weighting from these data without additional assumptions or information. A fitted treatment model that predicts a small positive probability in this cell may produce finite weights, but it does not change the empirical fact that the cell has no observed treated outcomes. Model-based predictions there would require extrapolation beyond observed treatment support.
Worked Example: A Single Large Weight Can Dominate an Outcome Mean
Consider four observed outcomes \(Y=(0,1,0,10)\) with weights \(W=(1,1,1,20)\). The unweighted mean is \((0+1+0+10)/4=2.75\). The weighted mean is
The fourth record contributes \(20/23\approx0.87\) of total weight and \(200/201\approx0.995\) of the weighted outcome numerator. The estimate is therefore almost entirely determined by that record. Its large weight signals high leverage on the weighted result, and if it belongs to a thinly supported history, it also signals reliance on sparse outcome information.
If the weight is truncated from 20 to 5, the estimate becomes \((50+1)/(5+1+1+1)=51/8=6.375\). This change demonstrates sensitivity to the tail; it does not show that 6.375 is less biased or closer to the target. Truncation changes the weighting rule and can trade reduced variance for increased bias.
Reading Diagnostics and Choosing a Response
Weight-tail summaries and plots across time, treatment groups, or subgroups can locate where extreme ratios arise. The interpretation becomes stronger when these summaries are paired with direct examination of how many records occupy the relevant histories and what fraction of total weighted mass they carry. A large maximum weight alone says little about whether it affects the estimate; a moderately large weight can matter greatly if it is attached to an unusual outcome or a narrow subgroup. Earlier material on effective sample size and weight leverage offers complementary summaries of concentration and influence.
A useful distinction is between practical sparsity and structural absence. With practical sparsity, the required history-treatment combination occurs, but rarely; large weights and unstable estimates are possible. With structural absence, the combination never occurs in the population under the observed process, so positivity fails. A finite-sample zero count does not by itself prove population probability is zero, but it is a warning that the data may not distinguish a very small probability from no support.
For diagnosing extrapolation signals, ask:
- Do extreme weights recur across many observations, or are they concentrated in one or two records?
- Which treatment times and histories generate the largest ratios?
- How many observations occupy those histories, and how much weighted mass do they represent?
- Does the estimate change materially under reasonable alternative weight specifications or target populations?
These checks should not be confused with a test of whether the treatment model is correctly specified. Nor do they establish that weighted groups have comparable covariate distributions; that is a separate diagnostic question. A model may fit the observed treatment assignments well while still yielding a target that depends on sparse histories.
Compared with weighting, direct restriction to a population with better observed support can reduce dependence on rare histories, but it changes the target population. Truncating weights can limit the influence of the tail, but it generally no longer implements the original target weighting exactly. Outcome regression or g-computation can use a different modeling route, but predictions for treatment-history combinations with little or no data may depend even more explicitly on extrapolation assumptions. No approach manufactures information where the data have none.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- If \(P(R)=0.04\) and \(Q(R)=0.40\), what is the region-level likelihood ratio, and what does it imply?
- Can a large weight alone establish that the target intervention is unsupported? Explain.
- What is the main consequence of truncating a weight from 20 to 5?
- Why can a treatment model that predicts a small positive probability fail to resolve a history-treatment cell with no observed outcomes?
Solutions. (1) The ratio is \(0.40/0.04=10\). The target assigns ten times the relative mass to \(R\) as the observed process, so observations in \(R\) may represent many target records; the ratio does not say how many such observations are in the sample. (2) No. A large weight signals a rare observed treatment choice relative to the target choice, but the choice may have positive population probability; inspect support and sample counts. (3) Truncation limits that record's influence but changes the weighting rule, potentially adding bias while reducing sensitivity to the tail. (4) The model's prediction does not supply observed outcomes in that cell. Estimating outcomes there requires a modeling assumption that extrapolates beyond the observed data.