From Claims Records to a Causal Question
A claims-based high-dimensional propensity score (HDPS) analysis is not simply a large prediction exercise. Its purpose is to compare treatment groups that are more alike on measured, pretreatment characteristics, while keeping the population and causal question explicit. A long feature list cannot repair a poorly defined cohort, a variable recorded after treatment, or an unmeasured cause of both treatment and outcome.
Consider a fully hypothetical study of adults with a chronic condition who initiate Drug A or an active-comparator Drug B. The outcome is whether a patient has at least one all-cause hospitalization during the 12 months after initiation. Let \(A=1\) denote initiating Drug A and \(A=0\) initiating Drug B. Let \(Y\) be the binary hospitalization outcome, and let \(X\) denote measured pretreatment characteristics, including claims-derived features.
The target estimand here is the average treatment effect (ATE) in the eligible cohort: the difference between the cohort’s average potential outcome under Drug A and its average potential outcome under Drug B. For a binary outcome, this is a risk difference, expressed in percentage points. The estimand concerns treatment strategies at initiation and the specified 12-month outcome window; it is not automatically the effect in every population or under continued treatment.
Define the Cohort and the Feature Window
Worked Example: Setting the Claims-Based Cohort
Suppose a hypothetical database contains 20,000 eligible adults who meet all of these criteria: a recorded diagnosis of the chronic condition, at least 12 months of continuous enrollment before treatment initiation, no claim for either study drug during that lookback period, and a first observed initiation of Drug A or Drug B during the enrollment window. There are 8,000 Drug A initiators and 12,000 Drug B initiators.
Set each person’s index date to the date of first initiation of either drug. The 12 months ending immediately before that date are the lookback period for baseline claims. Follow-up for the outcome begins at the index date and lasts 12 months, subject to a prespecified rule for disenrollment or death. This aligns eligibility, treatment assignment, and the start of outcome follow-up.
A diagnosis, procedure, medication, or utilization measure is a baseline candidate only if its defining information falls in the lookback period. A claim recorded after the index date cannot be treated as a baseline confounder just because it appears in the same enrollment record. Eligibility and feature rules should be applied in the same way to both treatment groups.
The resulting estimand is the ATE among these 20,000 eligible initiators, not an effect among all adults with the condition. Continuous enrollment helps establish observable history; it does not establish that all clinically important history is recorded.
Claims features often begin as code-based indicators: whether a person had a diagnosis, procedure, or medication claim in a defined lookback window. They can also include prespecified counts or utilization summaries. The feature matrix should document the code lists, time windows, handling of repeated claims, and any collapsing of codes into broader categories. These choices determine what a feature means.
The key temporal rule is simple: construct \(X\) from information available before treatment initiation. Including post-index utilization can condition on a consequence of treatment, a consequence of early outcome, or both. Such adjustment can create bias rather than remove it.
Screen and Select High-Dimensional Features
A claims database may generate thousands of candidate features, many of which are rare or redundant. A practical HDPS workflow defines candidate dimensions in advance (for example, diagnoses, procedures, and medications), applies consistent prevalence rules, and then narrows the candidates using a prespecified selection approach. Clinically established confounders should be retained even if they are uncommon; an automated prevalence screen is not a substitute for subject-matter judgment.
Worked Example: Prioritizing Claims Features
Suppose the 20,000-person cohort yields 2,400 candidate claims indicators after applying the prespecified lookback and prevalence rules. Among these candidates are a prior COPD-related visit, a prior oral steroid claim, and a preventive-care visit. In the hypothetical cohort, their proportions are as follows.
| Baseline feature | Drug A group | Drug B group | Absolute difference |
|---|---|---|---|
| Prior COPD-related visit | 18% | 8% | 10 percentage points |
| Prior oral steroid claim | 10% | 5% | 5 percentage points |
| Prior preventive-care visit | 12% | 25% | 13 percentage points |
For the COPD feature, the treatment-group difference is \(0.18-0.08=0.10\), or 10 percentage points. For the preventive-care feature, it is \(0.12-0.25=-0.13\), so its absolute difference is 13 percentage points. These differences flag measured treatment-group separation; they do not show that either feature is a confounder or that adjusting for it will necessarily reduce bias.
In this illustrative workflow, a prespecified treatment-association screen prioritizes candidates from the eligible claims dimensions, while known clinical confounders are forced into the adjustment set. A sparse logistic propensity model is then fitted to the retained baseline features, with its penalty tuned for causal balance rather than classification accuracy alone. The observed study outcome is not used to choose features in this example. That restriction reduces one route to outcome-driven overfitting, but it does not prove that the retained set captures all confounding.
The fitted propensity model can be written as a logistic regression,
A penalty can shrink some \(\beta_j\) values to zero, producing a sparse model. The resulting scores are estimates of treatment probabilities conditional on the included features; they are not probabilities that an individual will benefit from treatment. The choice of penalty and selected features must be checked against balance and overlap, not defended solely by a good treatment-prediction score.
Balance, Overlap, and the Weighted Estimate
Under the assumptions stated below, the ATE can be estimated with inverse-probability weights. For a person with \(A=1\), the ATE weight is \(1/e(X)\); for a person with \(A=0\), it is \(1/\{1-e(X)\}\). The weighted treatment groups represent the same measured baseline population when the propensity model has achieved adequate balance and the scores are not too close to zero or one.
The weighting identity follows by conditioning on \(X\). Under conditional exchangeability and consistency,
Averaging over the baseline covariate distribution gives \(E\{AY/e(X)\}=E\{Y(1)\}\). The corresponding control identity is \(E[\{1-A\}Y/\{1-e(X)\}]=E\{Y(0)\}\). In a finite sample, normalized weighted means estimate these expectations; extreme weights make those estimates unstable even if the algebra is valid.
Worked Example: Checking a Binary Feature After Weighting
Suppose the weighted prevalence of a baseline binary feature is 30% in Drug A initiators and 26% in Drug B initiators. A standardized mean difference (SMD) for a binary feature can be calculated as the difference in proportions divided by the square root of the average of the two Bernoulli variances:
The weighted prevalence difference is 4 percentage points and the SMD is about 0.0892. This is below the commonly used descriptive threshold of 0.10, suggesting small measured imbalance for this feature. It is not a hypothesis test, does not establish exact balance, and says nothing directly about unmeasured confounding.
Balance diagnostics should cover the retained features and important baseline variables, not just a few prominent claims codes. Inspect distributions of estimated scores and weights, including regions with little treatment-group overlap. Large or highly variable weights indicate that some observations have substantial influence; trimming or changing the target population may be considered, but either choice changes the analysis and should be reported transparently.
Worked Example: Estimating the ATE With Stabilized Score Strata
For a transparent arithmetic illustration, suppose the fitted scores divide the 20,000 people into two groups. In a lower-score group of 10,000, 2,000 initiate Drug A and 8,000 Drug B, corresponding to \(e(X)=0.20\). In a higher-score group of 10,000, 6,000 initiate Drug A and 4,000 Drug B, corresponding to \(e(X)=0.60\). Suppose the observed hospitalization risks are 8% and 10% for Drug A and Drug B, respectively, in the lower-score group, and 6% and 9% in the higher-score group.
The Drug A weights are \(1/0.20=5.00\) in the lower-score group and \(1/0.60=1.667\) in the higher-score group. The weighted Drug A denominator is \(2{,}000(5.00)+6{,}000(1.667)\approx20{,}000\). Its weighted event numerator is \(2{,}000(0.08)(5.00)+6{,}000(0.06)(1.667)\approx1{,}400\). Therefore, the weighted Drug A risk is \(1{,}400/20{,}000=0.0700\), or 7.00%.
The Drug B weights are \(1/(1-0.20)=1.25\) and \(1/(1-0.60)=2.50\). Its weighted denominator is \(8{,}000(1.25)+4{,}000(2.50)=20{,}000\). Its weighted event numerator is \(8{,}000(0.10)(1.25)+4{,}000(0.09)(2.50)=1{,}900\). Thus, the weighted Drug B risk is \(1{,}900/20{,}000=0.0950\), or 9.50%.
The estimated ATE risk difference is \(0.0700-0.0950=-0.0250\): in this hypothetical analysis, Drug A is associated with an estimated 2.50-percentage-point lower 12-month hospitalization risk than Drug B in the eligible cohort. A causal interpretation requires the stated assumptions; the arithmetic alone does not show that those assumptions hold. No uncertainty interval is calculated here.
Diagnostics, Alternatives, and Limitations
A well-executed analysis checks the design as well as the fitted scores. Confirm that every feature was measured before the index date, verify that repeated claims and missing enrollment are handled consistently, and look for coding patterns that may differ between treatment groups. After fitting, examine balance for important individual covariates and feature blocks, score overlap, weight tails, and effective sample size. Persistent imbalance calls for revisiting feature construction, model specification, or the target population rather than declaring success from the model’s predictive performance.
| Approach | Role in this setting | Important caution |
|---|---|---|
| HDPS with a sparse propensity model | Uses structured claims candidates to expand measured baseline adjustment. | Automated selection cannot identify unrecorded causes or guarantee balance. |
| Manually specified propensity model | Can emphasize established clinical confounders and interpretable covariates. | May omit relevant claims history if specification is too narrow. |
| Matching or flexible propensity learners | Can be alternatives for constructing comparisons or estimating treatment probabilities. | Each still requires overlap, diagnostics, and an estimand that matches the resulting analysis. |
Claims data may omit clinical severity, patient preferences, over-the-counter use, or prescribing judgment. A code can also be a noisy proxy for a condition, and coding intensity may vary with care access. Consequently, a low SMD across thousands of recorded features is evidence about measured covariate balance, not proof of conditional exchangeability. The estimate is also sensitive to the cohort definition, claims-code granularity, and lookback period.
- Post-treatment adjustment: a claim after initiation is not a baseline confounder merely because it predicts the outcome. Restrict baseline features to the prespecified pre-index window.
- Prediction mistaken for causal quality: high treatment classification accuracy can accompany poor overlap or residual imbalance. Select and tune with balance and the intended estimand in view.
- Positivity hidden by an average: a tolerable average weight can coexist with a small number of extreme weights. Inspect the full weight distribution and influential observations.
- Balance mistaken for exchangeability: matching measured features does not demonstrate that unmeasured confounding is absent. State the limitation directly.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A feature is a diagnosis recorded two months after the index date. Should it be included as a baseline HDPS feature? Why?
- For a binary feature, weighted prevalence is 30% in Drug A and 26% in Drug B. Calculate its SMD using the average Bernoulli variance.
- For a person with \(e(X)=0.80\) who initiated Drug A, what is the ATE inverse-probability weight?
- Name one reason claims-based balance may coexist with residual confounding.
Solutions. (1) No. It occurs after treatment assignment and is not a baseline feature; adjusting for it may introduce post-treatment bias. (2) The SMD is \(0.04/\sqrt{\{0.30(0.70)+0.26(0.74)\}/2}=0.04/\sqrt{0.2012}\approx0.0892\). (3) The weight is \(1/0.80=1.25\). (4) Claims may not capture a confounder such as clinical severity or prescribing judgment, so conditional exchangeability may fail despite measured balance.