Why HDPS Prioritizes Features
Claims and administrative records can contain many candidate features, including diagnoses and procedures that may act as proxies for health status or access to care. Adjusting for every recorded feature is not automatically a good strategy: many may be uninformative, noisy, or too uncommon to support stable estimation. The high-dimensional propensity score (HDPS) algorithm uses observed data to prioritize candidates for adjustment. Its ranking is a screening device, not a causal verdict about what any feature represents.
The process begins with the candidate features defined from the data dimensions. The earlier material on Encoding Diagnosis and Procedure Histories establishes how histories can be represented as features. Here, the focus is on how to rank those candidates using their prevalence and their observed associations with treatment and outcome.
Prevalence and Association Are Different Signals
Prevalence is a first-pass screening signal. A feature observed in very few people may be difficult to estimate reliably and may contribute little to population-level adjustment. A more prevalent feature provides more information for comparison, although prevalence alone does not say whether the feature is related to treatment or outcome. A common feature can be irrelevant to both, while an uncommon feature can be strongly associated with them.
For a binary feature \(C\) and binary treatment \(A\), let \(p_1=P(C=1\mid A=1)\) and \(p_0=P(C=1\mid A=0)\). The difference \(p_1-p_0\) describes the direction and size of the prevalence imbalance across treatment groups. The outcome association can be summarized, for example, by the risk ratio \(RR_{CY}=P(Y=1\mid C=1)/P(Y=1\mid C=0)\), when those risks are estimable. A value above 1 indicates a higher observed outcome risk among people with the feature; a value below 1 indicates a lower observed risk. Neither comparison alone demonstrates a causal relationship.
Association measures can also be expressed as odds ratios. For a two-by-two table with cell counts \(n_{11}, n_{10}, n_{01}, n_{00}\), the odds ratio is \(n_{11}n_{00}/(n_{10}n_{01})\). This follows by dividing the odds in one group, \(n_{11}/n_{10}\), by the odds in the comparison group, \(n_{01}/n_{00}\). An odds ratio of 1 indicates no observed association on the odds scale. Ranking by distance from 1 is often expressed as the absolute log odds ratio, since \(\log(1)=0\) and equal reciprocal odds ratios have equal absolute log values. This measure describes association strength, not confounding by itself.
Worked Example: Screening Candidates by Prevalence
Suppose a hypothetical claims cohort contains 2,000 eligible people. Before examining treatment and outcome associations, an analyst records the counts for four binary candidate features.
| Feature | People with feature | Prevalence |
|---|---|---|
| F1 | 600 | 600/2,000 = 0.300 |
| F2 | 240 | 240/2,000 = 0.120 |
| F3 | 20 | 20/2,000 = 0.010 |
| F4 | 1,000 | 1,000/2,000 = 0.500 |
The prevalence ranking is F4 (0.500), F1 (0.300), F2 (0.120), then F3 (0.010). If the analysis has a prespecified screening limit, this order can help decide which features proceed to association assessment. It does not mean F4 is the most important confounder: its treatment and outcome associations have not yet been examined. Nor does the low prevalence of F3 prove it should never be considered; a rare feature may still warrant attention for substantive reasons or under a prespecified design.
Combining Treatment and Outcome Information
A feature is a more plausible source of confounding bias when it is associated with treatment and also predicts the outcome. HDPS therefore assesses both sides of this relationship rather than selecting features solely because they predict treatment. One standard screening summary for a binary feature is a bias factor based on its prevalence in each treatment group and its outcome risk ratio. For this summary, write \(p_1=P(C=1\mid A=1)\), \(p_0=P(C=1\mid A=0)\), and \(RR_{CY}\) for the observed outcome risk ratio comparing those with and without \(C\).
The factor is a screening approximation under a binary-feature risk-ratio formulation: it assumes the feature’s outcome association is suitably represented by \(RR_{CY}\) across treatment groups and does not model effect modification. A value farther from 1 indicates greater potential for the feature’s imbalance and outcome association to shift a crude treatment–outcome risk ratio. Ranking by \(|\log(BF_C)|\) puts factors above and below 1 on a common scale. This is not a substitute for a causal model, and the score is not an estimate of the feature’s causal effect.
Worked Example: Calculating a Feature’s Bias Factor
In a hypothetical cohort, 30% of treated people and 10% of untreated people have feature C, so \(p_1=0.30\) and \(p_0=0.10\). The observed outcome risks are 12% among people with C and 6% among people without C. Thus \(RR_{CY}=0.12/0.06=2.00\).
Substitute these values into the screening formula:
The factor is above 1 because the feature is more prevalent in the treated group and is associated with higher observed outcome risk. Under the formula’s assumptions, the feature could inflate a crude treatment-group risk ratio by a factor of about 1.182. This is a screening interpretation, not a claim that the feature causes the outcome or that the actual treatment effect is biased by exactly this amount after all other covariates are considered. The corresponding ranking quantity is \(|\log(1.182)|\), approximately 0.167.
Comparing Candidate Rankings
The next hypothetical example illustrates why both associations matter. Suppose the following values have been estimated for three features. The table uses the same binary-feature risk-ratio formulation as above.
Worked Example: Ranking by Combined Potential
| Feature | \(p_1\) | \(p_0\) | \(RR_{CY}\) | \(BF_C\) | \(|\log(BF_C)|\), approximately |
|---|---|---|---|---|---|
| A | 0.30 | 0.10 | 2.00 | 1.182 | 0.167 |
| B | 0.45 | 0.25 | 1.30 | 1.056 | 0.054 |
| C | 0.22 | 0.20 | 3.00 | 1.029 | 0.028 |
For A, the calculation is \([1+0.30(2.00-1)]/[1+0.10(2.00-1)]=1.30/1.10=1.182\). For B, it is \([1+0.45(1.30-1)]/[1+0.25(1.30-1)]=1.135/1.075=1.056\). For C, it is \([1+0.22(3.00-1)]/[1+0.20(3.00-1)]=1.44/1.40=1.029\). The ranking by absolute log factor is A, then B, then C.
C has the largest outcome risk ratio, but its prevalence differs little between treatment groups. Its combined factor is consequently close to 1. B is more imbalanced than C, but its outcome association is weaker. A ranks highest because it has both a noticeable treatment-group imbalance and an outcome association. The ranking reflects the combination, not a judgment that A is necessarily a true confounder.
From Ranking to Adjustment
In practice, HDPS implementations first identify eligible features and screen them using prevalence, then estimate treatment and outcome associations for candidates that pass the screening stage. A chosen prioritization rule orders the candidates, after which a prespecified number or set may be taken forward for adjustment. The number selected and the details of the feature matrix are separate design choices; a ranking does not determine them automatically. In particular, this tutorial’s ranking logic should not be confused with a general rule to include every feature with a large score.
The main alternatives have different limitations. Ranking by prevalence alone favors common features but ignores their relationship to treatment and outcome. Ranking only by treatment prediction can elevate features that explain treatment assignment without being outcome predictors, a problem emphasized in the earlier material on Treatment Prediction Versus Confounder Control. Ranking only by outcome association can favor strong prognostic features that are similarly distributed between treatment groups. Combining both signals is intended to prioritize candidates relevant to both sides of confounding, while leaving substantive and design checks essential.
Empirical associations are sensitive to measurement and sampling. Sparse features can have unstable estimates or very wide uncertainty; strongly correlated features can provide redundant signals; and estimates based on many candidates can be noisy. A feature’s measured association may also reflect coding practices, healthcare contact, or other processes rather than the underlying clinical state it is intended to proxy. For outcomes that are not rare, an odds ratio can differ materially from a risk ratio, so a risk-ratio bias-factor formula should not be applied to odds ratios as though they were identical. The analyst should state the association scale, the screening rule, and any approximations used.
Diagnostics and Common Mistakes
Before relying on a ranking, check the candidate counts and denominators, confirm that prevalence is computed in the intended eligible cohort, and inspect treatment and outcome associations for plausible direction and precision. Verify that association estimates are not dominated by empty or nearly empty cells. Compare the highest-ranked candidates with clinical and subject-matter knowledge, and examine whether the ordering changes under reasonable choices of prevalence threshold or association measure. After adjustment, assess whether the resulting design achieves acceptable covariate balance; a high ranking score does not guarantee balance, and a low score does not prove balance is adequate.
- Mistake: “This feature is most prevalent, so it must be the most important confounder.” Correction: prevalence is an initial screening signal; treatment and outcome associations, along with substantive knowledge, also matter.
- Mistake: “A strong outcome association alone makes a feature a confounder.” Correction: the feature must also be distributed differently across treatment groups to contribute to this form of confounding.
- Mistake: “A high HDPS score proves the feature causes the outcome.” Correction: the score summarizes observed associations and is a prioritization aid, not causal evidence.
- Mistake: “The top-ranked features are automatically safe to adjust for.” Correction: ranking cannot replace causal and design review; empirical associations alone do not establish that adjustment is appropriate.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- A feature is present in 40% of treated people and 39% of untreated people, and has an outcome risk ratio of 2.5. Why might its combined bias factor remain close to 1?
- For a feature, \(p_1=0.20\), \(p_0=0.10\), and \(RR_{CY}=1.5\). Calculate its bias factor.
- Why can prevalence-only ranking miss an important candidate?
- What does a high absolute log bias factor establish, and what does it not establish?
- Name two checks to perform before treating a ranked feature as a useful adjustment candidate.
Solutions. (1) The feature is almost equally prevalent in the two treatment groups, so the imbalance component is small even though its outcome association is strong. (2) \(BF_C=[1+0.20(1.5-1)]/[1+0.10(1.5-1)]=1.10/1.05=1.048\), approximately. (3) A less prevalent feature may still have substantial treatment and outcome associations; prevalence alone does not measure either association. (4) It indicates a larger screening summary of potential bias under the formula’s assumptions; it does not prove causal confounding or quantify the final adjusted treatment effect. (5) For example, inspect cell counts and estimate precision, confirm the denominator and association scale, review clinical plausibility, and assess balance after adjustment.