Why Sparse Categories Matter
A categorical covariate may have many possible levels, while only a few people in the data have some of them. In a logistic propensity model, an indicator for a rare level is informed by the treatment assignments among those few people. If one or both treatment groups are scarcely represented, the estimated treatment odds for that level can be highly uncertain. A large or erratic coefficient is not, by itself, evidence of a large treatment-assignment difference; it may chiefly reflect limited information.
The issue matters because the propensity score is used to adjust for measured covariates, not merely to predict treatment well. A coefficient that varies sharply with a small number of observations can produce unstable fitted scores. Conversely, pooling categories without a defensible reason can hide meaningful treatment-assignment differences and weaken adjustment for the original covariate.
Model and the Source of Uncertainty
Let \(T_i\) indicate treatment for person \(i\), with \(T_i=1\) for treated and \(T_i=0\) for untreated, and let \(X_i\) denote measured covariates. The propensity score is \(e(X_i)=P(T_i=1\mid X_i)\). For a categorical covariate with levels A, B, and C, a logistic model using C as the reference level can be written
Here the model assumes independent treatment assignments conditional on the modeled covariates, a correctly specified logit-linear form for the included terms, and maximum-likelihood estimation. With only this categorical covariate, the model is saturated across its levels: the fitted treatment probability in each level is that level’s observed treated proportion. This simple setting makes the instability visible. In a larger model, a category coefficient is generally not just a two-by-two table contrast, but limited counts can still leave little information for estimating the corresponding parameters.
For a level with \(a\) treated and \(b\) untreated people, its observed treated proportion is \(\hat p=a/(a+b)\). Its estimated log odds are \(\log(a/b)\). The approximate variance of this logit follows from the binomial delta method. If \(n=a+b\), then the binomial variance of \(\hat p\) is \(p(1-p)/n\), and the derivative of \(\operatorname{logit}(p)\) is \(1/\{p(1-p)\}\). Substituting the estimated proportion gives
For the coefficient comparing one level with a reference level, the two levels contain independent observations. Their estimated log odds therefore have approximate variances that add. If the comparison level has counts \(c\) treated and \(d\) untreated, the estimated coefficient and its approximate standard error are
This approximation relies on the binomial model and is most useful when the counts are not so small that the normal approximation is plainly unreliable. The reciprocal-count terms also show why a small cell can dominate the uncertainty: when any count is small, its reciprocal is large.
Worked Example: Separate Estimates for Two Small Categories
Consider hypothetical data on treatment assignment and a three-level covariate. The table gives the number treated and untreated at each level.
| Covariate level | Treated | Untreated | Total |
|---|---|---|---|
| A | 4 | 6 | 10 |
| B | 10 | 19 | 29 |
| C (reference) | 30 | 70 | 100 |
With C as the reference, the fitted coefficient for A is the difference between A’s and C’s log odds:
Its approximate standard error is
For B, the corresponding calculations are
Both coefficients are positive in this sample, but their uncertainty is substantial relative to their estimated size. A coefficient of 0.4418 corresponds to estimated treatment odds in A about 1.56 times those in C; it does not imply a precise difference. In particular, A has only four treated and six untreated people, so the estimate depends heavily on a small number of assignments. The calculations describe the fitted model, not a causal effect of the category.
Pooling: What Changes and When It Can Help
Pooling replaces two or more levels with a combined level and fits one treatment-assignment parameter for that group. It can reduce estimation noise when the categories are genuinely similar for the treatment-assignment model and the separate levels are sparsely represented. Pooling is not merely a numerical repair: it changes the model by constraining the pooled levels to share the same fitted propensity, conditional on the other modeled covariates.
A defensible pooling decision uses substantive knowledge—for example, levels that are adjacent bands of the same measure or categories that represent equivalent clinical states—and is specified without searching for a combination that produces a desired treatment contrast. A rare category is not automatically appropriate to combine with whichever other category makes its estimate look stable. If the categories differ in their relationship to treatment or in their relevance to confounding, pooling can obscure that difference.
Worked Example: Pooling Two Similar Sparse Levels
Suppose A and B in the hypothetical table above are two adjacent, substantively similar categories, and the analysis plan combines them for that reason. The pooled level has \(4+10=14\) treated and \(6+19=25\) untreated people. Compared with reference C, its fitted coefficient is
The approximate standard error is
An approximate 95% Wald interval for the pooled coefficient is
This interval is for the conditional log-odds coefficient, not for a causal treatment effect. Its corresponding odds-ratio interval is approximately \([\exp(-0.5142),\exp(1.0491)]=[0.598,2.855]\). The pooled point estimate lies between the separate estimates for A and B, and its standard error is smaller than either separate standard error here. That gain in precision comes with the pooling constraint: A and B now receive the same fitted treatment probability, \(14/39\approx0.3590\), whereas their separate fitted proportions were \(4/10=0.4000\) and \(10/29\approx0.3448\). Whether that constraint is reasonable depends on the covariate and design, not just on the smaller standard error.
Diagnosing Instability and Choosing a Representation
A sparse-category review should examine the joint treatment-by-category counts, not just the overall frequency of each covariate level. A level may be common overall but have very few treated people, or vice versa. Other warning signs include a large coefficient paired with a large standard error, estimates that change markedly when a few observations are removed, and fitted propensities that are highly sensitive to modest changes in model specification. These signals call for investigation; none alone determines the correct remedy.
Retaining separate indicators preserves the original distinctions and avoids imposing equality across levels, but may yield noisy estimates. Pooling can stabilize estimates when substantive similarity supports it, but can conceal within-pooled-group differences. Omitting the covariate avoids estimating its category coefficients, but also removes adjustment for that measured feature; it is not a harmless substitute for pooling. Whatever representation is chosen, check balance on the original covariate levels after matching or weighting. Balance on the pooled label does not establish balance across its component levels.
A useful diagnostic calculation is the fitted probability in a level when the model contains only that categorical covariate. For example, a level with two treated and one untreated person has fitted probability \(2/3\), but this estimate is based on only three observations. A moderate-looking probability can therefore still be imprecise. With additional covariates, predictions are model-based combinations of information across the data; sparse combinations of covariate levels may be especially weakly supported.
Worked Example: A Sparse Level With a Finite Estimate
In another hypothetical data set, a rare covariate level has two treated and one untreated person; the reference level has 18 treated and 79 untreated people. Both treatment groups occur in each level, so the saturated categorical model has finite log-odds estimates. The rare level’s fitted probability is \(2/(2+1)=0.6667\), while the reference level’s is \(18/(18+79)\approx0.1856\). The coefficient contrast is
Its approximate standard error is
The approximate 95% Wald interval is \(2.1722\pm1.96(1.2523)\), or about \([-0.2823,4.6267]\). Exponentiating gives an odds-ratio interval of approximately \([0.754,102.2]\). Although the estimated odds ratio is large, the interval is extremely wide: the data are compatible with a range from lower odds to odds far higher than the reference level. The small count, rather than the point estimate alone, is the key diagnostic. If a treatment-by-category cell were empty, the ordinary maximum-likelihood estimate could instead be nonfinite; that boundary case is a distinct logistic-fitting problem.
Practical Limits and Common Mistakes
Pooling is a modeling choice, not a guarantee of causal validity. Propensity-score adjustment still depends on appropriate measured confounders, adequate overlap, and the other causal assumptions required by the analysis. A pooled model may improve numerical stability while worsening adjustment if it merges levels with materially different treatment assignment or confounding relationships. Conversely, separate indicators do not guarantee successful adjustment when their estimates are weakly supported.
A common mistake is to pool categories solely because one coefficient is large or its standard error is inconvenient, then to claim the resulting score is better because it is more stable. Stability alone is not the criterion. The correction is to justify a pooled grouping using substantive meaning, compare the implied assignment model with the separate-level fit, and assess balance using the original covariate levels. There is no universal minimum count that makes a category safe to retain or automatically mandates pooling.
Inspect treated and untreated counts for every category, including rare levels and any combinations with other covariates that matter to the fitted model.
Review coefficient magnitudes, standard errors, fitted probabilities, and sensitivity to a small number of observations; interpret unstable estimates cautiously.
Use substantive similarity to motivate categories that share a parameter; do not let a desirable result alone determine the grouping.
After the chosen propensity-score procedure, assess balance at the original category level, including distinctions hidden by any pooling.
Check Your Understanding
Answer each question before reviewing the solutions.
- A category has 5 treated and 5 untreated people, while its reference category has 20 treated and 80 untreated people. Write the approximate standard error for the log-odds contrast.
- Why does pooling two categories change more than just the standard error?
- In a hypothetical level with 3 treated and 9 untreated people, what is the fitted treatment probability in a saturated model containing only that categorical covariate?
- Give one reason to retain separate indicators and one reason that could support pooling.
- After pooling levels A and B, what balance check should not be skipped?
Solutions. (1) The approximate standard error is \(\sqrt{1/5+1/5+1/20+1/80}=\sqrt{0.4625}\approx0.6801\). (2) Pooling constrains the levels to share a fitted treatment probability, so it changes the model’s representation of treatment assignment as well as the estimated uncertainty. (3) The fitted probability is \(3/(3+9)=0.2500\). (4) Retain separate indicators when the levels have meaningfully different treatment-assignment relationships; pooling may be reasonable when the levels are substantively similar and sparse. (5) Check balance across A and B separately, using their original levels, because balance on the pooled group alone can conceal imbalance within it.