Positivity Checks Within Important Subgroups

Why Check Positivity Within Subgroups?

An overall diagnostic can look reassuring even when treatment is unavailable for some patients in an important clinical group. For example, a pooled propensity-score plot may show treated and untreated patients at similar scores, but the overlap may come entirely from one site or demographic group. If a second prespecified group has no patients in one treatment arm, its treatment effect cannot be learned from direct comparisons within that group without relying on additional assumptions or extrapolation.

The relevant question is therefore not only whether treatment and control patients overlap overall, but also whether both treatment options occur for the covariate profiles represented in each subgroup of interest. A subgroup might be defined by clinical status, age category, demographic group, or site. These groups should be chosen for substantive reasons before interpreting the diagnostic; searching among many groups after seeing the results can produce misleading conclusions.

Definition: For a binary treatment $A$, covariates $X$, and subgroup label $G$, conditional positivity within subgroup $g$ requires $$ 0 < P(A=1\mid X=x,G=g) < 1 $$ for covariate values $x$ that occur in the target population within that subgroup. Equivalently, both treatment levels must be possible for the relevant covariate profiles in each subgroup. A propensity-score check assesses this condition using $e(x,g)=P(A=1\mid X=x,G=g)$, or another score known to be valid within the subgroup.

Positivity is a population condition; a finite sample can only provide evidence about it. Seeing at least one treated and one untreated person in a subgroup does not establish positivity for every relevant covariate profile. Conversely, an empty treatment arm in a subgroup is a strong warning that subgroup-specific comparisons are unsupported in the observed data. This check addresses positivity, not the other assumptions needed for causal identification, such as consistency and conditional exchangeability.

Subgroup Rates and Conditional Support

A useful first step is to tabulate treatment counts within each important subgroup. The treatment rate $P(A=1\mid G=g)$ is a marginal rate: it averages over the covariates $X$ in subgroup $g$. It can reveal that a treatment arm is rare or absent, but it does not establish conditional positivity. A subgroup might have a moderate overall treatment rate while some covariate profiles within it have treatment probability near zero or one.

For that reason, inspect treatment availability across relevant covariate patterns as well as subgroup totals. Propensity-score distributions can help summarize this information, provided the score is appropriate for the subgroup. Plot or summarize the scores separately by treatment arm within each subgroup. Look for empty arms, extreme scores, and regions in which one arm has little or no representation. Do not treat a single summary, such as the subgroup mean score, as evidence of overlap: means can be similar even when the underlying distributions barely intersect.

Worked Example: Similar Subgroup Rates, Different Conditional Support

Suppose a hypothetical study records treatment $A$, a two-level clinical risk variable $X$, and a prespecified subgroup $G$. The table gives observed counts. The risk variable is defined in the same way in each subgroup.

Subgroup	Risk level	Treated	Untreated	Observed treatment rate
Stable	Low	18	42	18/60 = 0.300
Stable	High	12	8	12/20 = 0.600
Acute	Low	2	18	2/20 = 0.100
Acute	High	18	2	18/20 = 0.900

The stable subgroup has $30$ treated patients out of $80$, so its marginal treatment rate is $30/80=0.375$. The acute subgroup has $20$ treated patients out of $40$, giving $20/40=0.500$. Looking only at those totals might suggest that neither subgroup has a major treatment-availability problem.

The risk-specific rates tell a more nuanced story. In the acute subgroup, the observed rate is $2/20=0.100$ for low-risk patients and $18/20=0.900$ for high-risk patients. Both treatment arms occur in both cells, but the rates are close to the boundaries, and each cell has only two patients in the less common arm. These counts raise a practical positivity concern: comparisons for those profiles may be imprecise and may depend heavily on a few observations. The stable subgroup also varies by risk level, but its observed rates, $0.300$ and $0.600$, are less extreme.

These are sample rates, not known population probabilities. The table therefore flags where closer investigation is needed; it does not prove that the population probabilities equal the displayed fractions. In particular, the presence of both treatment arms in each observed cell is not proof that all relevant covariate profiles have adequate support.

How Pooled Scores Can Mislead

A pooled propensity score is $P(A=1\mid X=x)$. The subgroup-specific treatment probability is $P(A=1\mid X=x,G=g)$. They are not automatically the same. By the law of total probability,

P(A=1\mid X=x) = \sum_g P(A=1\mid X=x,G=g)P(G=g\mid X=x).

Thus, at a fixed value of $X$, the pooled probability is an average of subgroup-specific probabilities, weighted by subgroup prevalence at that value of $X$. That average can differ from the probability in any particular subgroup. A pooled score is valid within a subgroup when it gives the correct treatment probability there. For example, this is true if $G$ is included in or determined by $X$, or if treatment probability does not vary by $G$ conditional on $X$. Those are sufficient examples, not an exhaustive list of possible reasons the probabilities might agree.

The condition $A$ is conditionally independent of $G$ given $X$ is one way to express that treatment probability does not vary by subgroup after conditioning on $X$. But subgroup membership need not be part of $X$ for that condition to hold. Conversely, merely fitting a score that omits $G$ does not establish that the pooled probability is valid within each subgroup. If subgroup treatment probabilities differ at the same $X$, the pooled score can obscure the difference.

Worked Example: When a Pooled Probability Is Valid Within Subgroups

Consider a hypothetical study in which $X$ has values 0 and 1 and $G$ has two categories. The groups are distinct: knowing $X$ does not determine $G$. At $X=0$, the observed counts are 16 treated and 64 untreated in $G=0$, and 4 treated and 16 untreated in $G=1$. At $X=1$, the counts are 64 treated and 16 untreated in $G=0$, and 16 treated and 4 untreated in $G=1$.

At $X=0$, the subgroup rates are $$ P(A=1\mid X=0,G=0)\ \text{estimated by}\ 16/80=0.200, \qquad P(A=1\mid X=0,G=1)\ \text{estimated by}\ 4/20=0.200. $$ The pooled rate at $X=0$ is $20/100=0.200$. At $X=1$, both subgroup rates are $0.800$, and the pooled rate is $80/100=0.800$. In this example, the observed rates agree across subgroups at each value of $X$. A pooled score can represent the subgroup probabilities if this equality holds in the population and the score model estimates those probabilities adequately. The fact that $G$ is neither included in nor determined by $X$ does not itself invalidate the pooled score.

Now change only the $X=0,G=1$ treated count from 4 to 12, leaving its untreated count at 8. Its subgroup rate becomes $12/20=0.600$. The pooled $X=0$ rate is now $(16+12)/(80+20)=28/100=0.280$. That pooled value is neither subgroup rate: it understates the rate in $G=1$, which is $0.600$, and overstates the rate in $G=0$, which is $0.200$. In this setting, use a treatment-probability model that conditions on $G$, or otherwise establish that the score used is valid within the subgroup. A pooled score alone would not accurately describe treatment probability for both groups at $X=0$.

Aggregate Overlap Can Hide a Subgroup Failure

A pooled overlap plot combines patients across subgroups. It can look adequate because one subgroup supplies treated patients and another supplies untreated patients, even if a particular subgroup has only one treatment arm. The following example illustrates why a pooled picture should be checked against subgroup-specific support.

Worked Example: Pooled Overlap Despite No Treated-Control Comparison in a Subgroup

Suppose a hypothetical sample has 40 patients in a renal-impairment subgroup, all treated. Their true treatment probability for the covariate profile represented there is $e(X,G)=1$. A second subgroup has 100 patients, with 50 treated and 50 untreated; for its represented profile, $e(X,G)=0.5$.

The treated patients have scores of 1 for the 40 renal-impairment patients and 0.5 for the 50 treated patients in the second subgroup. The untreated patients all have scores of 0.5. Thus, at score 0.5, there are 50 treated and 50 untreated patients. A pooled plot therefore displays a shared score region, even though the renal-impairment subgroup has no untreated patients.

Within the renal-impairment subgroup, $P(A=0\mid X,G)=0$ for the represented profile, so the positivity requirement fails there. The pooled overlap is supplied by the second subgroup and cannot create a treated-control comparison within the renal-impairment subgroup. A subgroup-specific plot or count table exposes the failure that the aggregate display conceals.

Diagnostics, Alternatives, and Limitations

A practical subgroup assessment combines several views rather than relying on a single threshold or plot. For each prespecified subgroup, record the number treated and untreated; examine treatment frequencies across key covariate patterns; and compare score distributions separately by treatment status. Where a fitted score is used, verify that its conditioning set and specification are suitable for the subgroup comparisons. If treatment probability may vary by subgroup at the same $X$, examine a score that allows for that variation rather than assuming a pooled score is valid.

Several warnings merit attention:

An empty treatment arm: no treated or no untreated observations in a subgroup means the observed data provide no within-subgroup contrast for that arm.
Extreme subgroup-specific scores: values near zero or one indicate that one treatment is rarely observed for some profiles. This can make estimates unstable even when both arms appear somewhere in the subgroup.
Misleading summaries: similar means, ranges, or marginal treatment rates do not establish overlap across the full score distributions or across covariate profiles.
Overreliance on sample counts: small counts can reflect sampling variability, while a cell containing both arms does not rule out unsupported profiles elsewhere.
Overreliance on a fitted score: model misspecification can make estimated probabilities look moderate even when actual treatment assignment is highly separated.

Subgroup-specific score plots and tables complement, but do not replace, covariate-support diagnostics. A plot or summary of estimated scores alone does not establish adequate finite-sample covariate balance: the balancing property applies to the true propensity score conditional on its value, while estimated scores may be misspecified or leave residual imbalance. Likewise, a pooled plot is useful for describing overall score distributions, but it does not answer whether support is adequate in a particular subgroup. These checks diagnose potential problems; they do not by themselves determine how to change the target population or what restriction rule to use.

Empirical subgroup diagnostics also have limitations. Subgroups with few patients produce unstable estimated rates and score distributions. Many subgroup cuts can create a large set of noisy comparisons, so prioritize clinically or scientifically justified groups and report how they were chosen. Finally, estimated support is itself uncertain: the observed score distribution is not the population distribution, and a probability estimated near zero or one may be imprecise. Treat subgroup warnings as information about the strength and limits of the available comparisons, not as a mechanical proof that a causal estimate is valid or invalid.

A Short Justification for the Subgroup Check

The reason for conditioning the positivity check on $G$ follows directly from the treatment probability. For a given $x$, the pooled probability averages over subgroup membership. If, for example, one subgroup has treatment probability zero and another has probability one, the pooled average can be strictly between zero and one whenever both subgroups occur. That pooled value does not make either subgroup’s treatment probability interior.

P(A=1\mid X=x,G=g_1)=0,\qquad P(A=1\mid X=x,G=g_2)=1 $$ $$ \Longrightarrow\quad P(A=1\mid X=x)=\sum_g P(A=1\mid X=x,G=g)P(G=g\mid X=x),

If both groups occur at $x$, then $0<P(G=g_2\mid X=x)<1$, so the pooled probability is interior even though positivity fails in both subgroups. The aggregate probability answers a different question: the average treatment probability across groups at that $x$. It cannot substitute for the subgroup-specific condition when the target or interpretation concerns a particular group.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A subgroup has an observed treatment rate of 0.50. Does this establish conditional positivity within that subgroup? Why or why not?
At a fixed $X=x$, treatment probabilities are 0.10 in one subgroup and 0.70 in another. Can their pooled probability be used as the treatment probability for both subgroups?
A pooled score plot shows treated and untreated patients at score 0.4. What additional check is essential if subgroup effects are of interest?
At one covariate profile in a subgroup, all 25 observed patients are treated. What does this show, and what does it not show?
Suppose $G$ is not included in $X$, but treatment probability is the same across values of $G$ conditional on $X$. Is a pooled score necessarily invalid within the subgroups?

Solutions. (1) No. The rate is marginal over $X$; some covariate profiles may still have treatment probability zero or one, or nearly so. (2) No. The pooled probability is a weighted average of the subgroup probabilities and generally differs from each one. A score used within a subgroup must be valid for that subgroup. (3) Examine treated and untreated score support separately within each prespecified subgroup, and check treatment availability across relevant covariate profiles. (4) It shows that the observed profile has no untreated patients and is a warning of unsupported comparison in the sample. It does not establish the exact population treatment probability or describe other profiles. (5) No. If treatment probability truly does not vary by $G$ conditional on $X$, the pooled probability can be valid within each subgroup even though $G$ is neither part of nor determined by $X$.