What an Overlap Plot Shows
A propensity-score overlap plot places estimated treatment probabilities on the horizontal axis and displays their distributions separately for treated and untreated individuals. Its central question is descriptive: across which score values are observations from both treatment groups represented, and where are observations scarce or absent? This is useful because a plot can make regions of separation or thin representation visible before an analysis relies on comparisons in those regions.
Let \(A\) be a binary treatment indicator, with \(A=1\) for treated and \(A=0\) for untreated, and let \(X\) denote measured pretreatment covariates. The propensity score is the conditional treatment probability given those covariates. In observed data, a fitted model usually supplies an estimated score for each individual.
The plotted distributions are group-specific: they describe scores among treated individuals and scores among untreated individuals. A histogram may show counts, proportions, or density heights; a smooth curve may estimate a group-specific distribution. The vertical axis therefore matters. In a histogram of within-group proportions, each bin height is a proportion of that treatment group. In a density histogram, height depends on both the count and bin width, and the total area across bins is one. A density height is not itself the probability of one exact score.
A useful first reading is to locate the score ranges represented in each group, then look for shared regions, gaps, and bins or tails with little data. A region with observations in both groups offers some empirical representation for comparisons there. A region populated almost exclusively by one group signals that comparisons involving the other group may be weakly supported by the observed sample.
How the Plot Relates to Positivity
For a target population and covariate values that occur in that population, conditional positivity requires \(0<e(X)<1\): individuals with those covariates must have a nonzero probability of receiving either treatment. Structural positivity violations occur when treatment is impossible for one group at some covariate values. Practical violations arise when one group is so sparsely represented that estimation is unstable, even if its probability is not literally zero. These distinctions are useful when interpreting the distribution plot, but a plot of estimated scores cannot by itself establish which kind of problem produced a gap.
The plot is a diagnostic of the fitted scores in the observed sample, not a direct display of every covariate pattern. Scores are estimated, and the plot can change with the propensity-score model, the sample, and the plotting choices. A gap may reflect a genuine lack of representation, model behavior, or limited sample size. Conversely, smooth curves that overlap do not prove that the score model is adequate or that all relevant causal assumptions hold. In particular, overlap alone does not establish exchangeability, correct outcome measurement, or a causal effect.
Why Overlapping Distributions Need Not Match
The two group-specific score distributions need not have the same shape for there to be overlap. In fact, when the score is the true propensity score, Bayes' rule gives a particular relationship between the distributions. Let \(f_1(e)\) and \(f_0(e)\) denote the densities of the true score among treated and untreated individuals, respectively, and let \(P(A=1)\) be the treatment prevalence. For score values where the densities are defined and the relevant probabilities are positive:
To see why, apply Bayes' rule to the score value \(e(X)=e\). The probability of treatment conditional on that score is \(e\), so \(f_1(e)=e f(e)/P(A=1)\), where \(f(e)\) is the marginal score density. Similarly, \(f_0(e)=(1-e)f(e)/P(A=0)\). Dividing the two expressions yields the displayed ratio. This identity describes the true score, not necessarily an estimated score from a misspecified model.
Thus, the groups can have visibly different score distributions even when both have observations across a substantial shared range. A plot is not expected to show identical curves. The practical reading is about representation and scarcity, not about whether the two distributions match perfectly.
Worked Example: Finding a Shared Region and Separated Bins
Suppose a hypothetical study has 80 treated and 90 untreated individuals. The table gives the number of estimated scores in five bins. Calculate the within-group percentage in each bin and identify the pattern.
| Estimated-score bin | Treated count | Untreated count |
|---|---|---|
| [0, 0.2) | 0 | 25 |
| [0.2, 0.4) | 5 | 30 |
| [0.4, 0.6) | 20 | 25 |
| [0.6, 0.8) | 35 | 10 |
| [0.8, 1] | 20 | 0 |
For a treated bin, divide its count by 80; for an untreated bin, divide its count by 90. For example, the treated proportion in \([0.6,0.8)\) is \(35/80=0.4375\), or 43.75%, while the untreated proportion there is \(10/90=0.1111\), or 11.11%. Applying the same calculation to every bin gives:
| Estimated-score bin | Treated within-group proportion | Untreated within-group proportion |
|---|---|---|
| [0, 0.2) | 0/80 = 0% | 25/90 = 27.78% |
| [0.2, 0.4) | 5/80 = 6.25% | 30/90 = 33.33% |
| [0.4, 0.6) | 20/80 = 25.00% | 25/90 = 27.78% |
| [0.6, 0.8) | 35/80 = 43.75% | 10/90 = 11.11% |
| [0.8, 1] | 20/80 = 25.00% | 0/90 = 0% |
Both groups are represented in the three middle bins, but not in the two extreme bins in this sample. The bins at the extremes suggest empirical separation; the middle bins show a shared score region, though the group proportions differ substantially within some bins. These are binned observations, so the table does not establish that every score inside a bin is represented, nor that the population has a structural positivity violation. The sample sizes and covariate patterns would need further consideration.
Thin Tails and Sparse Information
A plot can show both treatment groups across most of the score range and still reveal a thin tail: a part of the range with very few observations from one or both groups. Distinguish the shape of a normalized curve from the number of individuals that support it. A tail can occupy a visible area in a density plot while containing only a small number of people, particularly if the sample is large in the center. Sparse observations make estimates in that region more sensitive to individual cases and modeling choices.
For weighting analyses, score values near 0 or 1 can also correspond to large inverse-probability weights for some individuals. The earlier material on inverse probability weighting explains how such weights enter an estimator. Here the important diagnostic point is that a thin tail can flag where only a few observations may carry substantial influence. A plot does not calculate that influence, and the visual appearance alone does not determine whether any particular analysis is acceptable.
Worked Example: Reading a Thin Upper Tail
In a hypothetical sample, there are 200 treated and 200 untreated individuals. Counts in three broad score regions are as follows. The final region is an upper tail with scores above 0.9.
| Score region | Treated count | Untreated count |
|---|---|---|
| Below 0.1 | 2 | 30 |
| 0.1 to 0.9 | 193 | 168 |
| Above 0.9 | 5 | 2 |
The treated share above 0.9 is \(5/200=0.025\), or 2.5%; the untreated share is \(2/200=0.010\), or 1.0%. Together, only \(5+2=7\) of the 400 individuals lie in that region. The plot therefore indicates that the upper tail has little empirical representation, especially among untreated individuals. It does not show that the true probability of untreated treatment assignment is zero above 0.9: two untreated observations are present, and sample evidence cannot establish a population probability from a zero or small count alone.
The corresponding lower region is also asymmetric: \(2/200=1.0\%\) of treated individuals versus \(30/200=15.0\%\) of untreated individuals. A careful reading therefore reports both the region and the group in which it is thin, rather than describing the entire plot simply as “good” or “bad” overlap. The score values are probabilities and have no physical units.
Group Size, Binning, and Plot Choices
Raw counts and within-group proportions answer different questions. Counts show how many observations are available, while within-group proportions show how each group's score distribution is shaped without allowing the larger group to dominate solely because it has more members. A normalized density also permits shape comparison, but it can make a small group look as substantial as a large group. Read the group sample sizes and the vertical-axis label alongside the curves.
For a histogram with bin width \(w\), the density height for a bin containing \(c\) observations from a group of size \(n\) is \(c/(nw)\). For example, with 12 observations in a bin of width 0.2 among 120 individuals, the height is \(12/(120\times0.2)=0.5\). The bin's area is \(0.5\times0.2=0.10\), which corresponds to 10% of that group. This is why unequal-width bins should not be compared by counts or heights without checking how the plot was scaled.
Bin width and smoothing affect what becomes visible. Wide bins can conceal a narrow gap or combine a thin tail with a more populated region. Very narrow bins can look irregular because of ordinary sampling variation. Smooth curves can clarify broad patterns, but their appearance depends on smoothing choices and may blur a sharp boundary. If a conclusion depends on a narrow feature, inspect the underlying counts or try a reasonable alternative display rather than treating one visual rendering as definitive.
Worked Example: Why Normalization and Counts Both Matter
Consider a hypothetical sample of 100 treated and 900 untreated individuals. In low, middle, and high score regions, the counts are 20, 60, and 20 among treated individuals, and 450, 405, and 45 among untreated individuals.
The treated percentages are \(20/100=20\%\), \(60/100=60\%\), and \(20/100=20\%\). The untreated percentages are \(450/900=50\%\), \(405/900=45\%\), and \(45/900=5\%\). Comparing counts alone might suggest that the untreated group dominates every region, because its total sample is nine times larger. Comparing within-group proportions instead shows that the high-score region contains 20% of treated individuals but only 5% of untreated individuals.
Yet the count in that high region is 20 treated and 45 untreated, not 20 and 5. The percentage comparison describes relative group distributions; the counts describe how many observations contribute. Both matter: an overlaid density plot can make shapes comparable, while group sample sizes and bin counts help reveal how much data supports those shapes. Neither display alone gives a complete assessment of the precision or validity of a causal estimate.
What the Plot Can and Cannot Establish
An overlap plot is most useful as an initial visual diagnostic, interpreted together with the model that generated the scores, sample sizes, and the scientific target population. It can reveal obvious group separation, regions of shared empirical representation, and sparse tails. It cannot verify the propensity-score model, prove conditional positivity throughout the target population, or establish that treatment groups are comparable on all relevant covariates.
A score plot is also different from a covariate-balance assessment. Overlapping score distributions do not guarantee that the covariates are balanced after a chosen analysis, just as a balance summary does not show how many observations occupy every score region. The appropriate conclusion from a plot is therefore limited: it describes the observed distribution of fitted scores by group and identifies areas that deserve closer scrutiny.
Common reading errors include:
- Calling any difference in curve shape a positivity failure. The two distributions need not be identical; examine whether both groups are represented in the relevant score regions.
- Treating a zero sample count as proof of structural impossibility. A finite sample can omit a group even when its population probability is positive.
- Reading normalized curves as raw sample sizes. Density or within-group proportion plots compare distributions, not absolute numbers.
- Ignoring axes, bin widths, or smoothing. These choices can alter the apparent size and location of gaps or tails.
- Concluding that overlap proves causal validity. The plot concerns estimated scores; it does not establish the other assumptions needed for a causal interpretation.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A treated group has 40 observations and an untreated group has 400. A density plot gives both curves the same total area. Does that mean the groups have the same sample size?
- In a score bin, there are 6 treated individuals out of 120 treated and 18 untreated individuals out of 360 untreated. Calculate each within-group proportion and interpret them.
- A plot shows no treated observations in a low-score bin. What is justified, and what is not justified, by this observation?
- Why might a density plot suggest substantial overlap while counts reveal that only a few individuals lie in one tail?
- Does a visually overlapping pair of score distributions establish that all causal assumptions hold? Give one reason for your answer.
Solutions. (1) No. Equal total area is a normalization convention for comparing distribution shapes; the group sizes are 40 and 400. (2) The treated proportion is \(6/120=0.05\), or 5%; the untreated proportion is \(18/360=0.05\), also 5%. The bin contains the same fraction of each group, but the counts and group totals remain distinct. (3) It is justified to say that the observed sample has no treated individuals in that bin. It is not justified to conclude from that fact alone that treatment is structurally impossible there in the population. (4) A density plot is normalized by group size, so a sparse tail can have a visible plotted height or area despite containing few individuals; report counts or sample sizes as well. (5) No. The plot describes estimated scores, not whether all relevant confounding is controlled or whether other causal assumptions hold.