Quantifying Overlap With Density Measures

Why Quantify Density Overlap?

A propensity-score plot can show whether treated and control units tend to occupy similar parts of the score scale. A numerical summary can make that comparison easier to report and compare, but it should preserve what the plot means: the extent to which the two score distributions place density in the same regions. The overlap coefficient does this by integrating the lower of the two densities at every score value.

This is a descriptive measure of the distributions of scores among treated and control units. It does not estimate a treatment effect, and a single number cannot show where the distributions differ. Interpretation therefore starts with the density curves and uses the coefficient as a compact summary alongside them.

Definition: Let $S$ be a propensity score on $[0,1]$, and let $f_1(s)$ and $f_0(s)$ be the probability density functions of $S$ among treated and control units, respectively. The overlap coefficient is $$ \operatorname{OVL}(f_1,f_0)=\int_0^1 \min\{f_1(s),f_0(s)\}\,ds. $$ Each density is normalized to integrate to 1 within its treatment group. The overlap coefficient is between 0 and 1.

At a score value where one group has more density than the other, the smaller density is the amount that can be matched by density from the other group at that location. Integrating these pointwise minima gives the total shared density mass. A coefficient of 1 means the two density functions are equal almost everywhere; a coefficient of 0 means they have no region of positive-width shared density. Values between these extremes summarize partial overlap.

A Useful Identity and Its Interpretation

The coefficient can also be expressed using the total variation distance between the two densities. This identity explains why the overlap coefficient is a similarity measure.

\min\{f_1(s),f_0(s)\} =\frac{f_1(s)+f_0(s)-|f_1(s)-f_0(s)|}{2}.

Integrate both sides over the score scale. Since each density integrates to 1, the sum of their integrals is 2. Therefore,

\operatorname{OVL}(f_1,f_0) =1-\frac{1}{2}\int_0^1 |f_1(s)-f_0(s)|\,ds.

The quantity $\frac{1}{2}\int |f_1-f_0|$ is the total variation distance between the distributions. Thus, greater differences between the density curves correspond to a smaller overlap coefficient. Also, because the pointwise minimum is nonnegative and no larger than either density, its integral cannot exceed 1. These facts establish the stated range.

There is a useful interpretation from classification, with a narrow scope. If a person is selected with equal prior probability from the treated or control score distributions, and the score is used to classify which group the person came from, the smallest possible classification error is $\operatorname{OVL}/2$. The error is the integral of the smaller of the two equally weighted densities, $ \frac{1}{2}\min(f_1,f_0)$. This is a statement about distinguishing the two score distributions; it is not a measure of causal bias or treatment-effect identification.

Worked Example: Overlap Between Two Uniform Score Distributions

Suppose a hypothetical treated group has scores uniformly distributed from 0.20 to 0.80, while a hypothetical control group has scores uniformly distributed from 0.40 to 1.00. Find the overlap coefficient.

A uniform density over an interval of width 0.60 has height $1/0.60=1.6667$. Thus $f_1(s)=1.6667$ on $[0.20,0.80]$, and $f_0(s)=1.6667$ on $[0.40,1.00]$; each is zero outside its stated interval. The shared interval is $[0.40,0.80]$, of width 0.40. Within it, the two density heights are equal, so their minimum is 1.6667.

The integral is the rectangle's height multiplied by its width:

\operatorname{OVL} =\int_{0.40}^{0.80}1.6667\,ds =1.6667(0.40) =0.6667.

The score distributions have an overlap coefficient of about 0.667. This does not mean that 66.7% of all individuals have a particular treatment status or that 66.7% of a causal effect is identified. It means that the two normalized score densities share 0.667 units of density mass under the pointwise-minimum definition.

Calculating Overlap From a Density Plot

For estimated densities, the integral is usually calculated numerically. If the plot uses piecewise-constant densities over bins, multiply the smaller density height in each bin by that bin's width, then add the bin areas. If data are reported as proportions within bins, the equivalent calculation is to add the smaller of the two group proportions in each bin when the bins have the same boundaries for both groups.

The calculation must use density heights or appropriately scaled bin masses, not simply count the number of bins in which both groups appear. A narrow bin and a wide bin can have the same height but contribute different areas. Likewise, if a histogram displays counts rather than normalized densities, the two groups' counts cannot be compared as density heights without accounting for the different group sizes.

Worked Example: A Binned Density Approximation

Consider hypothetical normalized histograms using four equal-width bins: $[0,0.2)$, $[0.2,0.4)$, $[0.4,0.6)$, and $[0.6,0.8]$. Suppose the treated-group proportions in these bins are 0.10, 0.40, 0.40, and 0.10. The corresponding control-group proportions are 0.40, 0.40, 0.15, and 0.05. Each set sums to 1.

Because each bin has width 0.20, a bin's density height is its group proportion divided by 0.20. For example, in the first bin the treated and control heights are $0.10/0.20=0.50$ and $0.40/0.20=2.00$. The smaller height contributes $0.50(0.20)=0.10$ to the integral, equal to the smaller bin proportion.

The same calculation applies to each bin: the smaller proportions are 0.10, 0.40, 0.15, and 0.05. Therefore,

\widehat{\operatorname{OVL}} =0.10+0.40+0.15+0.05 =0.70.

This histogram-based estimate is 0.70. The bin contributions also reveal where the shared mass occurs. The second bin contributes 0.40, while the first and last contribute only 0.10 and 0.05. A plot makes that location-specific information visible; the total alone does not.

With smooth estimated density curves, numerical integration approximates the area under the pointwise minimum curve. The result can depend on the density-estimation method and its smoothing choices, especially in a small sample. A histogram approximation also depends on bin boundaries and widths. These are reasons to report the method used and examine the plot rather than treating the calculated value as an exact population property.

Connection to the True Propensity Score

For a true propensity score, the treated and control score densities have a useful form. Let $S=e(X)=P(A=1\mid X)$, let $f(s)$ be the marginal density of $S$, and let $\pi=P(A=1)$ be the treated proportion in the population. Because the true propensity score satisfies $P(A=1\mid S=s)=s$, Bayes' rule gives

f_1(s)=f(s\mid A=1)=\frac{s f(s)}{\pi}, \qquad f_0(s)=f(s\mid A=0)=\frac{(1-s) f(s)}{1-\pi}.

For example, the first equality follows by writing the conditional density as $f(s\mid A=1)=P(A=1\mid S=s)f(s)/P(A=1)$ and substituting $P(A=1\mid S=s)=s$. The control expression follows in the same way using $P(A=0\mid S=s)=1-s$. These identities concern the true score and population distributions. Densities of fitted scores in a finite sample are estimates, so the identities need not hold exactly for them.

Worked Example: Exact Overlap for a Uniform Marginal Score

Suppose, hypothetically, that the true propensity score $S$ is uniform on $[0,1]$ in a population. Its marginal density is $f(s)=1$, and its mean, which equals the treated prevalence $\pi=E[S]$, is 0.50. Use the density identities to find the overlap coefficient.

The treated and control score densities are $f_1(s)=s/0.50=2s$ and $f_0(s)=(1-s)/0.50=2(1-s)$. They are equal when $2s=2(1-s)$, so the curves cross at $s=0.50$. Below that point, $2s$ is the smaller density; above it, $2(1-s)$ is the smaller density.

\begin{aligned} \operatorname{OVL} &=\int_0^{0.50}2s\,ds+\int_{0.50}^1 2(1-s)\,ds\\ &=[s^2]_0^{0.50}+[2s-s^2]_{0.50}^1\\ &=0.25+0.25\\ &=0.50. \end{aligned}

The exact coefficient is 0.50. Both score densities are positive throughout the interior of $[0,1]$, yet they are not equal: treated scores are more concentrated toward higher values, and control scores toward lower values. This illustrates why support overlap and the amount of density overlap are different descriptions.

Reading the Number Alongside the Plot

A sound interpretation combines the coefficient with the shapes and locations of the density curves. Two plots can have the same coefficient while differing substantially: one may show moderate differences across most of the score range, while another may show close agreement in the center and a pronounced difference over a smaller region. The integral compresses that structure into one number and therefore cannot identify where a concern arises.

The overlap coefficient also concerns only the one-dimensional score distributions. Similar score distributions do not prove that treated and control units are comparable on every covariate: distinct covariate patterns can map to the same propensity score. Conversely, different score densities do not by themselves identify the size or direction of bias in a treatment-effect estimate. The measure is a diagnostic for score-distribution similarity, not a replacement for the causal assumptions or other design checks needed for an analysis.

In practice, density estimates introduce additional uncertainty and choices. Small group sizes can make curves unstable, and smoothing can blur narrow features or create apparent differences. Useful checks include confirming that each density is normalized, checking the score axis and bin boundaries, and comparing the computed area with the plotted curves. A reported coefficient should be identified as applying to the observed or estimated score distributions and to the particular density-estimation procedure used.

Common Mistake and Alternatives

Common mistake: “The treated and control densities both appear somewhere across the same score range, so overlap must be high.” Shared range alone does not measure shared density mass: the groups could place most of their density in very different parts of that range. Calculate the area under the pointwise minimum, and inspect where that area comes from. Also, do not treat a high score-density overlap coefficient as proof of covariate-level positivity or causal identification.

The coefficient is most useful as a compact companion to a plot. A plot preserves the location and shape of differences, while the coefficient summarizes their total shared mass. A support comparison instead asks whether the groups' score distributions occupy common regions at all; it does not convey how much density is concentrated there. Covariate-support assessment asks a different question again, because the score is a summary of covariates rather than the covariates themselves. These approaches complement one another rather than competing as interchangeable measures.

The coefficient is limited by its compression of the distributions to one number, by estimation variability, and by the fact that score overlap is not equivalent to covariate overlap. It is not a threshold that automatically determines whether a design is acceptable. Its role is to describe and communicate a feature of the score distributions, with the plot and the study context supplying the detail needed for judgment.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

If two normalized score densities are identical, what is their overlap coefficient? Explain using the definition.
Two equal-width bins have treated proportions 0.30 and 0.20, and control proportions 0.10 and 0.40. What is their total contribution to a histogram overlap estimate?
Why does a high overlap coefficient not establish covariate-level positivity?
For the identity $\operatorname{OVL}=1-\frac{1}{2}\int |f_1-f_0|$, what happens to the coefficient as the total variation distance increases?

Solutions. (1) The densities are equal at every score value, so their pointwise minimum is either density. Its integral is 1, giving an overlap coefficient of 1. (2) In each equal-width bin, the contribution is the smaller group proportion: $\min(0.30,0.10)+\min(0.20,0.40)=0.10+0.20=0.30$. (3) The coefficient summarizes only the distributions of the one-dimensional propensity score. Different covariate patterns can have similar scores, so score-density similarity does not establish comparable covariate support. (4) The overlap coefficient decreases as total variation distance increases, because it equals 1 minus that distance.