Estimating Mass-Constrained Density Regions

From Probability Content to Region Size

A density level set is usually specified by a numerical threshold: retain points where the density exceeds that level. A different question starts with a probability target: what is the smallest region that contains a specified fraction of the distribution? This formulation is useful when a region is meant to summarize where observations are concentrated, rather than to describe the shape of the density at an arbitrarily chosen height.

The answer is generally a density superlevel set, but a threshold alone does not always specify the whole region. If the density is constant on a set of positive volume, an optimal region may need to include only part of that set. This distinction matters both for the exact optimization problem and for interpreting regions estimated from data.

Definition: Let $X$ have density $f$ on $\mathbb{R}^d$, and let $p\in(0,1)$ be a target probability mass. Write $\operatorname{Vol}(A)$ for the Lebesgue measure of a measurable set $A$. A minimum-volume region of mass $p$ is a measurable set $A^*$ such that $P(X\in A^*)\ge p$ and $$ \operatorname{Vol}(A^*)=\inf\{\operatorname{Vol}(A): A\text{ measurable and }P(X\in A)\ge p\}. $$

The constraint uses at least $p$, so a candidate is allowed to contain more than the target mass. Under the conditions below, however, an optimal region contains exactly $p$. Volume here means ordinary $d$-dimensional volume, not probability: a region can have small volume because it lies where the density is high.

The Density-Threshold Characterization

Choose a density threshold $t$ so that the mass strictly above it is no more than $p$, while the mass at or above it is at least $p$. Such a threshold exists for $p\in(0,1)$. Indeed, $Y=f(X)$ is finite and positive with probability one; taking a quantile of $Y$ gives a $t>0$ satisfying the inequalities below. Positivity holds because $P(f(X)=0)=\int_{\{f=0\}}f(x)\,dx=0$. At a threshold with a tie, select just enough of the level set $\{x:f(x)=t\}$ to reach mass $p$.

Theorem: Let $f$ be a probability density on $\mathbb{R}^d$, and let $p\in(0,1)$. Choose $t>0$ such that $$ P(f(X)>t)\le p\le P(f(X)\ge t). $$ There is a measurable subset $B\subseteq\{x:f(x)=t\}$ for which $$ A^*=\{x:f(x)>t\}\cup B,\qquad P(X\in A^*)=p. $$ This set has minimum volume among all measurable regions with probability at least $p$. Any minimum-volume region must include the part where $f>t$, exclude the part where $f<t$, up to sets of volume zero, and contain enough of the level set where $f=t$ to have mass $p$.

To see that the subset $B$ can be chosen, note that $P(f(X)\ge t)-P(f(X)>t)=t\operatorname{Vol}\{f=t\}$. Thus the required volume for $B$ is $[p-P(f(X)>t)]/t$, which lies between zero and $\operatorname{Vol}\{f=t\}$. Since Lebesgue measure is nonatomic, a measurable subset of the level set with this volume exists.

Here is the optimization argument. It also shows why the density threshold, rather than the geometric shape of a proposed region, determines the solution.

Compare an arbitrary candidate with the threshold region.
Let $C$ be any measurable set with finite volume and $P(X\in C)\ge p$. Write $1_C$ and $1_{A^*}$ for their indicator functions.

Use the sign of the density relative to the threshold.
On $\{f>t\}$, $A^*$ includes every point; on $\{f<t\}$, it excludes every point. Consequently, $(1_C-1_{A^*})(f-t)\le 0$ almost everywhere.

Integrate and rearrange.
Integrating the pointwise inequality gives $P(X\in C)-p\le t\{\operatorname{Vol}(C)-\operatorname{Vol}(A^*)\}$. Since the left side is nonnegative and $t>0$, $C$ cannot have smaller volume.

For completeness, the key identity in the last step is

\begin{aligned} P(X\in C)-p &=\int (1_C-1_{A^*})f(x)\,dx\\ &=t\{\operatorname{Vol}(C)-\operatorname{Vol}(A^*)\} +\int(1_C-1_{A^*})(f(x)-t)\,dx\\ &\le t\{\operatorname{Vol}(C)-\operatorname{Vol}(A^*)\}. \end{aligned}

The final integral is nonpositive by the sign argument, and $P(X\in A^*)=p$. If $C$ had smaller volume, the right side would be negative, contradicting $P(X\in C)-p\ge0$. Moreover, if $C$ is also optimal, the inequality forces its mass to equal $p$ and the integral to be zero. A disagreement with $A^*$ at a point where $f\ne t$ makes the integrand strictly negative, so an optimal set cannot make such a disagreement on a set of positive volume. This proves the characterization. $\square$

If $\operatorname{Vol}\{f=t\}=0$, there is no positive-volume tie to divide: the optimal region is $\{f>t\}$, up to volume-zero changes. If the level set has positive volume, different choices of $B$ can give different but equally optimal regions. The probability-mass condition determines how much of the tied level to include, not which particular points to choose from it.

Worked Examples

Worked Example: A Central Region for a Standard Normal Distribution

Let $X\sim N(0,1)$, and seek a minimum-volume region with mass $p=0.90$. The standard normal density is $f(x)=(2\pi)^{-1/2}\exp(-x^2/2)$, which decreases as $|x|$ increases. The region between the 0.05 and 0.95 quantiles has endpoints approximately $-1.6449$ and $1.6449$, and its probability is $0.95-0.05=0.90$.

The density at either endpoint is $t=(2\pi)^{-1/2}\exp(-1.6449^2/2)$. Since $1.6449^2/2\approx1.3528$, this gives $t\approx0.3989\exp(-1.3528)\approx0.1031$. The superlevel set $\{x:f(x)\ge t\}$ is exactly $[-1.6449,1.6449]$, with volume (length) $1.6449-(-1.6449)=3.2898$. The theorem establishes that no other measurable set of probability at least 0.90 has smaller length.

Worked Example: An Upper-Tail Region for an Exponential Distribution

Suppose $X$ has density $f(x)=e^{-x}$ for $x\ge0$, and $f(x)=0$ for $x<0$. For target mass $p=0.80$, let $b$ satisfy $P(X\le b)=0.80$. Since $P(X\le b)=1-e^{-b}$, solving $1-e^{-b}=0.80$ gives $e^{-b}=0.20$ and $b=-\log(0.20)=\log 5\approx1.6094$.

The density threshold at this endpoint is $t=e^{-b}=0.20$. The superlevel region is $[0,b]$; its probability is $1-e^{-b}=0.80$, and its volume is its length, $b\approx1.6094$. The threshold theorem proves that every region containing at least 0.80 probability has length at least $1.6094$. In particular, taking a longer interval into the lower-density tail is less volume-efficient than retaining points near the density peak.

Worked Example: A Flat Density and Nonunique Optimal Regions

Let $X$ be uniform on $[0,1]$, and take $p=0.40$. Its density equals 1 on $[0,1]$. The threshold $t=1$ has $P(f(X)>1)=0$ and $P(f(X)\ge1)=1$. To attain mass 0.40, choose any measurable subset $B$ of $[0,1]$ with length 0.40.

For example, both $B=[0,0.40]$ and $B=[0.30,0.70]$ have probability 0.40 and volume 0.40. No set of volume less than 0.40 can contain probability 0.40, because the density is at most 1 and therefore $P(X\in C)=\int_C f(x)\,dx\le\operatorname{Vol}(C)$. The two choices are both optimal. The threshold identifies the tied region but cannot distinguish among its subsets.

How Threshold Estimation Affects Region Recovery

In practice, the density and the mass-calibrating threshold are estimated. A useful deterministic comparison separates their effects. Define the target set $A=\{x:f(x)\ge t\}$ and an estimated set $\widehat A=\{x:\widehat f(x)\ge\widehat t\}$. Assume that $f$ and $\widehat f$ are measurable and that, on the domain of interest, $\sup_x|\widehat f(x)-f(x)|\le\delta$ and $|\widehat t-t|\le\eta$. Points classified differently by the two sets must lie near the true density boundary.

Result: Under the preceding uniform error bounds, $$ A\mathbin{\triangle}\widehat A \subseteq \{x:|f(x)-t|\le\delta+\eta\}, $$ where $\triangle$ denotes symmetric difference. Therefore $$ P(X\in A\mathbin{\triangle}\widehat A) \le P(|f(X)-t|\le\delta+\eta). $$

To verify the inclusion, consider a point in $A\setminus\widehat A$. Then $f(x)\ge t$ but $\widehat f(x)<\widehat t$. The error bounds imply $f(x)<\widehat t+\delta\le t+\eta+\delta$, while $f(x)\ge t$. Thus $0\le f(x)-t<\delta+\eta$. For a point in $\widehat A\setminus A$, $\widehat f(x)\ge\widehat t$ and $f(x)<t$; hence $f(x)\ge\widehat t-\delta\ge t-\eta-\delta$, so $|f(x)-t|\le\delta+\eta$ there as well. These are the only two ways membership can differ, proving the inclusion. Taking probabilities gives the bound.

This result separates two sources of error: the density estimate contributes $\delta$, and threshold error contributes $\eta$. Even a small combined error need not guarantee a small probability of disagreement if much probability lies near the threshold. Conversely, if little probability lies near $t$, the region can be stable despite some pointwise error. The bound concerns probability of disagreement; a corresponding volume bound follows by taking Lebesgue measure of the same band when that measure is finite.

Threshold calibration result: A useful special case bounds threshold error when the estimated density is uniformly close and its threshold is calibrated to have mass $p$ under the true distribution. Suppose $P(f(X)\ge t)=p$, $P(\widehat f(X)\ge\widehat t)=p$, $\sup_x|\widehat f(x)-f(x)|\le\delta$, and the survival function $S(u)=P(f(X)\ge u)$ is continuous and strictly decreasing in a neighborhood of $t$. If the thresholds lie in that neighborhood, then $|\widehat t-t|\le\delta$.

Indeed, $\{f\ge\widehat t+\delta\}\subseteq\{\widehat f\ge\widehat t\}\subseteq\{f\ge\widehat t-\delta\}$. Taking probabilities gives $S(\widehat t+\delta)\le p=S(t)\le S(\widehat t-\delta)$. Since $S$ is strictly decreasing near $t$, the first inequality implies $\widehat t+\delta\ge t$, and the second implies $\widehat t-\delta\le t$. Together these yield $t-\delta\le\widehat t\le t+\delta$. Substituting $\eta=\delta$ into the region bound places all disagreements in the band $\{|f-t|\le2\delta\}$. This calibration condition is a population statement; a threshold chosen from a finite sample can have additional mass-calibration error.

Worked Example: A Density Estimate Shifts the Calibrated Boundary

On $[0,1]$, consider the true density $f(x)=2(1-x)$ and target mass $p=0.75$. The true mass of $[0,b]$ is $\int_0^b 2(1-x)\,dx=2b-b^2=1-(1-b)^2$. Setting this to 0.75 gives $(1-b)^2=0.25$, so $b=0.50$. The true threshold is $t=f(0.50)=1$.

Now use the valid estimated density $\widehat f(x)=1.9-1.8x$. It integrates to $1.9-0.9=1$, and it is positive on $[0,1]$. Its uniform error is $|\widehat f(x)-f(x)|=|0.1(2x-1)|\le0.1$. Calibrate its region to have estimated mass 0.75. Since $\widehat f$ decreases on $[0,1]$, the region is $[0,\widehat b]$, where $\int_0^{\widehat b}(1.9-1.8x)\,dx=1.9\widehat b-0.9\widehat b^2=0.75$. Solving the quadratic gives $\widehat b=(1.9-\sqrt{0.91})/1.8\approx0.5256$; the other root exceeds 1 and is not a boundary in the domain. Thus $\widehat t=\widehat f(\widehat b)\approx0.9539$, and the estimated endpoint is about $0.0256$ to the right of the true endpoint. Under the true density, this estimated region has mass $1-(1-0.5256)^2\approx0.7749$, not 0.75. Calibration under an estimated density does not by itself guarantee exact mass under the true distribution.

Common Pitfall

A common mistake is to define the optimal region as $\{f\ge t\}$ and assume it always has exactly the requested mass. That reasoning fails when the density has a plateau at the threshold: in the uniform example, $\{f\ge1\}=[0,1]$ has mass 1, not 0.40. The correction is to use the two inequalities $P(f>t)\le p\le P(f\ge t)$ and, when necessary, select only a suitable part of $\{f=t\}$. A second, related mistake is to treat density estimation error as the only source of recovery error. The threshold error contributes too, and observations with density values close to $t$ can change membership under either perturbation.

Key takeaway: A minimum-volume region of prescribed mass keeps the highest-density points first. At a tied density level, include only enough of that level to reach the target mass. For estimated regions, recovery depends on both density and threshold accuracy, as well as the probability concentrated near the threshold.

Check Your Understanding

Use the density-threshold characterization and the error bounds to answer these questions.

Why does a density threshold need to satisfy two probability inequalities rather than just $P(f(X)\ge t)=p$?
For a density with no positive-volume level set at the threshold, what does the threshold characterization imply about the optimal region?
For the uniform distribution on $[0,1]$ and target mass 0.25, give two distinct minimum-volume regions and verify their masses and volumes.
If $\sup_x|\widehat f(x)-f(x)|\le0.03$ and $|\widehat t-t|\le0.02$, what density band contains every point where the estimated and true threshold sets disagree?
Why can a region calibrated to have estimated-density mass $p$ fail to have true probability mass $p$?