Why Density Height Is Not Point Probability

Density Height Is Not a Probability

A probability cannot exceed one, so a density value greater than one can initially seem impossible. The apparent contradiction disappears once we distinguish probability over a set from density height at a location. A density describes how probability accumulates across intervals; its height alone is not the probability of observing a particular value.

For a continuous law, an interval can have positive probability even though each exact point has probability zero. The interval's probability comes from the area under the density across that interval. Consequently, a density can be tall if it is concentrated over a sufficiently narrow range.

Definition: A random variable $X$ has a probability density function $f$ if $f$ is a nonnegative measurable function on the real line, $\int_{-\infty}^{\infty} f(x)\,dx=1$, and for every Borel set $A$, $$ P(X\in A)=\int_A f(x)\,dx. $$ A law with such a density is called absolutely continuous (with respect to Lebesgue measure). In particular, for $a\le b$, $$ P(a\le X\le b)=\int_a^b f(x)\,dx. $$

The integral, not the height at an endpoint or at a single interior point, gives the probability. The height $f(x)$ can be thought of as probability per unit of the measurement scale. If $f$ is roughly $3$ over an interval of width $0.01$, the probability there is roughly $3(0.01)=0.03$, not $3$.

Why a Density May Exceed One

The density must be nonnegative and have total area one. There is no requirement that its height be at most one. A narrow rectangle illustrates the point: a rectangle of height $5$ and width $0.2$ has area $5(0.2)=1$. Its height exceeds one, while the probability represented by the full rectangle is exactly one.

Worked Example: A Uniform Law With Density Height Five

Let $X$ be uniform on $[0,0.2]$. Its density is $$ f(x)= \begin{cases} 5, & 0\le x\le 0.2,\\ 0, & \text{otherwise}. \end{cases} $$ Check that this is a density and calculate two interval probabilities.

Normalization. The total integral is $$ \int_{-\infty}^{\infty}f(x)\,dx=\int_0^{0.2}5\,dx=5(0.2)=1. $$ Thus the density height of $5$ does not violate the probability bound: the total probability is the area, which is one.

An interval probability. For $0.04\le X\le0.10$, the interval has width $0.10-0.04=0.06$, so $$ P(0.04\le X\le0.10)=\int_{0.04}^{0.10}5\,dx=5(0.06)=0.30. $$ For the smaller interval $[0.04,0.05]$, its width is $0.01$, and $$ P(0.04\le X\le0.05)=\int_{0.04}^{0.05}5\,dx=5(0.01)=0.05. $$ In each calculation, probability equals height times width. The height is not itself either probability.

This relationship also depends on the units of measurement. Suppose the same variable is measured in meters or centimeters. An interval that is $0.2$ meters wide is $20$ centimeters wide, so a density expressed per centimeter must have a different numerical height from a density expressed per meter. The underlying probabilities do not change; only the scale on which density is measured changes.

Worked Example: Changing Units Changes Density Height

Let $X$ be uniform on $[0,0.2]$ meters, as above, and define $Y=100X$, the same length measured in centimeters. Then $Y$ is uniform on $[0,20]$ centimeters. Its density is $$ f_Y(y)= \begin{cases} \frac{1}{20}=0.05, & 0\le y\le20,\\ 0, & \text{otherwise}. \end{cases} $$ The full interval has probability $\int_0^{20}0.05\,dy=0.05(20)=1$.

Consider the same physical portion of the range: $0.04$ to $0.10$ meters, which is $4$ to $10$ centimeters. Using meters gives $$ P(0.04\le X\le0.10)=5(0.10-0.04)=5(0.06)=0.30. $$ Using centimeters gives $$ P(4\le Y\le10)=0.05(10-4)=0.05(6)=0.30. $$ The numerical density height changed from $5$ per meter to $0.05$ per centimeter, but the interval probability stayed the same. Density has units reciprocal to the units of $X$; it is not a unit-free probability.

Why a Singleton Has Probability Zero

A singleton is a set containing exactly one value, such as $\{x_0\}$. Under a law with density, its probability is the integral of the density over that singleton. A single point has length zero, so this integral is zero, even when the density is positive or large at that point.

Theorem: If $X$ has a density $f$, then for every real number $x_0$, $$ P(X=x_0)=0. $$

Proof. By the definition of a density, $$ P(X=x_0)=P(X\in\{x_0\})=\int_{\{x_0\}}f(x)\,dx. $$ The set $\{x_0\}$ has Lebesgue measure zero. The integral of a nonnegative measurable function over a set of measure zero is zero: indeed, nonnegative simple functions have integral zero on that set, and the nonnegative Lebesgue integral is obtained as a limit of integrals of such simple functions. Therefore $\int_{\{x_0\}}f(x)\,dx=0$, proving the claim. $\square$

It follows that including or excluding either endpoint of an interval does not affect its probability under a density. For example, $$ P(a<X\le b)=P(a\le X\le b)=\int_a^b f(x)\,dx, $$ because the two interval events differ only by the possible inclusion of $a$, and $P(X=a)=0$. This endpoint conclusion is specific to laws with densities; it should not be assumed for every probability distribution.

Worked Example: A Positive Density at a Point but Zero Point Probability

Let $X$ have density $f(x)=2x$ for $0\le x\le1$, and $f(x)=0$ otherwise. First check normalization: $$ \int_0^1 2x\,dx=\left[x^2\right]_0^1=1. $$ At $x_0=0.9$, the density height is $f(0.9)=2(0.9)=1.8$, which is greater than one. Nevertheless, $$ P(X=0.9)=\int_{\{0.9\}}2x\,dx=0. $$ The probability of a nearby interval is not zero. For example, $$ P(0.8\le X\le1)=\int_{0.8}^1 2x\,dx=\left[x^2\right]_{0.8}^1=1-0.64=0.36. $$ Thus a high density at a point is compatible both with zero probability at that exact point and with positive probability in a surrounding interval.

Interval Probabilities Explain the Height

The area rule gives the exact interpretation of density height. If a density is constant on an interval, probability over that interval is exactly height times width. If it varies, the probability is the area under its graph; equivalently, the probability divided by the interval width is the average density over that interval.

Theorem: Suppose $f$ is a density and is continuous at $x_0$. For $h>0$ small enough that the interval lies in the domain under consideration, $$ \lim_{h\downarrow0}\frac{P(x_0\le X\le x_0+h)}{h}=f(x_0). $$ Thus, at a point of continuity, density height is the limiting probability per unit length of intervals beginning at that point, not the probability of the point itself.

Proof. By the density formula, $$ \frac{P(x_0\le X\le x_0+h)}{h} =\frac{1}{h}\int_{x_0}^{x_0+h}f(x)\,dx. $$ Subtract $f(x_0)$ and use the triangle inequality: $$ \left|\frac{1}{h}\int_{x_0}^{x_0+h}f(x)\,dx-f(x_0)\right| \le \frac{1}{h}\int_{x_0}^{x_0+h}|f(x)-f(x_0)|\,dx. $$ Given any $\varepsilon>0$, continuity at $x_0$ gives a $\delta>0$ such that $|f(x)-f(x_0)|<\varepsilon$ whenever $|x-x_0|<\delta$. For $0<h<\delta$, the right-hand side is at most $$ \frac{1}{h}\int_{x_0}^{x_0+h}\varepsilon\,dx=\varepsilon. $$ The difference therefore tends to zero as $h\downarrow0$, which proves the limit. $\square$

Worked Example: Approximating a Short-Interval Probability

For the density $f(x)=2x$ on $[0,1]$, consider the interval from $0.9$ to $0.91$. Its width is $0.01$, and the density at the left endpoint is $f(0.9)=1.8$. Using the local-height interpretation gives the approximation $$ P(0.9\le X\le0.91)\approx f(0.9)(0.01)=1.8(0.01)=0.018. $$ The exact probability is $$ P(0.9\le X\le0.91)=\int_{0.9}^{0.91}2x\,dx =\left[x^2\right]_{0.9}^{0.91} =0.91^2-0.9^2 =0.8281-0.81 =0.0181. $$ The approximation is close because the interval is short and the density changes only slightly over it. The exact probability is still the integral, not the density height $1.8$.

A Subtlety: Density Values at Isolated Points

A probability law determines its density only up to changes on sets of length zero. For example, changing the value of $f$ at just $x_0$ leaves every integral $\int_A f(x)\,dx$ unchanged, so it leaves every probability unchanged. A density can therefore be redefined at one point to be $100$, or any other nonnegative finite number, without changing the law.

This fact does not undermine the interval interpretation. When a density is continuous near a point, its values nearby determine the local probability per unit length, as the limit result shows. But an isolated value assigned at a single point has no effect on any interval probability. It is the density's behavior across sets of positive length, not an isolated height, that encodes the law.

Common Mistakes

Reading $f(x_0)$ as $P(X=x_0)$. The density is not the point probability. For a law with density, $P(X=x_0)=0$; interval probabilities are obtained by integrating.
Assuming a density must be at most one. Only its total integral must be one. A tall density over a narrow range can have total area one.
Forgetting units. A density is measured per unit of the variable. Rescaling the variable changes the numerical height but not probabilities of corresponding events.
Using the short-interval approximation as an exact identity. $P(x_0\le X\le x_0+h)\approx f(x_0)h$ is a local approximation when the density changes little; the exact probability is the integral.
Applying the singleton-zero rule to every law. The proof uses the existence of a density. It does not establish that every distribution assigns zero probability to every point.

Key takeaway: A density is probability per unit length, not probability at a point. Integrating over an interval gives its probability, a density height can exceed one, and every singleton has probability zero under a law with a density.

Check Your Understanding

Answer each question by distinguishing density height from probability.

A density equals $4$ throughout an interval of width $0.05$. What is the probability of that interval?
If $X$ has a density $f$ with $f(2)=0.7$, what is $P(X=2)$, and why?
A variable is uniform on an interval of width $0.1$. What is its density height on that interval, and how can that height be greater than one without violating normalization?
Why does changing the value of a density at one point leave the distribution unchanged?
Under what condition does the ratio $P(x_0\le X\le x_0+h)/h$ converge to $f(x_0)$ as $h$ decreases to zero?