Tutorials › Stochastic Processes › Continuous Laws and Probability Densities

Home  /  Tutorials  /  Random Variables, Expectation and Conditional Expectation

Random Variables, Expectation and Conditional Expectation · Tutorial 4 of 994

Continuous Laws and Probability Densities

Interpret densities as functions whose integrals give probabilities, and use them to check normalization and calculate interval probabilities for three standard continuous laws.

Intermediate 9 min read

What You'll Learn

  • Define a probability density through its integrals over measurable sets.
  • Check that a proposed density is nonnegative and integrates to one.
  • Compute interval probabilities while handling endpoints correctly.
  • Normalize and use uniform and exponential densities.
  • Use the standard normal density to express probabilities with its distribution function.
  • Explain why a density value is not the probability of a point.

What a Probability Density Measures

A probability density describes how probability is spread across the real line. Its value at one point is not, in general, the probability of that point. To obtain a probability, integrate the density over a set: the interval, or more generally the measurable region, whose probability is wanted. This distinction is important when interpreting continuous measurements, such as waiting times or errors.

Let \(X\) be a real-valued random variable on a probability space \((\Omega,\mathcal{F},\mathbb{P})\). A density is a measurable function \(f:\mathbb{R}\to[0,\infty]\) that represents the law of \(X\) by integration. Integrals below are Lebesgue integrals with respect to length on the real line. The relevant sets \(A\) are Borel sets, which include intervals and the usual sets formed from intervals.

Definition: The random variable \(X\) has probability density \(f\) if, for every Borel set \(A\subseteq\mathbb{R}\), \(\mathbb{P}(X\in A)=\int_A f(x)\,dx\). In particular, a density must be nonnegative and must satisfy \(\int_{\mathbb{R}}f(x)\,dx=1\). A function meeting these two conditions defines a probability law by this formula.

The integral over the entire real line is the total probability, so normalization is essential. A density may be greater than one at some points; what must equal one is its total integral. Also, the density need not be uniquely specified at every point. Changing its value at a single point does not change any of its integrals, and therefore does not change the law it represents.

Theorem (Density characterization): A measurable function \(f:\mathbb{R}\to[0,\infty]\) is a probability density for some real-valued random variable if and only if \(\int_{\mathbb{R}}f(x)\,dx=1\). If \(X\) has density \(f\), then for every Borel set \(A\), \(\mathbb{P}(X\in A)=\int_A f(x)\,dx\).

Proof. First suppose \(f\) is a density for a random variable \(X\). Taking \(A=\mathbb{R}\) in the definition gives \(\int_{\mathbb{R}} f(x)\,dx=\mathbb{P}(X\in\mathbb{R})=1\). Nonnegativity is part of the definition. Conversely, suppose \(f\) is measurable, nonnegative, and has integral one. On the measurable space \((\mathbb{R},\mathcal{B}(\mathbb{R}))\), define \(\mathbb{P}_f(A)=\int_A f(x)\,dx\). This assigns zero to the empty set, assigns one to \(\mathbb{R}\), and is countably additive: for pairwise disjoint Borel sets \(A_1,A_2,\ldots\), the indicator of their union is the sum of their indicators, so additivity of the nonnegative integral gives \(\mathbb{P}_f(\bigcup_{k\ge1}A_k)=\sum_{k\ge1}\mathbb{P}_f(A_k)\). Thus \(\mathbb{P}_f\) is a probability measure. On this probability space, let \(X(x)=x\), the identity random variable. Then \(\mathbb{P}_f(X\in A)=\mathbb{P}_f(A)=\int_A f(x)\,dx\), so \(X\) has density \(f\). \(\square\)

A useful consequence is the interval formula. If \(a<b\), then the probability that \(X\) lies between \(a\) and \(b\) is the integral of \(f\) over that interval. Whether either endpoint is included does not change the answer: a singleton has length zero and hence has integral zero. The following result makes this precise.

Proposition (Interval probabilities and point probabilities): If \(X\) has density \(f\), then for finite \(a<b\), \(\mathbb{P}(a<X<b)=\mathbb{P}(a\le X\le b)=\int_a^b f(x)\,dx\). Moreover, \(\mathbb{P}(X=c)=0\) for every real \(c\).

Proof. Since a singleton has Lebesgue measure zero, its integral is zero: \(\int_{\{c\}}f(x)\,dx=0\). The density definition therefore gives \(\mathbb{P}(X=c)=0\). The sets \((a,b)\), \([a,b]\), \([a,b)\), and \((a,b]\) differ only by one or both of the endpoint singletons \(\{a\}\) and \(\{b\}\). Those singletons have probability zero, so all four sets have the same probability. By the density definition, the probability of any of them equals the integral over that interval; including or excluding its endpoints does not affect the integral. \(\square\)

For a density \(f\), the distribution function is consequently \(F_X(x)=\mathbb{P}(X\le x)=\int_{-\infty}^{x}f(u)\,du\). This expresses a cumulative probability as an integral, but a density is more than a formula for an interval: its integral gives the probability of every Borel set, including sets that are not single intervals.

Three Standard Densities

Worked Example: A Uniform Law on an Interval

Suppose \(X\) is uniform on \([1,5]\). Its proposed density is \(f(x)=\tfrac14\) for \(1\le x\le5\), and \(f(x)=0\) otherwise. Check that this is normalized, then find \(\mathbb{P}(2\le X<4)\).

The function is nonnegative. Its total integral is

$$ \int_{\mathbb{R}}f(x)\,dx =\int_1^5 \tfrac14\,dx =\tfrac14(5-1) =1. $$

It is therefore a density. The requested set includes 2 but excludes 4; by the interval formula, endpoint choices do not affect the integral. Hence

$$ \mathbb{P}(2\le X<4) =\int_2^4 \tfrac14\,dx =\tfrac14(4-2) =\tfrac12. $$

Check: The interval \([2,4)\) has length \(4-2=2\), and the density there is \(\tfrac14\), so its probability is \(2\cdot\tfrac14=\tfrac12\), as computed. The density height is \(\tfrac14\); it is not the probability of any particular point.

Worked Example: An Exponential Waiting-Time Law

For rate \(2\), the exponential density is \(f(x)=2e^{-2x}\) for \(x\ge0\), and \(f(x)=0\) for \(x<0\). Verify the normalization and calculate the probability that \(X\) lies in \((\tfrac12,1]\).

The density is nonnegative on its support. Its total integral is

$$ \int_{\mathbb{R}}f(x)\,dx =\int_0^\infty 2e^{-2x}\,dx =\left[-e^{-2x}\right]_0^\infty =0-(-1) =1. $$

Thus it is normalized. The requested interval is contained in the nonnegative support. Integrating gives

$$ \mathbb{P}\left(\tfrac12<X\le1\right) =\int_{1/2}^{1}2e^{-2x}\,dx =\left[-e^{-2x}\right]_{1/2}^{1} =-e^{-2}+e^{-1} =e^{-1}-e^{-2}. $$

Check: The probability from \(0\) to \(1\) is \(\int_0^1 2e^{-2x}\,dx=1-e^{-2}\), while the probability from \(0\) to \(\tfrac12\) is \(\int_0^{1/2}2e^{-2x}\,dx=1-e^{-1}\). Subtracting gives \((1-e^{-2})-(1-e^{-1})=e^{-1}-e^{-2}\), the same result.

Worked Example: The Standard Normal Law

The standard normal density is \(\phi(x)=\tfrac{1}{\sqrt{2\pi}}e^{-x^2/2}\) for \(x\in\mathbb{R}\). We verify its normalization and express \(\mathbb{P}(-1<X\le1)\) in terms of its distribution function.

For normalization, let \(I=\int_{-\infty}^{\infty}e^{-x^2/2}\,dx\). The integrand is positive, so multiplying the two one-dimensional integrals gives the nonnegative double integral \(I^2=\int_{\mathbb{R}^2}e^{-(x^2+y^2)/2}\,dx\,dy\). Evaluating this integral in polar coordinates, whose area element is \(r\,dr\,d\theta\), gives

$$ I^2 =\int_0^{2\pi}\int_0^\infty e^{-r^2/2}r\,dr\,d\theta =2\pi\left[-e^{-r^2/2}\right]_0^\infty =2\pi. $$

Since \(I>0\), it follows that \(I=\sqrt{2\pi}\), and therefore \(\int_{\mathbb{R}}\phi(x)\,dx=I/\sqrt{2\pi}=1\).

Define \(\Phi(x)=\int_{-\infty}^{x}\phi(u)\,du\), the standard normal distribution function. Then

$$ \mathbb{P}(-1<X\le1) =\int_{-1}^{1}\phi(x)\,dx =\Phi(1)-\Phi(-1) =2\Phi(1)-1. $$

The last equality follows from symmetry: \(\phi(-x)=\phi(x)\), so the probability to the left of \(-1\) equals the probability to the right of \(1\), giving \(\Phi(-1)=1-\Phi(1)\). The integral has no elementary antiderivative, so expressing the answer using \(\Phi\) is a standard exact form.

Check: Symmetry also gives \(\int_{-1}^{1}\phi(x)\,dx=2\int_0^1\phi(x)\,dx\). Since \(\int_0^1\phi(x)\,dx=\Phi(1)-\Phi(0)=\Phi(1)-\tfrac12\), this equals \(2\Phi(1)-1\), agreeing with the interval calculation.

Interpreting and Checking a Density

These examples illustrate a practical procedure. First check that the function is measurable and nonnegative. Then integrate over its full support and confirm that the result is one. Only after these checks should it be used to calculate probabilities. For an interval, integrate over the part of the interval that lies in the support. A density that is not normalized cannot be used directly as a probability law; it must first be corrected if a normalized version is intended.

The phrase “density at \(x\)” refers to a function value, not a point probability. For example, in the uniform example, \(f(3)=\tfrac14\), but \(\mathbb{P}(X=3)=\int_{\{3\}}f(x)\,dx=0\). In a small interval around a point, density height helps describe probability per unit length, but the probability is still an integral over the whole interval. The value at a single point alone does not determine that integral.

Common mistake. A tempting but incorrect calculation is \(\mathbb{P}(X=3)=f(3)=\tfrac14\) for the uniform law on \([1,5]\). This confuses a density value with a point mass. The correction is to integrate over the singleton: \(\mathbb{P}(X=3)=\int_{\{3\}}\tfrac14\,dx=0\). For an interval of positive length, such as \([2,4]\), integration instead gives \(\int_2^4\tfrac14\,dx=\tfrac12\).

The density description also makes endpoint conventions manageable. For any finite endpoints \(a<b\), the probabilities of \((a,b)\), \([a,b]\), \([a,b)\), and \((a,b]\) agree for a variable with a density. This does not mean endpoint conventions are always interchangeable for arbitrary random variables: a variable with a point mass can assign positive probability to an endpoint. The zero endpoint probability here follows from the density assumption and the fact that a singleton has zero length.

Check Your Understanding

For each question, state the integration region and keep track of whether the stated endpoints are included or excluded.

  1. A proposed density equals \(\tfrac13\) on \([0,3]\) and zero elsewhere. Verify normalization and calculate \(\mathbb{P}(1<X\le2)\).
  2. For the rate-\(2\) exponential density, calculate \(\mathbb{P}(X>1)\) by integrating over its full tail.
  3. Why does a density value \(f(c)\) not, by itself, give \(\mathbb{P}(X=c)\)? State the point-probability result for a variable with a density.
  4. Express the standard normal probability \(\mathbb{P}(0<X\le1)\) using \(\Phi\), and explain why including \(0\) would not change it.
  5. What two checks must a proposed nonnegative function pass before it can be used as a probability density?

Previous

Next

Back to Tutorials