Separating Discrete, Density, and Singular Features

Why One Probability Law Can Have Different Features

A probability law is sometimes described by a probability mass function, sometimes by a density, and sometimes by its distribution function. These descriptions are not interchangeable. A point mass assigns positive probability to one exact value; a density assigns probability through integration over sets; and a singular-continuous law has no point masses and is concentrated on a set of Lebesgue measure zero. A single law can combine more than one of these features.

The useful question is not simply whether a law is “discrete” or “continuous.” It is: what portions of its probability are atomic, density-based, or singular continuous? The following classification makes that question precise.

Definition: Let \(X\) be a real-valued random variable with law \(\mu\), so \(\mu(B)=P(X\in B)\) for every Borel set \(B\subseteq\mathbb R\). A point \(x\) is an atom if \(\mu(\{x\})>0\). The atomic part of \(\mu\) is the measure \(\mu_{\mathrm{at}}=\sum_{x:\,\mu(\{x\})>0}\mu(\{x\})\delta_x\), where \(\delta_x\) puts probability one at \(x\).

An atom concerns the probability of a single value, not the height of a density. A law is purely discrete if its atoms carry all its probability: \(\mu_{\mathrm{at}}(\mathbb R)=1\). Equivalently, it is concentrated on a finite or countable set. A law is absolutely continuous with respect to Lebesgue measure if there is a nonnegative measurable function \(f\), with \(\int_{\mathbb R}f(x)\,dx=1\), such that \(\mu(B)=\int_B f(x)\,dx\) for every Borel set \(B\). The function \(f\) is a density.

Definition: A finite measure \(\nu\) is singular with respect to Lebesgue measure if it is concentrated on a Borel set \(N\) with Lebesgue measure zero: \(\nu(\mathbb R\setminus N)=0\). A probability law is singular continuous if it is singular and has no atoms. It is continuous in the sense that its distribution function has no jumps, not in the sense that it has a density.

The terms describe different properties. A purely discrete law is made entirely of atoms. An absolutely continuous law has no atoms and assigns zero probability to every Lebesgue-null set. A singular-continuous law also has no atoms, but it places all its probability on a Lebesgue-null set. Mixed laws can have positive amounts of more than one type.

The Canonical Separation

For probability laws on the real line, these features can be separated into three measures. The result uses the Lebesgue decomposition theorem: any finite measure on \(\mathbb R\) has a unique decomposition into a measure absolutely continuous with respect to Lebesgue measure and a measure singular with respect to it.

Theorem (three-part decomposition): Every probability law \(\mu\) on \(\mathbb R\) has a unique representation \(\mu=\mu_{\mathrm{at}}+\mu_{\mathrm{ac}}+\mu_{\mathrm{sc}}\), where \(\mu_{\mathrm{at}}\) is its atomic part, \(\mu_{\mathrm{ac}}(B)=\int_B f(x)\,dx\) for a nonnegative integrable function \(f\), and \(\mu_{\mathrm{sc}}\) is singular with respect to Lebesgue measure and has no atoms. The three measures are nonnegative; their total masses sum to one. Any of the three may have mass zero.

Here the measures in the representation need not themselves be probability laws: they may have total mass less than one. Their masses indicate how much of the law belongs to each feature. The density of \(\mu_{\mathrm{ac}}\) need not integrate to one; it integrates to \(\mu_{\mathrm{ac}}(\mathbb R)\).

Justification and uniqueness. First, the atom locations form a countable set. For each positive integer \(m\), there can be only finitely many locations whose probability is at least \(1/m\): \(m+1\) such disjoint singleton events would have total probability at least \((m+1)/m>1\). Every positive atom belongs to one of these finite sets for some \(m\), so the set of all atom locations is countable. For any Borel set \(B\), countable additivity gives \(\mu_{\mathrm{at}}(B)\leq\mu(B)\). Thus \(\mu_0=\mu-\mu_{\mathrm{at}}\) is a nonnegative measure. At each singleton, \(\mu_0(\{x\})=0\): if \(x\) is an atom its full mass was removed, and otherwise its mass was already zero.

Apply the Lebesgue decomposition theorem to the finite measure \(\mu_0\), obtaining \(\mu_0=\mu_{\mathrm{ac}}+\nu_{\mathrm{s}}\), with \(\mu_{\mathrm{ac}}\) absolutely continuous and \(\nu_{\mathrm{s}}\) singular. Since \(\mu_{\mathrm{ac}}\) assigns zero mass to every singleton and \(\mu_0\) has no atoms, \(\nu_{\mathrm{s}}(\{x\})=\mu_0(\{x\})-\mu_{\mathrm{ac}}(\{x\})=0\) for every \(x\). Hence \(\nu_{\mathrm{s}}\) is singular continuous. The atomic measure is determined by all the singleton probabilities, and the Lebesgue decomposition of the remainder is unique. This proves the stated uniqueness. \(\square\)

Result: The set of atoms of any probability law on \(\mathbb R\) is at most countable.

The countability claim also follows directly from the proof: for each \(m\), the set of atoms of size at least \(1/m\) is finite, and every atom has size at least \(1/m\) for some sufficiently large \(m\). In particular, “atomic” does not mean that there must be only finitely many possible values.

Worked Classifications

Worked Example: A Law with an Atom and a Density

Suppose \(X\) equals zero with probability \(1/4\), and with probability \(3/4\) it has the uniform distribution on \((0,2)\). Classify the parts of its law.

The uniform distribution on \((0,2)\) has density \(1/2\) on that interval. Therefore the full law is \(\mu=\frac14\delta_0+\frac38\mathbf{1}_{(0,2)}(x)\,dx\). The atomic part has mass \(1/4\), located at zero. The density-based part has density \(f(x)=\frac38\mathbf{1}_{(0,2)}(x)\), and its total mass is \(\int_0^2\frac38\,dx=\frac34\). There is no singular-continuous part.

The density integrates to \(3/4\), not to one, because it describes only the continuous portion of this mixed law. The full law is not absolutely continuous: its atom at zero gives positive probability to the null set \(\{0\}\).

Worked Example: A Countable Purely Discrete Law

Let \(P(X=1/k)=2^{-k}\) for each positive integer \(k\), with no other possible values. Verify that this defines a purely discrete law and identify its atomic mass.

Each listed probability is positive, so every \(1/k\) is an atom. The total mass is \(\sum_{k=1}^{\infty}2^{-k}=\lim_{m\to\infty}\sum_{k=1}^{m}2^{-k}=\lim_{m\to\infty}(1-2^{-m})=1\). Thus these atom probabilities specify a probability law, and their countable set carries all its mass. Its atomic part has total mass one; its absolutely continuous and singular-continuous parts both have mass zero. The fact that there are infinitely many atoms does not make the law mixed.

Worked Example: A Singular-Continuous Component

Let \(C\) have the standard Cantor distribution, and form a law that selects a point mass at zero with probability \(1/5\), a uniform distribution on \((0,1)\) with probability \(3/10\), and the law of \(C\) with probability \(1/2\). Classify its three components.

The weights sum to \(1/5+3/10+1/2=2/10+3/10+5/10=1\). The first component contributes the atom \((1/5)\delta_0\). The uniform component contributes the density \((3/10)\mathbf{1}_{(0,1)}(x)\), whose integral is \(3/10\). The Cantor component is atomless and concentrated on the Cantor set, which has Lebesgue measure zero. It therefore contributes a singular-continuous measure of mass \(1/2\).

Although zero lies in the Cantor set and is also the location of the separate point mass, the Cantor law itself has no atom there. The total law’s atom at zero is therefore \(1/5\), while the Cantor component remains singular continuous. This example has all three features, with masses \(1/5\), \(3/10\), and \(1/2\), respectively.

Why Having No Atoms Is Not Enough

A common mistake is to reason: “There are no point masses, so the law must have a density.” That conclusion is false. Absence of atoms rules out only the atomic part; it does not rule out a singular-continuous part. The Cantor law is the standard counterexample: it assigns probability one to a Lebesgue-null set, yet assigns probability zero to each singleton.

There is also a precise reason a singular-continuous probability law cannot have a density. Suppose \(\mu\) is concentrated on a Borel null set \(N\), and suppose it also has a density \(f\). Then \(\mu(N)=\int_N f(x)\,dx=0\), because \(N\) has Lebesgue measure zero. But concentration on \(N\) requires \(\mu(N)=1\), a contradiction. Thus a singular-continuous law has no density representation with respect to Lebesgue measure.

Likewise, a law with any positive atom cannot be represented by a density alone: a density assigns probability zero to every singleton, since the Lebesgue integral over a singleton is zero. When a law has both atoms and a density-based component, the density describes just one part of it.

Key takeaway: Inspect point probabilities to identify atoms, check whether the remaining law is represented by a density, and allow for a singular-continuous remainder. “No atoms” does not imply “has a density,” and “has a density component” does not imply that the whole law has a density.

Check Your Understanding

For each question, distinguish the type of feature involved rather than relying only on whether the distribution function is continuous.

What makes a point \(x\) an atom, and why must the atoms of a probability law on \(\mathbb R\) be at most countable?
A law has an atom of mass \(0.2\) and an absolutely continuous component of mass \(0.8\). Can the whole law have a density? Explain.
Why is a singular-continuous law not purely discrete, even though it is concentrated on a special set?
In the three-component example, which feature contributes the density, and what is the total mass of that component?
Why does the absence of point masses not establish that a law is absolutely continuous?

Separating Discrete, Density, and Singular Features

What You'll Learn