Probability Mass Functions for Discrete Laws

From a Discrete Random Variable to Its Mass Function

A discrete random variable can take only finitely or countably many values with probability one. Its probability mass function records the probability at each individual value. Once those point probabilities are known, probabilities of events can be found by adding the masses of the values the event includes.

The relevant events must be measurable. A real-valued random variable is understood to be measurable with respect to the Borel sets, so probabilities such as $P(X\in A)$ are guaranteed to be defined when $A$ is a Borel subset of $\mathbb{R}$. This condition will be stated explicitly in the event-probability formula below.

Definition: A real-valued random variable $X$ is discrete if there is a finite or countable set $S\subseteq\mathbb{R}$ such that $P(X\in S)=1$. Its probability mass function is $f(x)=P(X=x)$, for $x\in\mathbb{R}$. The points with positive mass form the set $\{x\in\mathbb{R}:f(x)>0\}$, which is finite or countable. At points outside this set, $f(x)=0$.

A point mass is the probability of one exact value. In particular, $f(x)$ is not the probability that $X$ is at most $x$; that is a cumulative probability. If an event includes several possible values, its probability is found by adding their point masses.

The masses are nonnegative and add to one over the possible values. If $S$ is any finite or countable set with $P(X\in S)=1$, countable additivity gives

\sum_{x\in S} f(x) =\sum_{x\in S}P(X=x) =P(X\in S) =1.

The middle equality holds because the events $\{X=x\}$, for distinct $x\in S$, are disjoint and their union is $\{X\in S\}$.

Getting an Event Probability from the Masses

For a Borel event $A\subseteq\mathbb{R}$, the outcomes counted by $P(X\in A)$ are exactly those at which $X$ equals a value in $S\cap A$, apart from a possible event of probability zero outside $S$. The corresponding formula is the basic way to use a mass function.

Theorem: If $X$ is discrete and $S$ is a finite or countable set with $P(X\in S)=1$, then for every Borel set $A\subseteq\mathbb{R}$, $$ P(X\in A)=\sum_{x\in S\cap A} f(x). $$ Consequently, the mass function determines the distribution of $X$ on all Borel sets.

Proof. Since $X$ is a random variable and $A$ is Borel, the event $\{X\in A\}$ is measurable. The event $\{X\in S\}$ is also measurable and has probability one. Therefore, removing outcomes outside $S$ does not change the probability:

P(X\in A)=P(X\in A,\ X\in S).

The event on the right is the union of the events $\{X=x\}$ over $x\in S\cap A$. This is a finite or countable union of pairwise disjoint measurable events. By countable additivity,

P(X\in A) =\sum_{x\in S\cap A}P(X=x) =\sum_{x\in S\cap A}f(x).

If two discrete random variables have the same mass function, this formula gives the same probability for every Borel set $A$. Thus they have the same distribution. $\square$

Worked Examples

Worked Example: A Fair Six-Sided Die

Let $X$ be the number shown on a fair six-sided die. Its possible values are $1,2,3,4,5,6$, each with probability $1/6$. Thus $f(k)=1/6$ for $k\in\{1,2,3,4,5,6\}$, and $f(x)=0$ at all other real values. The masses sum to

\sum_{k=1}^{6} f(k)=6\left(\frac{1}{6}\right)=1.

Find the probability that the result is $1$, $4$, or $6$. The event contains three values, so its probability is the sum of those three masses:

P(X\in\{1,4,6\}) =f(1)+f(4)+f(6) =\frac{1}{6}+\frac{1}{6}+\frac{1}{6} =\frac{1}{2}.

Each mass is a probability at one value; the probability of the event is the total mass over its specified set of values.

Worked Example: Number of Heads in Two Tosses

Toss a fair coin twice, and let $X$ be the number of heads. The four equally likely outcomes are HH, HT, TH, and TT. The value $X=0$ occurs for TT, $X=1$ occurs for HT and TH, and $X=2$ occurs for HH. Therefore,

f(0)=\frac{1}{4},\qquad f(1)=\frac{2}{4}=\frac{1}{2},\qquad f(2)=\frac{1}{4}.

These are nonnegative and their sum is $1/4+1/2+1/4=1$. For the event that at least one head occurs, the included values are $1$ and $2$, giving

P(X\geq 1)=f(1)+f(2) =\frac{1}{2}+\frac{1}{4} =\frac{3}{4}.

For the event that the number of heads is odd, only $X=1$ is included, so $P(X\text{ is odd})=f(1)=1/2$. The two event probabilities differ because they include different sets of values.

Worked Example: Waiting Time Until the First Success

Suppose independent trials each result in success with probability $p=0.2$, and let $T$ be the trial number of the first success. For $k=1,2,\ldots$, the event $T=k$ requires $k-1$ failures followed by a success. Independence gives

f(k)=P(T=k)=(0.8)^{k-1}(0.2).

For example, $f(1)=0.2$, $f(2)=0.8(0.2)=0.16$, and $f(3)=0.8^2(0.2)=0.128$. The event that the first success occurs on an odd-numbered trial has probability

\begin{aligned} P(T\text{ is odd}) &=f(1)+f(3)+f(5)+\cdots\\ &=0.2+0.8^2(0.2)+0.8^4(0.2)+\cdots\\ &=0.2\sum_{j=0}^{\infty}(0.8^2)^j\\ &=\frac{0.2}{1-0.64} =\frac{5}{9}. \end{aligned}

The series formula applies because $0.8^2=0.64<1$. As a check, the probability of an even-numbered trial is $0.8/1.8=4/9$, and $5/9+4/9=1$.

When a List of Masses Defines a Law

The mass function is not merely a convenient summary. Subject to the nonnegativity and total-mass conditions, a finite or countable list of masses can itself be used to define a probability law. The construction below makes precise why those conditions are essential.

Theorem: Let $S\subseteq\mathbb{R}$ be finite or countable, and assign numbers $f(x)\geq 0$ for $x\in S$, with $\sum_{x\in S}f(x)=1$. On the sample space $S$, define $P(B)=\sum_{x\in B}f(x)$ for each subset $B\subseteq S$, and let $X(x)=x$. Then $P$ is a probability measure, $X$ is a random variable, and its mass function on $S$ is $f$.

Proof. The empty set has probability zero because its sum contains no terms. The whole space has probability one by the assumed total-mass condition. To check countable additivity, let $B_1,B_2,\ldots$ be pairwise disjoint subsets of $S$. Each point of their union occurs in exactly one of the sets, so rearranging the nonnegative terms gives

P\left(\bigcup_{j=1}^{\infty}B_j\right) =\sum_{x\in\bigcup_j B_j}f(x) =\sum_{j=1}^{\infty}\sum_{x\in B_j}f(x) =\sum_{j=1}^{\infty}P(B_j).

Thus $P$ is a probability measure on the power set of $S$. The identity map $X(x)=x$ is measurable as a real-valued random variable: the inverse image of any Borel set $A$ is $S\cap A$, a subset of $S$. Finally, for $x\in S$, $P(X=x)=P(\{x\})=f(x)$. This proves the claim. $\square$

The theorem gives a useful check: a proposed set of point masses cannot define a probability law if any mass is negative or if their total is not one. Determining how to adjust unnormalized weights is a separate calculation; the defining condition for a mass function is that its masses already total one.

Common Mistakes and a Measurability Caution

Using one mass as though it covered a larger event. $f(x)$ gives $P(X=x)$, not the probability of every event containing $x$. For an event containing several possible values, add all their masses.
Summing values instead of probabilities. To find $P(X\in A)$, sum $f(x)$ for $x\in S\cap A$; do not sum the values $x$ themselves.
Forgetting values outside the event. A point contributes to the event probability only when it belongs to the event. In the two-toss example, $X=0$ does not contribute to $P(X\geq1)$.
Writing the formula for arbitrary subsets of $\mathbb{R}$ without a measurability condition. For a general real-valued random variable, $\{X\in A\}$ is guaranteed to be an event when $A$ is Borel. The formula above is therefore stated for Borel $A$. One should not claim that $P(X\in A)$ is defined for every subset $A$ unless measurability of the inverse image has been established.

Key takeaway: A discrete mass function assigns nonnegative probabilities to individual values, with total mass one. For every Borel event $A$, its probability is the sum of the masses at the possible values lying in $A$.

Check Your Understanding

Answer each question using the definition and event-summing rule.

A discrete variable has masses $f(0)=0.1$, $f(1)=0.6$, and $f(3)=0.3$. Find $P(X\in\{0,3\})$.
For the two-toss variable $X$, which point masses contribute to $P(X\leq1)$, and what is the resulting probability?
Why is the sum of all point masses of a discrete random variable equal to one?
What two conditions on a finite or countable list of masses are needed for it to define a probability law?
Why is the event-probability formula stated for Borel sets rather than unrestricted subsets of $\mathbb{R}$?