Probability Mass Functions For Discrete Responses

From Discrete Responses to Probability Masses

A discrete response takes values in a finite or countable set. Knowing which values are valid is not yet enough to describe uncertainty: we also need to specify how probability is assigned across those values. A probability mass function (PMF) does this by assigning a probability to each individual response value. Once the masses are known, the probability of an event—such as a response belonging to a specified set—is found by adding the masses of the values in that set.

Let \(X\) be a random variable whose response space \(S\) is finite or countably infinite. The set \(S\) contains the values under consideration. A value outside \(S\) has probability zero. The PMF records the probability at each value, rather than a probability spread over intervals.

Definition: The probability mass function of a discrete random variable \(X\) is the function \(p_X\) given by \(p_X(x)=P(X=x)\) for \(x\in S\). It satisfies \(p_X(x)\geq 0\) for every \(x\in S\) and

\sum_{x\in S}p_X(x)=1.

For \(x\notin S\), set \(p_X(x)=0\). The set of values in \(S\) with strictly positive mass is \(\{x\in S:p_X(x)>0\}\).

If \(S\) is finite, the sum in the definition is an ordinary finite sum. If \(S\) is countably infinite, the sum is a series of nonnegative terms. It can be defined as the limit of its partial sums after enumerating the values of \(S\); because all terms are nonnegative, the resulting total does not depend on the enumeration. In either case, nonnegativity prevents positive and negative terms from cancelling, and the total must be exactly one.

A response space may include a value with mass zero. Such a value remains valid as a value being considered, but the event \(X=x\) has probability zero. Thus “valid response value” and “value with positive probability” need not be identical.

Probabilities of Events

An event about \(X\) can be described by a subset \(A\) of its response space: the event occurs when \(X\in A\). Its probability is the sum of the masses assigned to the values in \(A\). If \(A\) contains infinitely many values, the sum is a countable series.

Theorem: If \(X\) has PMF \(p_X\) on a finite or countable response space \(S\), then for every subset \(A\subseteq S\),

P(X\in A)=\sum_{x\in A}p_X(x).

Proof. The event \(\{X\in A\}\) is the union of the events \(\{X=x\}\) for \(x\in A\). These events are pairwise disjoint: if \(x\neq y\), \(X\) cannot equal both \(x\) and \(y\) in the same outcome. Countable additivity of probability therefore gives

P(X\in A) =P\left(\bigcup_{x\in A}\{X=x\}\right) =\sum_{x\in A}P(X=x) =\sum_{x\in A}p_X(x).

For finite \(A\), this is a finite addition. For countably infinite \(A\), it is the countable-additivity rule for a disjoint union. \(\square\)

Two useful consequences follow directly. The probability of a single response value is its mass, \(P(X=x)=p_X(x)\). Also, the probability of an event and its complement add to one:

P(X\in A)+P(X\notin A) =\sum_{x\in A}p_X(x)+\sum_{x\in S\setminus A}p_X(x) =\sum_{x\in S}p_X(x) =1.

The two sums combine because \(A\) and \(S\setminus A\) partition the response space. This can be convenient when one part of the space has fewer values to sum over than the other.

Worked Example: Adding Masses on a Finite Response Space

Suppose a response \(X\) has possible values \(0,1,2,3\), with \(p_X(0)=0.10\), \(p_X(1)=0.30\), \(p_X(2)=0.40\), and \(p_X(3)=0.20\). Verify that these are valid masses, then find the probability that \(X\) is at least 2 and the probability that \(X\) is either 1 or 3.

Check the PMF. Every listed mass is nonnegative, and their total is

$$ 0.10+0.30+0.40+0.20=1.00. $$

Thus the values satisfy the two PMF requirements. For the event \(X\geq 2\), the relevant values are 2 and 3, so

P(X\geq 2)=p_X(2)+p_X(3)=0.40+0.20=0.60.

For the event \(X\in\{1,3\}\), add the masses at 1 and 3:

P(X\in\{1,3\})=p_X(1)+p_X(3)=0.30+0.20=0.50.

The event probability is not the number of values in the event. It is the total mass assigned to those values.

Countably Infinite Response Spaces

For a countably infinite response space, event probabilities are still obtained by summing masses. The difference is that a sum may have infinitely many terms. A valid infinite PMF must have partial sums that approach one when all the response values are included. Similarly, an event involving infinitely many values has probability given by the corresponding infinite series.

A basic way to verify an infinite total is to identify the partial sums and take their limit. The following example uses a telescoping series: most terms cancel when a finite number are added, leaving a simple expression whose limit can be checked.

Worked Example: A Telescoping PMF on the Positive Integers

For \(k=1,2,3,\ldots\), define \(p_X(k)=1/[k(k+1)]\). Verify that this is a PMF, then calculate \(P(X\geq 4)\) and \(P(X\in\{1,2,5\})\).

Each mass is positive because \(k\) and \(k+1\) are positive. To check the total, use

\frac{1}{k(k+1)}=\frac{1}{k}-\frac{1}{k+1}.

The partial sum through \(N\) is therefore

\sum_{k=1}^{N}p_X(k) =\sum_{k=1}^{N}\left(\frac{1}{k}-\frac{1}{k+1}\right) =1-\frac{1}{N+1}.

As \(N\) increases without bound, \(1/(N+1)\) tends to zero, so the partial sums tend to one. The masses are nonnegative and sum to one, as required.

For \(X\geq 4\), the same telescoping identity gives

P(X\geq 4) =\sum_{k=4}^{\infty}\left(\frac{1}{k}-\frac{1}{k+1}\right) =\lim_{N\to\infty}\left(\frac{1}{4}-\frac{1}{N+1}\right) =\frac{1}{4}.

For the finite event \(\{1,2,5\}\), add its three masses:

P(X\in\{1,2,5\}) =\frac{1}{1(2)}+\frac{1}{2(3)}+\frac{1}{5(6)} =\frac{1}{2}+\frac{1}{6}+\frac{1}{30} =\frac{15+5+1}{30} =\frac{7}{10}.

The event calculation uses only the values in the event, even though the full response space is infinite.

Constructing a PMF From Weights

Sometimes numbers are provided as relative weights rather than probabilities. Nonnegative weights become a PMF after each is divided by the total weight, provided that the total is positive and finite. For a countably infinite collection, the sum of the weights must be finite and positive.

Theorem: Let \(S\) be finite or countable, and let \(w(x)\geq 0\) for \(x\in S\). If \(0<W=\sum_{x\in S}w(x)<\infty\), then \(p_X(x)=w(x)/W\) is a PMF on \(S\).

Proof. Since \(W>0\), division by \(W\) is defined, and \(w(x)\geq 0\) implies \(p_X(x)\geq 0\) for every \(x\). Also,

\sum_{x\in S}p_X(x) =\sum_{x\in S}\frac{w(x)}{W} =\frac{1}{W}\sum_{x\in S}w(x) =\frac{W}{W} =1.

Thus the normalized weights satisfy both PMF conditions. \(\square\)

Worked Example: Normalizing Finite Weights

Suppose the response space is \(S=\{0,1,2,3\}\), and the corresponding weights are \(2,1,3,2\). Construct the PMF and calculate the probability that the response is even.

The total weight is

$$ W=2+1+3+2=8. $$

Dividing each weight by 8 gives \(p_X(0)=2/8=1/4\), \(p_X(1)=1/8\), \(p_X(2)=3/8\), and \(p_X(3)=2/8=1/4\). Each is nonnegative, and their sum is

\frac{2}{8}+\frac{1}{8}+\frac{3}{8}+\frac{2}{8} =\frac{8}{8} =1.

The even responses are 0 and 2, so their masses give

P(X\text{ is even})=p_X(0)+p_X(2)=\frac{2}{8}+\frac{3}{8}=\frac{5}{8}.

Using the weights directly would give \(2+3=5\), which is not a probability. Normalization is what converts relative weights into probabilities.

Common Mistakes and Why PMFs Matter

A PMF makes a discrete response model concrete: it states how much probability belongs to each valid value, and it lets us calculate the probability of any specified set of values. That makes it essential to distinguish the response space from the probability assignment. The space lists the values; the PMF assigns their masses.

Checking only the total. A collection of numbers that sums to one is not a PMF if any number is negative. Check nonnegativity and the total.
Adding the wrong values. For \(P(X\in A)\), include precisely the masses for values in \(A\), not masses for values outside it.
Stopping an infinite sum too soon. A finite partial sum is not automatically the total probability. For a countably infinite space, establish the limit of the partial sums.
Treating weights as probabilities. Weights need not sum to one. Divide each weight by the finite, positive total before interpreting it as a probability.

A frequent error is to say, “The masses are valid because they add to one,” without checking whether each mass is nonnegative. For example, the numbers \(-0.1,0.6,0.5\) sum to one, but the negative entry cannot be the probability of any response value. The correction is to verify both defining conditions separately: every mass must be at least zero, and the sum over the entire response space must equal one.

For a countably infinite space, there is an analogous pitfall: a list of nonnegative terms may have partial sums that never reach a finite total. Nonnegativity alone does not guarantee a valid PMF. The infinite series must sum to one; if it sums to a different finite positive number, its terms can be normalized, while an infinite total cannot be normalized by dividing by that total.

Check Your Understanding

Use the definitions and calculations in this tutorial to answer the following questions.

What two conditions must a collection of masses satisfy to be a PMF?
If \(p_X(0)=0.2\), \(p_X(1)=0.5\), and \(p_X(2)=0.3\), calculate \(P(X\in\{0,2\})\).
Why does calculating an event probability on a countably infinite response space require considering an infinite series?
Weights on a finite response space sum to 12. What operation turns them into a PMF?
Can nonnegative masses with total 0.8 form a PMF on the stated response space? Explain.