Normalizing Finite and Countable Mass Functions

Why Normalization Matters

A proposed formula for probabilities can look plausible without actually assigning probabilities. Some of its values might be negative, or the values might add to something other than one. For a discrete law, checking these two conditions—nonnegativity and total mass one—is the essential test.

The same test applies whether the possible values form a finite set or a countably infinite one. Infinite supports require a little more care: the total mass is an infinite series, defined as the limit of its partial sums. This tutorial develops the test and shows how to use it to find a normalizing constant.

The Validity Test

Definition: Let $S$ be a nonempty finite or countable subset of the real line, and let $f(x)$ be a proposed mass at each $x\in S$. The formula defines a probability mass function on $S$ if and only if $$ f(x)\geq 0\quad\text{for every }x\in S, \qquad \sum_{x\in S} f(x)=1. $$ The sum over a countable set is the sum of a series of nonnegative terms, defined as the limit of its finite partial sums. For $x\notin S$, the mass is zero.

These conditions are necessary because $f(x)$ represents $P(X=x)$: probabilities cannot be negative, and the events $X=x$, for $x\in S$, are disjoint and together have probability one. They are also sufficient. To see why, take $S$ as the sample space and assign probability $f(x)$ to each point $x$. For any subset $A\subseteq S$, define $P(A)=\sum_{x\in A} f(x)$. Nonnegativity gives $P(A)\geq 0$, the total-mass condition gives $P(S)=1$, and for any pairwise disjoint sets $A_1,A_2,\ldots$, the nonnegative terms can be grouped by those sets, giving $$ P\left(\bigcup_{j=1}^{\infty} A_j\right) =\sum_{x\in \bigcup_j A_j}f(x) =\sum_{j=1}^{\infty}\sum_{x\in A_j}f(x) =\sum_{j=1}^{\infty}P(A_j). $$ Thus $P$ is a probability measure, and the coordinate $X(x)=x$ has mass function $f$. $\square$

In practice, write down the support first: it tells you which values are included in the sum. Then check each mass for nonnegativity and evaluate the total. A formula that gives nonnegative values but sums to $0.8$, for instance, is not yet a probability mass function.

Finding a Normalizing Constant

Often a formula contains an unknown constant $c$. A common setup is to begin with nonnegative weights $w(x)$ and propose $f(x)=c\,w(x)$. The weights determine the relative sizes of the masses; $c$ adjusts their total.

Normalization rule: Suppose $w(x)\geq 0$ on a finite or countable support $S$, and let $W=\sum_{x\in S}w(x)$. If $0<W<\infty$, the unique constant that makes $f(x)=c\,w(x)$ sum to one is $c=1/W$. The resulting masses are nonnegative and sum to one. If $W=0$, no constant gives total mass one. If $W=\infty$, no finite constant $c$ gives total mass one.

For a finite support, this is ordinary division by the sum of the weights. For a countable support, the same rule applies when the series of weights converges to a finite, positive number. If the sum is infinite, taking $c=0$ gives total mass zero, while any positive $c$ leaves the total infinite; a negative $c$ also violates nonnegativity whenever some weight is positive.

Worked Example: Normalizing Weights on a Finite Support

Suppose $f(k)=ck$ for $k\in\{1,2,3,4\}$, and $f(k)=0$ elsewhere. Find $c$ and verify that the result is a probability mass function.

The weights on the support are $1,2,3,4$, all nonnegative. Their sum is $$ 1+2+3+4=10. $$ Thus the total mass is $10c$, so the condition that it equal one requires $$ 10c=1,\qquad c=\frac{1}{10}. $$ The four masses are therefore $1/10,2/10,3/10,4/10$. They are all nonnegative, and $$ \frac{1}{10}+\frac{2}{10}+\frac{3}{10}+\frac{4}{10} =\frac{10}{10}=1. $$ So this choice of $c$ gives a valid probability mass function. Any other value of $c$ fails the total-mass condition.

Countably Infinite Supports and Geometric Series

For a countably infinite support, total mass is computed using an infinite series. For example, on $S=\{0,1,2,\ldots\}$, the total of masses $c r^k$ is $c\sum_{k=0}^{\infty}r^k$, when the series converges. The geometric-series calculation follows from finite partial sums.

Geometric-series result: If $0\leq r<1$, then $$ \sum_{k=0}^{\infty}r^k=\frac{1}{1-r}. $$ Indeed, for every integer $m\geq 0$, multiplication and subtraction give $$ (1-r)\sum_{k=0}^{m}r^k =(1+r+\cdots+r^m)-(r+r^2+\cdots+r^{m+1}) =1-r^{m+1}. $$ Since $0\leq r<1$, $r^{m+1}\to 0$ as $m\to\infty$. Dividing by $1-r>0$ and taking the limit of the partial sums proves the result. $\square$

The restriction $r\geq 0$ matters for a probability mass function, not just for this calculation. A geometric series can converge for some negative values of $r$, but the terms then alternate in sign and cannot all be probabilities.

Worked Example: Normalizing a Geometric Sequence

Let $f(k)=c(1/2)^k$ for $k=0,1,2,\ldots$, and let $f(k)=0$ elsewhere. Find $c$ and verify the two validity conditions.

Every weight $(1/2)^k$ is positive. By the geometric-series result with $r=1/2$, $$ \sum_{k=0}^{\infty}\left(\frac12\right)^k =\frac{1}{1-\frac12}=2. $$ The total mass is therefore $2c$. Setting it equal to one gives $c=1/2$, so $$ f(k)=\frac12\left(\frac12\right)^k=\frac{1}{2^{k+1}},\qquad k=0,1,2,\ldots. $$ These values are nonnegative. Their sum is $$ \sum_{k=0}^{\infty}\frac{1}{2^{k+1}} =\frac12\sum_{k=0}^{\infty}\left(\frac12\right)^k =\frac12\cdot 2=1. $$ Both conditions hold, so the formula is a probability mass function.

A related normalization can be expressed with a parameter $r$: the masses $f(k)=(1-r)r^k$ on $k=0,1,2,\ldots$ are valid when $0\leq r<1$. In that range $1-r\geq 0$, each power $r^k$ is nonnegative, and the geometric-series calculation gives total mass $(1-r)/(1-r)=1$. The endpoint $r=0$ means mass one at $k=0$ and zero at every $k\geq 1$.

Normalizing a Telescoping Series

Not every useful countable sum is geometric. Some series are easier to evaluate after rewriting each term as a difference. In a telescoping sum, most terms cancel in the partial sums, leaving an expression whose limit can be checked directly.

Worked Example: Normalizing Telescoping Masses

On the positive integers $k=1,2,\ldots$, consider $f(k)=c/[k(k+1)]$. Find the normalizing constant and verify the total mass.

First decompose the weight: $$ \frac{1}{k(k+1)}=\frac{1}{k}-\frac{1}{k+1}, $$ because the right-hand side is $((k+1)-k)/[k(k+1)]$. The partial sum through $m$ is therefore $$ \sum_{k=1}^{m}\frac{1}{k(k+1)} =\sum_{k=1}^{m}\left(\frac{1}{k}-\frac{1}{k+1}\right) =1-\frac{1}{m+1}. $$ Taking $m\to\infty$ gives total weight $1$. Thus the total mass is $c$, and normalization requires $c=1$. For this value, every mass is positive and $$ \sum_{k=1}^{\infty}\frac{1}{k(k+1)} =\lim_{m\to\infty}\left(1-\frac{1}{m+1}\right)=1. $$ So $f(k)=1/[k(k+1)]$ is a probability mass function on the positive integers.

Convergence Does Not Guarantee Validity

A convergent sum is not automatically a probability mass function. The terms must also be nonnegative, and their sum must be exactly one. This distinction is particularly important for formulas with alternating signs: cancellation can produce a finite total even though some proposed “masses” are negative.

Worked Example: A Convergent Series With Negative Masses

Consider $f(k)=c(-1/2)^k$ on $k=0,1,2,\ldots$. Show why choosing $c$ to make the series sum to one does not make this a probability mass function.

The geometric-series calculation for the partial sums, or the finite geometric formula, gives $$ \sum_{k=0}^{\infty}\left(-\frac12\right)^k =\frac{1}{1-(-1/2)}=\frac{2}{3}. $$ Thus the total would equal one if $c(2/3)=1$, which requires $c=3/2$. But then $$ f(1)=\frac32\left(-\frac12\right)=-\frac34<0. $$ In fact, every odd $k$ gives a negative value. Although the series converges and its sum is one after this choice of $c$, nonnegativity fails. The formula is not a probability mass function.

A Reliable Checking Procedure

When a proposed discrete formula includes a constant or an infinite support, it helps to keep the validity checks separate. First determine where the formula is meant to assign positive or potentially nonzero mass. Then check the signs and calculate the total on precisely that support.

Identify the support.
Write down the possible values of $X$. Do not omit an endpoint or add values that the formula does not include.

Check nonnegativity.
Verify that each proposed mass is at least zero throughout the support, including any restrictions needed on parameters.

Sum the masses.
Use a finite sum for a finite support, or compute the limit of partial sums for a countably infinite support.

Conclude only if both tests pass.
A total other than one requires normalization if possible; a negative term rules out a probability mass function regardless of the total.

A frequent mistake is to find a constant that makes the algebraic sum equal one and then stop. That reasoning checks only the total. For example, the alternating geometric sequence above has a choice of constant that gives total one, but it assigns negative values at odd integers. The correction is to test nonnegativity before declaring the formula valid. Another common oversight is summing over the wrong support: a formula on $k=1,2,\ldots$ does not include $k=0$, and including that value changes the total.

Key takeaway: A discrete formula is a probability mass function exactly when every mass on its support is nonnegative and the masses sum to one. Normalization fixes the total only when the nonnegative weights have a finite, strictly positive sum.

Check Your Understanding

For each proposed formula, identify the support and decide what must be checked before it can be called a probability mass function.

On $\{0,1,2\}$, let $f(k)=c(k+1)$. Find the normalizing constant and verify both conditions.
On $k=0,1,2,\ldots$, let $f(k)=c r^k$. For what values of $r$ can a positive $c$ produce a probability mass function?
Explain why a series of proposed masses that converges to one can still fail to define a probability mass function.
For weights $w(k)=1/[k(k+1)]$ on $k=1,2,\ldots$, write the partial sum through $m$ and its limit.
If nonnegative weights on a countable support have infinite sum, why can no finite constant normalize them to total mass one?