Normalizing Weights Into a Probability Law

From Weights to Probabilities

A model may specify that some outcomes are more strongly weighted than others without assigning probabilities directly. For example, weights $1,2,3$ express relative proportions: the third outcome is assigned three times the weight of the first. They are not yet probabilities, because probabilities must total one. Normalization converts these relative weights into a probability law, provided their total is finite and strictly positive.

The same idea works on a finite or countably infinite sample space. The key check is not whether each individual weight is finite, but whether the sum of all weights is finite and positive. A countable collection of finite weights can still have an infinite total.

Definition: Let $\Omega$ be a nonempty finite or countably infinite sample space, and assign a finite nonnegative weight $w(\omega)$ to each $\omega\in\Omega$. The normalizing constant is $$ Z=\sum_{\omega\in\Omega} w(\omega). $$ If $0<Z<\infty$, the normalized point probabilities are $$ p(\omega)=\frac{w(\omega)}{Z}, \qquad \omega\in\Omega. $$ For a countably infinite $\Omega$, the sum of nonnegative weights is defined as the limit of its finite partial sums, or equivalently as the supremum of the sums over finite subsets of $\Omega$.

The condition $0<Z<\infty$ has two separate parts. If $Z=0$, every nonnegative weight must be zero, so there is nothing to scale into total probability one. If $Z=\infty$, dividing each weight by $Z$ is not division by a finite normalizing constant and does not produce the desired probability assignment. When $0<Z<\infty$, each normalized value is nonnegative and the values sum to one.

Constructing Probabilities on Events

Once the point probabilities have been formed, the probability of an event $A\subseteq\Omega$ is the sum of the probabilities of its outcomes. This is the countable-space form of assigning probability from the point masses.

Theorem: Suppose $w(\omega)\geq 0$ for all $\omega\in\Omega$, and $0<Z=\sum_{\omega\in\Omega}w(\omega)<\infty$. Define $$ P(A)=\sum_{\omega\in A}\frac{w(\omega)}{Z}, \qquad A\subseteq\Omega. $$ Then $P$ is a probability law on the subsets of $\Omega$: $P(A)\geq 0$, $P(\Omega)=1$, and for every sequence of pairwise disjoint events $A_1,A_2,\ldots$, $$ P\left(\bigcup_{j=1}^{\infty}A_j\right)=\sum_{j=1}^{\infty}P(A_j). $$

Proof. Nonnegativity follows because every weight and $Z$ are nonnegative, with $Z>0$. Also, $$ P(\Omega)=\sum_{\omega\in\Omega}\frac{w(\omega)}{Z} =\frac{1}{Z}\sum_{\omega\in\Omega}w(\omega)=1. $$ For disjoint $A_j$, each outcome in their union belongs to exactly one of the events. Thus the sum of its normalized weights over the union equals the sum of those weights grouped by event: $$ \sum_{\omega\in\bigcup_j A_j}\frac{w(\omega)}{Z} =\sum_{j=1}^{\infty}\sum_{\omega\in A_j}\frac{w(\omega)}{Z}. $$ This regrouping is valid for nonnegative series. In detail, any finite subset of the union meets only finitely many of the disjoint sets and therefore has total weight at most the sum of the event totals. Conversely, finite subsets chosen from finitely many of the events form a finite subset of the union. Taking suprema over such finite subsets gives equality in both directions. The displayed identity is countable additivity, so $P$ is a probability law. $\square$

Equivalently, an event probability is its total weight divided by the total weight: $$ P(A)=\frac{\sum_{\omega\in A}w(\omega)}{\sum_{\omega\in\Omega}w(\omega)}. $$ This ratio is often the most direct calculation. A second useful fact is that multiplying every weight by the same positive finite constant does not change the law: both numerator and denominator are multiplied by that constant.

Worked Example: Normalizing Weights on a Finite Space

Let $\Omega=\{a,b,c,d\}$ have weights $w(a)=1$, $w(b)=2$, $w(c)=0$, and $w(d)=3$. Find the normalized probabilities and the probability of $A=\{b,d\}$.

The total weight is $$ Z=1+2+0+3=6. $$ Since $0<6<\infty$, normalization is valid. Dividing each weight by 6 gives $$ P(\{a\})=\frac16,\quad P(\{b\})=\frac26=\frac13,\quad P(\{c\})=0,\quad P(\{d\})=\frac36=\frac12. $$ Their sum is $1/6+1/3+0+1/2=1$, as required.

The event $A$ contains $b$ and $d$, so $$ P(A)=P(\{b\})+P(\{d\})=\frac13+\frac12=\frac56. $$ The zero weight on $c$ remains a zero probability after normalization; normalization does not require every outcome to have positive weight.

Checking a Countable Normalizing Sum

For an infinite sample space, the central work is evaluating or bounding the series $Z$. Familiar series tests and identities can settle this question. A geometric series with ratio of absolute value less than one has a finite total; a telescoping series may also have a finite total even when its individual terms do not look geometric. By contrast, a sequence of weights that decreases toward zero need not have a finite sum.

Worked Example: Geometric Weights on the Positive Integers

Take $\Omega=\{1,2,3,\ldots\}$ and $w(n)=(2/3)^{n-1}$. Find the normalizing constant, the probability of each integer, and the probability that the outcome is even.

The weights form a geometric series with first term 1 and ratio $2/3$. Because $0\leq 2/3<1$, $$ Z=\sum_{n=1}^{\infty}\left(\frac23\right)^{n-1} =\frac{1}{1-2/3}=3. $$ The normalized probability at $n$ is therefore $$ P(\{n\})=\frac13\left(\frac23\right)^{n-1}. $$ For the event of even integers, write $n=2,4,6,\ldots$. Its total weight is $$ \sum_{k=1}^{\infty}\left(\frac23\right)^{2k-1} =\frac23\sum_{k=0}^{\infty}\left(\frac49\right)^k =\frac23\cdot\frac{1}{1-4/9} =\frac65. $$ Dividing by $Z=3$ gives $$ P(\text{even integers})=\frac{6/5}{3}=\frac25. $$ The complement consists of odd integers and has probability $1-2/5=3/5$, which also checks the result.

Worked Example: Telescoping Weights

Let $\Omega=\{1,2,3,\ldots\}$ and $w(n)=1/[n(n+1)]$. Determine whether these weights can be normalized, and find the probability that the outcome is at least 3.

First decompose each weight: $$ \frac{1}{n(n+1)}=\frac1n-\frac{1}{n+1}. $$ The partial sum through $N$ is $$ \sum_{n=1}^{N}\left(\frac1n-\frac{1}{n+1}\right) =1-\frac{1}{N+1}. $$ As $N$ tends to infinity, this tends to 1. Hence $Z=1$, which is finite and positive, and the weights already sum to one. Thus $P(\{n\})=1/[n(n+1)]$.

For the event $n\geq 3$, the partial sum from 3 through $N$ is $$ \sum_{n=3}^{N}\left(\frac1n-\frac{1}{n+1}\right) =\frac13-\frac{1}{N+1}. $$ Taking the limit gives $P(n\geq3)=1/3$. As a check, the first two probabilities are $1/2$ and $1/6$, whose sum is $2/3$; the remaining probability is $1-2/3=1/3$.

When Normalization Fails

The positivity and finiteness checks cannot be skipped. Finite weights do not guarantee a finite total, and a finite total does not guarantee a positive one. In either failure case there is no probability law obtained by dividing all the given weights by their total.

Worked Example: Harmonic Weights Have an Infinite Total

Let $\Omega=\{1,2,3,\ldots\}$ and $w(n)=1/n$. Each individual weight is positive and finite, but the total diverges. To verify divergence, group terms in blocks: $$ \sum_{n=1}^{\infty}\frac1n =1+\frac12+\left(\frac13+\frac14\right) +\left(\frac15+\frac16+\frac17+\frac18\right)+\cdots. $$ In the block from $2^{k-1}+1$ through $2^k$, there are $2^{k-1}$ terms, each at least $1/2^k$. The block therefore has sum at least $$ 2^{k-1}\cdot\frac{1}{2^k}=\frac12. $$ There are infinitely many such blocks, so the partial sums grow without bound and $Z=\infty$. The weights cannot be normalized by their total. In particular, it would be incorrect to treat $1/n$ as a probability at outcome $n$: these values do not sum to one.

At the other extreme, if every weight is zero, then $Z=0$. For example, weights $w(a)=w(b)=0$ on $\{a,b\}$ cannot be normalized: the expression $w(\omega)/Z$ would be undefined, and no positive scaling of these weights can make their total one.

Why the Normalizing Constant Matters

The normalizing constant separates relative weighting from probability. The weights describe relative size, while $Z$ supplies the scale needed for total probability one. If $c>0$ is finite and the weights are replaced by $cw(\omega)$, then $$ \frac{cw(\omega)}{\sum_{\omega'\in\Omega}cw(\omega')} =\frac{cw(\omega)}{cZ} =\frac{w(\omega)}{Z}. $$ Thus the choice of common units for the weights is immaterial.

A common mistake is to reason that because every weight is finite, their total must be finite. The harmonic example shows why that is false: infinitely many positive terms can accumulate to an infinite sum. The correction is to evaluate the series or establish a valid upper bound for it before normalizing. Likewise, a proposed finite sum of zero is not usable merely because it is finite; the strict positivity condition matters too.

Key takeaway: Given nonnegative weights on a nonempty finite or countable outcome space, first calculate $Z=\sum_{\omega\in\Omega}w(\omega)$. Normalize by $p(\omega)=w(\omega)/Z$ only when $0<Z<\infty$; then event probabilities are sums of the normalized point probabilities.

Check Your Understanding

Try these questions using the definition and examples above.

On $\Omega=\{a,b,c\}$, the weights are $4,0,2$. Find $Z$ and all three normalized point probabilities.
Explain why the weights $w(n)=1/n$ cannot be normalized, even though each individual weight is finite.
For weights $w(n)=r^{n-1}$ on the positive integers, what condition on $r$ makes the geometric total finite and positive?
Suppose $0<Z<\infty$, and all weights are multiplied by 7. Explain why the resulting probability law is unchanged.
For weights $w(n)=1/[n(n+1)]$, use the telescoping form to find the probability of $n\geq 5$.