Bayes' Theorem From a Partition

Why Bayes’ Theorem Uses a Partition

Suppose an observed event could have come from several different sources. The probability of the observation depends on both how likely each source was beforehand and how likely that source is to produce the observation. Bayes’ theorem combines these two pieces of information to answer the reverse question: given the observation, how probable is each source?

This reverse calculation is useful whenever the source categories divide all possible cases into nonoverlapping groups. For example, a randomly selected item might come from one of several production lines, or a randomly selected claim might belong to one of several risk categories. In each case, the observed evidence updates the probabilities assigned to the categories.

Definition: Events $A_1,A_2,\ldots,A_n$ form a partition of the sample space if they are mutually exclusive and exhaustive: no two can occur together, and exactly one of them occurs. In probability notation, $A_i\cap A_j=\varnothing$ for $i\ne j$, and $A_1\cup\cdots\cup A_n$ is the entire sample space. For Bayes’ theorem, the partition events used in the denominator must have positive probabilities, and the observed event $B$ must have $P(B)>0$.

The events $A_1,\ldots,A_n$ describe the possible categories before observing $B$. Their probabilities $P(A_i)$ are called the prior probabilities. The conditional probabilities $P(B\mid A_i)$ describe how likely the evidence $B$ is within each category. Once $B$ is known to have occurred, $P(A_i\mid B)$ is the posterior probability of category $A_i$.

The law of total probability first combines the categories to find the overall probability of $B$. Bayes’ theorem then uses that overall probability to calculate the posterior probability for a particular category.

Theorem: If $A_1,\ldots,A_n$ partition the sample space, $P(A_i)>0$, and $P(B)>0$, then $$ P(B)=\sum_{j=1}^{n}P(B\mid A_j)P(A_j) $$ and, for each $i$, $$ P(A_i\mid B)= \frac{P(B\mid A_i)P(A_i)} {\displaystyle\sum_{j=1}^{n}P(B\mid A_j)P(A_j)}. $$

Deriving Bayes’ Theorem

The denominator is not an arbitrary adjustment. It is the probability of the observed event $B$, calculated by considering every partition category that could have produced it.

Split the evidence across the partition.
Because the $A_j$ are exhaustive and mutually exclusive, the events $B\cap A_1,\ldots,B\cap A_n$ are also mutually exclusive and their union is $B$. Therefore, $P(B)=\sum_{j=1}^{n}P(B\cap A_j)$.

Apply the multiplication rule.
For each $j$, $P(B\cap A_j)=P(B\mid A_j)P(A_j)$. Substitution gives $P(B)=\sum_{j=1}^{n}P(B\mid A_j)P(A_j)$.

Use conditional probability for the desired category.
Since $P(B)>0$, $P(A_i\mid B)=P(A_i\cap B)/P(B)$. The multiplication rule gives $P(A_i\cap B)=P(B\mid A_i)P(A_i)$; substituting the total-probability expression for $P(B)$ yields Bayes’ theorem.

The numerator is the joint probability $P(A_i\cap B)$: the probability that category $A_i$ applies and the evidence occurs. The denominator is the probability of all ways the evidence could occur. Thus, Bayes’ theorem divides the probability contributed by the category of interest by the total probability of the evidence.

Worked Applications

Worked Example: Finding the Source of a Defective Item

A hypothetical warehouse receives components from three suppliers. A randomly selected component came from supplier A with probability $0.50$, supplier B with probability $0.30$, or supplier C with probability $0.20$. These supplier events form a partition. The conditional probabilities that a component is defective, given its supplier, are $0.01$, $0.02$, and $0.05$, respectively. Given that a selected component is defective, find the probability it came from supplier C. Also find the probability it came from supplier A.

Method and assumptions. Let $D$ be the event that the component is defective, and let $A,B,C$ denote its supplier. The supplier probabilities sum to $0.50+0.30+0.20=1.00$, and exactly one supplier applies. Use the law of total probability to find $P(D)$, then apply Bayes’ theorem.

Step 1: Find the total probability of a defect. The three supplier-specific contributions are

\begin{aligned} P(D\cap A)&=P(D\mid A)P(A)=0.01(0.50)=0.005,\\ P(D\cap B)&=P(D\mid B)P(B)=0.02(0.30)=0.006,\\ P(D\cap C)&=P(D\mid C)P(C)=0.05(0.20)=0.010. \end{aligned}

Adding these disjoint cases gives $P(D)=0.005+0.006+0.010=0.021$.

Step 2: Find the posterior probabilities. For supplier C,

P(C\mid D)=\frac{P(D\mid C)P(C)}{P(D)} =\frac{0.05(0.20)}{0.021} =\frac{0.010}{0.021} \approx 0.4762.

For supplier A,

P(A\mid D)=\frac{0.01(0.50)}{0.021} =\frac{0.005}{0.021} \approx 0.2381.

Check. The remaining posterior is $P(B\mid D)=0.006/0.021\approx0.2857$. The three values sum to $0.4762+0.2381+0.2857=1.0000$, up to rounding. The calculation avoids the mistake of reporting supplier C’s original $0.20$ as its probability after learning that the component is defective.

Worked Example: Which Shipping Route Produced a Delay?

A hypothetical distributor uses three shipping routes. A randomly selected parcel uses route 1 with probability $0.40$, route 2 with probability $0.35$, and route 3 with probability $0.25$. The probabilities of a delay conditional on the route are $0.05$, $0.08$, and $0.20$, respectively. Given that a parcel was delayed, calculate the probability it used route 2 and the probability it used route 3.

Method and assumptions. Let $R_j$ mean that the parcel used route $j$, and let $L$ mean that it was delayed. The routes form a partition because every parcel uses exactly one of them, and $0.40+0.35+0.25=1$. First calculate the total delay probability; a route’s delay probability alone is not the denominator.

Step 1: Calculate the overall delay probability.

\begin{aligned} P(L)&=P(L\mid R_1)P(R_1)+P(L\mid R_2)P(R_2)+P(L\mid R_3)P(R_3)\\ &=0.05(0.40)+0.08(0.35)+0.20(0.25)\\ &=0.020+0.028+0.050\\ &=0.098. \end{aligned}

Step 2: Calculate the requested posterior probabilities.

P(R_2\mid L)=\frac{0.08(0.35)}{0.098} =\frac{0.028}{0.098} \approx0.2857, \qquad P(R_3\mid L)=\frac{0.20(0.25)}{0.098} =\frac{0.050}{0.098} \approx0.5102.

Check. The posterior probability for route 1 is $0.020/0.098\approx0.2041$. The three route probabilities after observing a delay sum to $0.2041+0.2857+0.5102=1.0000$. Route 3 accounts for one quarter of parcels before observing a delay but just over half of delayed parcels, because its conditional delay probability is comparatively high.

Common mistake avoided. Dividing $0.028$ by $0.08$ would divide the route 2 joint contribution by its conditional delay probability, not by the probability of observing a delay. The correct denominator is $P(L)=0.098$, which includes contributions from all three routes.

Worked Example: Updating the Probability of a High-Risk Category

For a hypothetical book of policies, a randomly selected policy is classified as low, medium, or high risk with probabilities $0.60$, $0.30$, and $0.10$. For the next year, the probabilities of at least one claim, conditional on the risk category, are $0.02$, $0.08$, and $0.25$. Find the probability that a policy is high risk given that it has at least one claim. Then find the probability that it is high risk given that it has no claim.

Method and assumptions. Let $H,M,L$ denote the high-, medium-, and low-risk events, and let $C$ denote at least one claim during the stated year. The categories partition the policies, and the given claim probabilities apply conditional on category. The no-claim event is $C^c$, with $P(C^c\mid H)=1-P(C\mid H)$, and likewise for the other categories.

Step 1: Find the probability of a claim.

\begin{aligned} P(C)&=0.02(0.60)+0.08(0.30)+0.25(0.10)\\ &=0.012+0.024+0.025\\ &=0.061. \end{aligned}

Step 2: Condition on a claim. The high-risk contribution to the claim probability is $P(C\cap H)=0.25(0.10)=0.025$. Therefore,

P(H\mid C)=\frac{0.025}{0.061}\approx0.4098.

Step 3: Find the probability of no claim.

$$ P(C^c)=1-P(C)=1-0.061=0.939. $$

Step 4: Condition on no claim. The high-risk no-claim contribution is $P(C^c\cap H)=P(C^c\mid H)P(H)=(1-0.25)(0.10)=0.075$. Thus,

P(H\mid C^c)=\frac{0.075}{0.939}\approx0.0799.

Check. For a claim, the three category contributions are $0.012,0.024,0.025$; their sum is $0.061$. For no claim, they are $0.60(0.98)=0.588$, $0.30(0.92)=0.276$, and $0.10(0.75)=0.075$; their sum is $0.939$. The result is plausible: claims make the high-risk category more probable than its prior probability $0.10$, while no claim makes it less probable. The calculation avoids using $P(C\mid H)$ as the posterior; a conditional claim rate within a category is not the probability of that category given a claim.

Interpreting and Checking the Result

Bayes’ theorem changes the probabilities assigned to the partition categories after an event is observed. A category’s posterior probability increases relative to its prior probability when its evidence contribution is relatively large. That contribution is the product $P(B\mid A_i)P(A_i)$, not merely the category’s conditional event probability. A category may have a high conditional rate but still contribute little overall if its prior probability is very small.

A useful calculation table keeps the roles of the quantities separate:

Quantity	Meaning	Role in the calculation
$P(A_i)$	Probability of category $A_i$ before observing $B$	Prior probability
$P(B\mid A_i)$	Probability of evidence $B$ within category $A_i$	Conditional evidence rate
$P(B\mid A_i)P(A_i)$	Probability of category $A_i$ and evidence $B$	Numerator contribution
$\sum_j P(B\mid A_j)P(A_j)$	Overall probability of evidence $B$	Bayes’ theorem denominator
$P(A_i\mid B)$	Probability of category $A_i$ after observing $B$	Posterior probability

There are two important checks. First, the denominator should be calculated using every category in the partition, so it equals $P(B)$ and is positive. Second, the posterior probabilities across all partition categories must sum to one:

\sum_{i=1}^{n}P(A_i\mid B) = \frac{\sum_{i=1}^{n}P(B\mid A_i)P(A_i)} {\sum_{j=1}^{n}P(B\mid A_j)P(A_j)} =1.

The theorem does not apply in this form if the proposed categories overlap or leave possible cases out: then their contributions do not correctly decompose the evidence event. Nor can $P(A_i\mid B)$ be defined by the conditional-probability formula when $P(B)=0$. For a valid partition and an evidence event with positive probability, the key discipline is to distinguish the direction of conditioning: $P(B\mid A_i)$ is given information about evidence within a category, whereas $P(A_i\mid B)$ is the posterior quantity being sought.

Check Your Understanding

For each problem, identify the partition category of interest, calculate the total probability of the evidence, and then apply Bayes’ theorem.

A hypothetical parts depot receives components from source X with probability $0.70$ and source Y with probability $0.30$. The probabilities of a flaw given the source are $0.04$ and $0.10$, respectively. Given a flaw, what is the probability the component came from Y? (Answer: $0.5172$)
A parcel uses route A, B, or C with probabilities $0.50, 0.30,$ and $0.20$. The conditional probabilities of a delay are $0.02, 0.06,$ and $0.15$. Given a delay, what is the probability the parcel used route C? (Answer: $0.5172$)
A hypothetical group of accounts is divided into categories 1, 2, and 3 with probabilities $0.20, 0.50,$ and $0.30$. The probabilities of an alert in those categories are $0.01, 0.04,$ and $0.12$. Given an alert, what is the probability the account is in category 3? (Answer: $0.6207$)
Three production lines account for $0.25, 0.45,$ and $0.30$ of output. Their conditional probabilities of producing an item that passes inspection are $0.90, 0.60,$ and $0.20$. Given that an item passes, what is the probability it came from line 1? (Answer: $0.4054$)
A hypothetical policy portfolio has low-, medium-, and high-risk categories with probabilities $0.40, 0.40,$ and $0.20$. Their probabilities of a claim during a specified year are $0.10, 0.30,$ and $0.80$. Given no claim during the year, what is the probability the policy is high risk? (Answer: $0.0588$)

Quantity	Meaning	Role in the calculation
\(P(A_i)\)	Probability of category \(A_i\) before observing \(B\)	Prior probability
\(P(B\mid A_i)\)	Probability of evidence \(B\) within category \(A_i\)	Conditional evidence rate
\(P(B\mid A_i)P(A_i)\)	Probability of category \(A_i\) and evidence \(B\)	Numerator contribution
\(\sum_j P(B\mid A_j)P(A_j)\)	Overall probability of evidence \(B\)	Bayes’ theorem denominator
\(P(A_i\mid B)\)	Probability of category \(A_i\) after observing \(B\)	Posterior probability

What You'll Learn