Deriving Bayes' Rule from a Probability Tree

Why Derive Bayes’ Rule From a Tree?

A probability tree organizes a sequence of choices or outcomes into branches. The probability of following a complete path is found by multiplying the probabilities along that path. When several paths can lead to the same observed outcome, their probabilities must be added. Those two operations—multiplication along paths and addition across paths—provide a direct route to Bayes’ rule.

Bayes’ rule connects a case or condition to an observed event. Its derivation matters because it shows where every part of the formula comes from: the numerator is the probability of one particular path, and the denominator is the probability of all paths that produce the observation. This also makes it easier to see which conditional probability is being calculated and which probability belongs in the denominator.

Events and Tree Probabilities

Suppose $A_1,\ldots,A_k$ are mutually exclusive events whose union is the entire sample space. In other words, exactly one of these cases occurs. Such a collection is called a partition. Let $B$ be an event that can occur within any of the cases. In a tree, the first set of branches represents the $A_i$, and the next branch records whether $B$ occurs.

Definition: For events $A$ and $B$ with $P(A)>0$, the conditional probability of $B$ given $A$ is $P(B\mid A)=P(A\cap B)/P(A)$. Rearranging gives the multiplication rule $P(A\cap B)=P(A)P(B\mid A)$. In a probability tree, this multiplication rule gives the probability of the path on which $A$ occurs and then $B$ occurs.

The condition $P(A)>0$ is essential: a conditional probability given an event of probability zero is not defined by this ratio. For a tree that uses $P(B\mid A_i)$ on every branch, each partition event must therefore have positive probability. If a case has probability zero, omit it from the tree and from formulas that condition on that case.

The law of total probability combines the paths that lead to $B$. Since the $A_i$ partition the sample space, the events $A_i\cap B$ are disjoint and together make up $B$. Thus the probability of $B$ is the sum of the probabilities of those paths.

P(B)=\sum_{i=1}^{k}P(A_i\cap B) =\sum_{i=1}^{k}P(A_i)P(B\mid A_i).

This result also explains the denominator in Bayes’ rule: it is the overall probability of the observed event $B$, found by adding the probabilities of all the paths that end in $B$.

Deriving Bayes’ Rule

To find the conditional probability of a particular case $A_j$ given that $B$ occurred, start with the definition of conditional probability. Because the probability of the joint event $A_j\cap B$ is the product of the first branch and the next branch, the numerator can be written using the tree. Then substitute the law of total probability for $P(B)$.

P(A_j\mid B) =\frac{P(A_j\cap B)}{P(B)} =\frac{P(A_j)P(B\mid A_j)} {\sum_{i=1}^{k}P(A_i)P(B\mid A_i)}.

Theorem: If $A_1,\ldots,A_k$ form a partition with $P(A_i)>0$ for every $i$, and $P(B)>0$, then $$ P(A_j\mid B) =\frac{P(A_j)P(B\mid A_j)} {\sum_{i=1}^{k}P(A_i)P(B\mid A_i)} $$ for each case $A_j$. The numerator is the probability of the path through $A_j$ and then $B$. The denominator is the sum of the probabilities of every path that ends at $B$.

The conditions matter. The positive probability of each $A_i$ ensures that every conditional branch probability $P(B\mid A_i)$ in the formula is defined. The condition $P(B)>0$ ensures that the requested conditional probability $P(A_j\mid B)$ is defined. Since $P(B)=\sum_i P(A_i)P(B\mid A_i)$, the denominator is positive whenever the evidence event has positive probability.

The derivation does not assume that $A_j$ and $B$ are independent. In fact, if they were independent, then $P(B\mid A_j)=P(B)$, and the conditional probability $P(A_j\mid B)$ would equal $P(A_j)$. In general, the branch probabilities can differ from case to case, so observing $B$ can change the relative probabilities of the cases.

Worked Example: Late Deliveries From Two Routes

Hypothetical data. A carrier uses Route 1 for 60% of deliveries and Route 2 for 40%. Among deliveries on Route 1, 5% arrive late; among those on Route 2, 20% arrive late. Given that a delivery arrived late, calculate the probability that it used Route 2.

Let $A_1$ be the event that Route 1 was used, $A_2$ the event that Route 2 was used, and $B$ the event that the delivery was late. The routes form a partition: one of them is used for each delivery, and $P(A_1)=0.60$ and $P(A_2)=0.40$ are both positive. The conditional late-arrival probabilities are $P(B\mid A_1)=0.05$ and $P(B\mid A_2)=0.20$.

The Route 1 path ending in a late delivery has probability $0.60(0.05)=0.030$. The corresponding Route 2 path has probability $0.40(0.20)=0.080$. Add these disjoint paths to find the probability of a late delivery:

$$ P(B)=0.60(0.05)+0.40(0.20)=0.030+0.080=0.110. $$

Because $P(B)=0.110>0$, conditioning on a late delivery is defined. Bayes’ rule gives

P(A_2\mid B) =\frac{P(A_2)P(B\mid A_2)}{P(B)} =\frac{0.40(0.20)}{0.110} =\frac{0.080}{0.110} \approx 0.7273.

Thus, among deliveries that arrived late, the probability of Route 2 is approximately 0.727, or 72.7%. The denominator includes late deliveries from both routes, not just the Route 2 path.

Reading the Formula as a Tree Calculation

The general formula can be read in three parts. First, $P(A_j)$ is the probability of entering case $A_j$. Second, $P(B\mid A_j)$ is the probability of taking the branch to $B$ from that case. Their product is the path probability $P(A_j\cap B)$. Finally, the denominator adds the path probabilities for all cases that lead to $B$, giving $P(B)$. The ratio is therefore the share of the evidence probability contributed by the path through $A_j$.

This reading helps distinguish two different questions. $P(B\mid A_j)$ starts by restricting attention to case $A_j$, then asks how often $B$ occurs. By contrast, $P(A_j\mid B)$ starts with outcomes where $B$ occurred and asks what fraction came from case $A_j$. The two quantities have different denominators and are not interchangeable.

Worked Example: Defects From Three Production Lines

Hypothetical data. A factory makes 50% of its items on Line 1, 30% on Line 2, and 20% on Line 3. The respective probabilities that an item is defective are 0.010, 0.020, and 0.050. Given that an item is defective, calculate the probability it came from Line 3.

Let $A_1,A_2,A_3$ denote the events that the item came from Lines 1, 2, and 3, and let $B$ denote the event that it is defective. The lines form a partition, and their probabilities $0.50,0.30,0.20$ are positive and sum to 1. Each conditional defect probability is between 0 and 1, as required.

The branch probabilities for defects from the three lines are:

P(A_1\cap B)=0.50(0.010)=0.005, \qquad P(A_2\cap B)=0.30(0.020)=0.006,

P(A_3\cap B)=0.20(0.050)=0.010.

The overall probability that an item is defective is the sum of these disjoint path probabilities:

$$ P(B)=0.005+0.006+0.010=0.021. $$

This is positive, so the requested conditional probability is defined. Applying Bayes’ rule,

P(A_3\mid B) =\frac{0.20(0.050)} {0.50(0.010)+0.30(0.020)+0.20(0.050)} =\frac{0.010}{0.021} \approx 0.4762.

Approximately 47.6% of defective items came from Line 3. Although Line 3 made only 20% of all items, its higher defect probability means that it contributes a larger share of the defective-item paths. This is a probability calculation, not evidence that the line itself caused a particular defect.

Why the Denominator Includes Every Case

A frequent error is to divide the target path by a conditional probability from that same branch. But after observing $B$, the relevant comparison set is all outcomes in which $B$ occurred. The denominator must therefore include the joint probability of $B$ within every partition case. The following example makes that accounting explicit.

Worked Example: A Response From One of Two Groups

Hypothetical data. At an event, 70% of attendees are in Group 1 and 30% are in Group 2. A randomly selected attendee responds to a follow-up message with probability 0.40 in Group 1 and 0.60 in Group 2. Given a response, calculate the probability that the attendee was in Group 1.

Let $A_1$ and $A_2$ indicate the attendee’s group, and let $B$ denote responding. The groups partition the attendees; both group probabilities are positive and sum to 1. The group-specific response probabilities are valid conditional branch probabilities. The two path probabilities ending in a response are

P(A_1\cap B)=0.70(0.40)=0.28, \qquad P(A_2\cap B)=0.30(0.60)=0.18.

Therefore, the total probability of a response is $P(B)=0.28+0.18=0.46$, which is positive. The probability that a responder was in Group 1 is

P(A_1\mid B) =\frac{0.70(0.40)}{0.70(0.40)+0.30(0.60)} =\frac{0.28}{0.46} \approx 0.6087.

So about 60.9% of responders were in Group 1. The denominator is 0.46, the probability of a response from either group. Dividing $0.28$ by $0.40$, for example, would use the conditional response probability within Group 1 rather than the overall evidence probability, and would not calculate $P(A_1\mid B)$.

Common Mistakes and a Useful Check

Using only one branch in the denominator. The denominator is $P(B)$, the sum of all path probabilities ending in $B$, not simply $P(B\mid A_j)$ or $P(A_j)$.
Reversing the conditional probabilities without changing the denominator. $P(B\mid A_j)$ and $P(A_j\mid B)$ answer different questions. Use the joint path probability divided by the total probability of the observed event.
Conditioning on a zero-probability case. If $P(A_i)=0$, $P(B\mid A_i)$ is undefined. Do not include that conditional term in the tree formula; omit the zero-probability case.
Forgetting to check the evidence probability. The requested $P(A_j\mid B)$ is defined only when $P(B)>0$.

A quick check is that each path probability is nonnegative and the denominator equals the sum of all paths ending at the evidence event. Also, if Bayes’ rule is applied to every case $A_j$, the resulting conditional probabilities must sum to 1, because the partition still covers all possibilities after conditioning on $B$.

\sum_{j=1}^{k}P(A_j\mid B) =\frac{\sum_{j=1}^{k}P(A_j)P(B\mid A_j)} {\sum_{i=1}^{k}P(A_i)P(B\mid A_i)} =1.

This identity follows because the numerator’s sum is exactly the denominator. It is a practical way to detect a missing case or an arithmetic error.

Key takeaway: A probability tree gives the joint probability $P(A_j\cap B)$ by multiplying along a path and gives $P(B)$ by adding all paths that end in $B$. Dividing the selected path probability by that total yields Bayes’ rule, provided the conditional probabilities used in the tree are defined and $P(B)>0$.

Check Your Understanding

Use a probability tree or the formulas in this tutorial to answer the following questions.

In Bayes’ rule, what event does the denominator represent, and why can it be written as a sum over partition cases?
A partition has cases $A_1$ and $A_2$, with $P(A_1)=0.25$, $P(A_2)=0.75$, $P(B\mid A_1)=0.20$, and $P(B\mid A_2)=0.04$. Find $P(B)$.
For the probabilities in Question 2, calculate $P(A_1\mid B)$. Show the joint path probability and the denominator.
Why must a case with $P(A_i)=0$ be omitted from a formula that includes $P(B\mid A_i)$?
Suppose $P(A_1\mid B)=0.35$ and $P(A_2\mid B)=0.65$ for a two-case partition. What useful check does their sum provide?