Sequential Bayesian Updating With Multiple Evidence

Updating Beliefs as Evidence Arrives

Bayes’ theorem can be applied more than once. After observing one piece of evidence, the resulting posterior probability becomes the prior probability for assessing the next observation. The key is to use the probability of the new evidence conditional on the evidence already observed. Replacing that probability with a marginal or unconditional likelihood can give the wrong answer.

This process is useful whenever information arrives in stages. An actuary might assess competing explanations for a claim after reviewing an initial report, then revise the assessment after an inspection. The calculations below use hypothetical scenarios; the probability values are assumptions for the examples, not industry data.

Definition: Let \(H\) be a hypothesis with prior probability \(P(H)\), and let \(E_1,\ldots,E_k\) be observed evidence with \(P(E_1\cap\cdots\cap E_k)>0\). Sequential Bayesian updating calculates \(P(H\mid E_1,\ldots,E_k)\) by updating after each observation. For each new observation, the likelihood is conditional on the evidence already observed.

The Sequential Updating Rule

After the first observation \(E_1\), Bayes’ theorem gives \(P(H\mid E_1)=P(H)P(E_1\mid H)/P(E_1)\). For the next observation, apply Bayes’ theorem again, now treating \(P(H\mid E_1)\) as the updated prior. The second likelihood must be \(P(E_2\mid H,E_1)\), not automatically \(P(E_2\mid H)\).

P(H\mid E_1,E_2) = \frac{P(H\mid E_1)P(E_2\mid H,E_1)} {P(E_2\mid E_1)}

The denominator can be calculated by partitioning on whether \(H\) is true: \(P(E_2\mid E_1)=P(H\mid E_1)P(E_2\mid H,E_1)+P(H^c\mid E_1)P(E_2\mid H^c,E_1)\). Substituting this expression gives a formula directly in terms of the prior and the evidence likelihoods.

P(H\mid E_1,E_2) = \frac{P(H)P(E_1\mid H)P(E_2\mid H,E_1)} {P(H)P(E_1\mid H)P(E_2\mid H,E_1) + P(H^c)P(E_1\mid H^c)P(E_2\mid H^c,E_1)}

For more observations, repeat the update. The corresponding joint likelihood is obtained by the chain rule: \(P(E_1,\ldots,E_k\mid H)=P(E_1\mid H)P(E_2\mid H,E_1)\cdots P(E_k\mid H,E_1,\ldots,E_{k-1})\). This factorization does not assume the observations are independent. It records the conditional probabilities needed for the observed sequence.

Start with the hypotheses and priors.
List the possible explanations and their probabilities before the new evidence is used.

Use the first observation.
Apply Bayes’ theorem to calculate the posterior probabilities after that observation.

Update with the next observation.
Use the probability of this observation conditional on each hypothesis and all evidence already seen.

Check the result.
Confirm the denominator includes all hypotheses and that the posterior probabilities sum to one.

Worked Examples

Worked Example: Reviewing Two Flags on a Hypothetical Claim

A hypothetical claims team classifies a claim as high-risk (\(H\)) or ordinary-risk (\(H^c\)). Before review, \(P(H)=0.30\). An initial document check produces a flag \(F\), with \(P(F\mid H)=0.70\) and \(P(F\mid H^c)=0.20\). After a flag, a specialist review produces a second flag \(G\), with \(P(G\mid H,F)=0.60\) and \(P(G\mid H^c,F)=0.10\). Find the probability the claim is high-risk after both flags.

Method. Apply Bayes’ theorem in sequence. The second-stage likelihoods are explicitly conditional on the first flag, so they are the appropriate likelihoods for the update.

First update after \(F\). The joint probability of high risk and the first flag is \(0.30(0.70)=0.21\). The joint probability of ordinary risk and the first flag is \(0.70(0.20)=0.14\). Thus \(P(F)=0.21+0.14=0.35\), and \(P(H\mid F)=0.21/0.35=0.60\). Consequently, \(P(H^c\mid F)=0.40\).

Now update using \(G\), conditional on \(F\). The high-risk contribution is \(0.60(0.60)=0.36\), and the ordinary-risk contribution is \(0.40(0.10)=0.04\). Therefore \(P(G\mid F)=0.36+0.04=0.40\), and

P(H\mid F,G)=\frac{0.36}{0.40}=0.90.

Check. The alternative posterior is \(0.04/0.40=0.10\), and \(0.90+0.10=1\). The claim’s posterior high-risk probability is \(0.9000\). Common mistake avoided: treating the second flag’s likelihood as unconditional or using it without conditioning on the first flag. The stated likelihoods apply specifically after \(F\).

Worked Example: Two Positive Results With Conditional Likelihoods

In a hypothetical screening program, the prior probability of a condition is \(0.10\). Test 1 is positive with probability \(0.80\) for someone with the condition and \(0.10\) for someone without it. Among people with a positive first result, Test 2 is positive with probability \(0.75\) if the condition is present and \(0.30\) if it is absent. Find the probability of the condition after two positive results.

Method. Update first on Test 1, then on Test 2 using the supplied conditional probabilities. No assumption about independence between test results is needed.

Let \(H\) denote having the condition, and let \(+\) denote a positive result. After the first positive result, the unnormalized weights are \(P(H)P(+_1\mid H)=0.10(0.80)=0.08\) and \(P(H^c)P(+_1\mid H^c)=0.90(0.10)=0.09\). Their sum is \(0.17\), so \(P(H\mid +_1)=0.08/0.17=8/17\), while \(P(H^c\mid +_1)=9/17\).

For the second positive result, the high-condition weight, using these updated probabilities, is \((8/17)(0.75)=6/17\). The no-condition weight is \((9/17)(0.30)=2.7/17\). The total conditional probability of the second positive result is \(8.7/17\). Therefore,

P(H\mid +_1,+_2) = \frac{6/17}{8.7/17} = \frac{6}{8.7} = \frac{20}{29} \approx 0.6897.

Check. The no-condition posterior is \(2.7/8.7=9/29\), and \(20/29+9/29=1\). The requested probability is \(0.6897\) to four decimal places. Common mistake avoided: multiplying two sensitivity values as if the problem supplied a sensitivity for the entire sequence. The second probability given is conditional on the first positive result; sequential updating uses that stated value.

Worked Example: Updating Across Three Competing Explanations

A hypothetical equipment review considers three possible causes of a warning: calibration error \(A\), a worn component \(B\), or a temporary obstruction \(C\). The prior probabilities are \(P(A)=0.50\), \(P(B)=0.30\), and \(P(C)=0.20\). A sensor records an unusual reading \(E_1\), with conditional probabilities \(0.60\), \(0.30\), and \(0.20\) under \(A,B,C\), respectively. A subsequent inspection finds residue \(E_2\). Given \(E_1\), the probability of \(E_2\) is \(0.50\) under \(A\), \(0.80\) under \(B\), and \(0.50\) under \(C\). Find the posterior probability of each cause after both observations.

Method. For each cause, multiply its prior by the first conditional likelihood and then by the second likelihood conditional on the first observation. Normalize the three resulting weights by their sum.

For \(A\), the weight after \(E_1\) is \(0.50(0.60)=0.30\). After \(E_2\), it is \(0.30(0.50)=0.150\). For \(B\), the weight after \(E_1\) is \(0.30(0.30)=0.090\). After \(E_2\), it is \(0.090(0.80)=0.072\). For \(C\), the weight after \(E_1\) is \(0.20(0.20)=0.040\). After \(E_2\), it is \(0.040(0.50)=0.020\). The total weight is \(0.150+0.072+0.020=0.242\).

P(A\mid E_1,E_2)=\frac{0.150}{0.242}=\frac{75}{121}\approx0.6198, \quad P(B\mid E_1,E_2)=\frac{0.072}{0.242}=\frac{36}{121}\approx0.2975, \quad P(C\mid E_1,E_2)=\frac{0.020}{0.242}=\frac{10}{121}\approx0.0826.

Check. The exact probabilities sum to \((75+36+10)/121=1\). Rounded to four decimal places, they sum to \(0.6198+0.2975+0.0826=0.9999\); the difference from one is due to rounding. Common mistake avoided: normalizing after considering only the two most likely causes. Bayes’ theorem requires including every hypothesis in the stated partition, including \(C\).

Why the Sequential Formula Works

The recursive update follows directly from the definition of conditional probability. For events with positive probability, \(P(H\mid E_1)=P(H\cap E_1)/P(E_1)\). Applying the same definition after also observing \(E_2\) gives

P(H\mid E_1,E_2) = \frac{P(H\cap E_1\cap E_2)}{P(E_1\cap E_2)} = \frac{P(H\mid E_1)P(E_2\mid H,E_1)} {P(E_2\mid E_1)}.

The last equality uses the multiplication rule in the numerator and denominator. It shows why the new observation is assessed relative to the evidence already seen: the denominator is \(P(E_2\mid E_1)\), and the numerator’s likelihood is \(P(E_2\mid H,E_1)\). The denominator can then be expanded across the hypotheses to normalize the updated weights.

Common Mistakes and Practical Checks

A frequent error is to use the right Bayes formula with the wrong likelihood. For example, suppose an analyst writes \(P(E_2\mid H)\) when the problem gives \(P(E_2\mid H,E_1)\). These quantities need not be equal. The second observation may be more or less likely depending on what has already occurred. Use only a likelihood that matches the conditioning information in the problem.

Another trap is to multiply likelihoods across observations without checking what they represent. The chain rule always permits multiplication of successive conditional probabilities. Multiplying marginal likelihoods, or likelihoods conditioned only on a hypothesis, requires additional justification and may be incorrect. In a calculation, keep the evidence sequence visible in the notation rather than silently dropping conditions.

For several hypotheses \(H_1,\ldots,H_r\) that form a partition, one useful calculation is to form weights \(w_j=P(H_j)P(E_1\mid H_j)\cdots P(E_k\mid H_j,E_1,\ldots,E_{k-1})\). The posterior for \(H_j\) is \(w_j/\sum_{m=1}^r w_m\). This is a convenient version of repeated Bayes’ theorem: all hypotheses use the same evidence sequence, and the weights are normalized only after every hypothesis has been included.

Key takeaway: After each observation, replace the prior probabilities with the current posterior probabilities. For the next update, use likelihoods conditional on both the hypothesis and all evidence already observed, then normalize across all possible hypotheses.

Check Your Understanding

For each problem, update through the evidence in the order given. Probabilities are rounded to four decimal places where needed.

A hypothetical sensor fault has prior probability \(0.25\). A first alert occurs with probability \(0.60\) if the fault is present and \(0.20\) otherwise. Given a first alert, a second alert occurs with probability \(0.50\) if the fault is present and \(0.25\) otherwise. Find the probability of a fault after both alerts. (Answer: \(0.6667\))
A hypothetical manufactured part has defect probability \(0.04\). An initial inspection alarm occurs with probability \(0.90\) for a defective part and \(0.05\) for a nondefective part. Given an initial alarm, a second alarm occurs with probability \(0.80\) for a defective part and \(0.20\) for a nondefective part. Find the probability the part is defective after two alarms. (Answer: \(0.7500\))
Three hypotheses \(A,B,C\) have priors \(0.40,0.35,0.25\). Evidence \(E\) has likelihoods \(0.50,0.20,0.40\), respectively. Find all three posterior probabilities after observing \(E\). (Answer: \(P(A\mid E)=20/37\), \(P(B\mid E)=7/37\), \(P(C\mid E)=10/37\))
A hypothetical risk category \(H\) has prior probability \(0.30\). Evidence \(E_1\) has likelihood \(0.70\) under \(H\) and \(0.10\) under \(H^c\). Given \(E_1\), evidence \(E_2\) has probability \(0.40\) under \(H\) and \(0.50\) under \(H^c\). Find \(P(H\mid E_1,E_2)\). (Answer: \(84/119\approx0.7059\))
A hypothetical condition has prior probability \(0.20\). Two observations occur in sequence. Their conditional probabilities are \(0.80\) and \(0.60\) given the condition, with the second probability conditional on the condition and the first observation. Without the condition, the corresponding probabilities are \(0.30\) and \(0.40\), with the second conditional on no condition and the first observation. Find the posterior probability of the condition after both observations. (Answer: \(0.5000\))

What You'll Learn