Bayes’ Theorem With Sequential Evidence

Why Sequential Evidence Requires Care

A single observation may shift probability toward one explanation and away from another. When evidence arrives in stages, the update must also reflect how likely each new observation is given the earlier evidence. A reading that is informative after one history may have a different likelihood after another. Reusing an unconditional probability, or treating observations as independent without support, can therefore produce the wrong posterior.

The method is to list the possible explanations as a partition, retain their prior probabilities, and compare how likely the complete observed sequence is under each explanation. This is an application of Bayes’ theorem aligned with the SOA Exam P (Probability) syllabus.

Definition: Let $H_1,\ldots,H_k$ be mutually exclusive events whose union is the entire sample space. They form a partition of the possible explanations. For evidence $E$ with $P(E)>0$, Bayes’ theorem gives $$ P(H_j\mid E)=\frac{P(H_j\cap E)}{\sum_{r=1}^{k}P(H_r\cap E)}. $$ If $P(H_j)>0$, the numerator can also be written $P(H_j)P(E\mid H_j)$. The probabilities $P(H_j)$ are the prior probabilities; $P(H_j\mid E)$ are the posterior probabilities.

Updating After a Sequence

Suppose the observed evidence is $E_1,\ldots,E_n$, and write $E=E_1\cap\cdots\cap E_n$. The direct form of Bayes’ theorem is

P(H_j\mid E_1,\ldots,E_n) = \frac{P(H_j\cap E_1\cap\cdots\cap E_n)} {\sum_{r=1}^{k}P(H_r\cap E_1\cap\cdots\cap E_n)}, \qquad P(E_1\cap\cdots\cap E_n)>0.

This joint-probability form is the safest general statement. If $P(H_j)>0$, its joint probability can be expressed using the conditional probability of the entire sequence given $H_j$. When all the conditional probabilities in the following chain are defined, the chain rule gives

P(E_1,\ldots,E_n\mid H_j) = P(E_1\mid H_j) \prod_{m=2}^{n}P(E_m\mid H_j,E_1,\ldots,E_{m-1}).

Thus each new factor is conditioned on the explanation $H_j$ and all earlier evidence. The chain rule does not say the observations are independent. It is a way to factor a joint probability into successive conditional probabilities.

Important condition: The direct joint-probability formula requires only that the total evidence have positive probability. A product of successive conditional likelihoods requires each conditional probability in that product to be defined. If an earlier evidence prefix has probability zero under a hypothesis, a later conditional probability given that prefix is undefined. In that case, use $P(H_j\cap E_1\cap\cdots\cap E_n)$ directly; do not insert an undefined factor into a product.

Why the Sequential Formula Holds

For a fixed hypothesis $H_j$, the chain rule follows by repeatedly applying the definition of conditional probability. For two observations, when the relevant probabilities are positive,

P(E_1,E_2\mid H_j) = P(E_1\mid H_j)P(E_2\mid H_j,E_1).

Indeed, the right-hand side equals $P(E_1\mid H_j)\,P(E_1\cap E_2\mid H_j)/P(E_1\mid H_j)$, which is $P(E_1\cap E_2\mid H_j)$. Repeating this argument yields the product for any finite sequence, provided each conditional factor used is defined. Multiplying this sequence likelihood by $P(H_j)$ gives the joint probability of the hypothesis and the evidence. Normalizing these joint probabilities across the partition gives the posterior formula.

List the hypotheses and priors.
Check that the hypotheses are mutually exclusive, exhaustive, and have prior probabilities summing to one.

Write the evidence in order.
For each hypothesis, identify the likelihood of each observation conditional on that hypothesis and the preceding observations.

Find joint weights.
Multiply the prior by a valid sequence likelihood, or calculate the joint probability directly if a conditional factor is undefined.

Normalize and check.
Divide each joint weight by their sum. The posterior probabilities must sum to one.

Worked Applications

Worked Example: Two Sequential Quality Checks

A hypothetical component comes from either line A or line B. The prior probabilities are $P(A)=0.30$ and $P(B)=0.70$. A first check gives a positive result $E_1$, with $P(E_1\mid A)=0.80$ and $P(E_1\mid B)=0.20$. A second check gives a negative result $E_2$, with $P(E_2\mid A,E_1)=0.25$ and $P(E_2\mid B,E_1)=0.50$. Find the probability the component came from A given both results. These are hypothetical probabilities for this example.

Method. Use the prior probability of each line multiplied by its likelihood for the complete sequence. The second likelihood is conditional on the first result.

For line A, the joint weight is $0.30\times0.80\times0.25=0.060$. For line B, it is $0.70\times0.20\times0.50=0.070$. The total probability of the observed sequence is $0.060+0.070=0.130$. Therefore,

P(A\mid E_1,E_2) = \frac{0.060}{0.130} = \frac{6}{13} \approx 0.4615.

The posterior probability of line A is 0.4615, to four decimal places. As a check, the posterior for B is $0.070/0.130=7/13\approx0.5385$, and $0.4615+0.5385=1.0000$ after rounding. The first positive result alone would favor A, but the second result changes the balance because its conditional likelihood is larger under B.

Worked Example: Three Explanations and Three Observations

A hypothetical sensor has one of three operating states, $H_1,H_2,H_3$, with priors $0.50,0.30,0.20$, respectively. It records $E_1$, then $E_2$, then $E_3$. The conditional probabilities are:

Under $H_1$: $P(E_1\mid H_1)=0.60$, $P(E_2\mid H_1,E_1)=0.50$, and $P(E_3\mid H_1,E_1,E_2)=0.20$.
Under $H_2$: the corresponding probabilities are $0.30,0.40,0.50$.
Under $H_3$: the corresponding probabilities are $0.10,0.80,0.25$.

Find the posterior probability of $H_1$ given all three observations. No independence assumption is made; each likelihood is supplied conditional on the preceding observations.

Method. Calculate one joint weight for each state by multiplying its prior by its three successive conditional likelihoods:

\begin{aligned} w_1&=0.50(0.60)(0.50)(0.20)=0.030,\\ w_2&=0.30(0.30)(0.40)(0.50)=0.018,\\ w_3&=0.20(0.10)(0.80)(0.25)=0.004. \end{aligned}

The total probability of the observed sequence is $0.030+0.018+0.004=0.052$. Hence

P(H_1\mid E_1,E_2,E_3) = \frac{0.030}{0.052} = \frac{15}{26} \approx 0.5769.

The posterior probabilities are $0.030/0.052=0.5769$, $0.018/0.052=0.3462$, and $0.004/0.052=0.0769$, to four decimal places. Their rounded sum is $1.0000$. The method avoids treating the three observations as independent: the supplied likelihoods explicitly depend on the evidence history.

Worked Example: A Zero-Likelihood Prefix

A hypothetical investigation has two explanations, $H_1$ and $H_2$, with priors $P(H_1)=0.40$ and $P(H_2)=0.60$. The first observation $E_1$ satisfies $P(E_1\mid H_1)=0$ and $P(E_1\mid H_2)=0.50$. Under $H_2$, the next observation has $P(E_2\mid H_2,E_1)=0.40$. Find the posterior probability of $H_2$ given $E_1,E_2$.

Method. Use joint probabilities for the complete evidence. The expression $P(E_2\mid H_1,E_1)$ is undefined because $P(H_1\cap E_1)=0$; it must not be used as a factor.

Under $H_1$, the joint probability of $H_1,E_1,E_2$ is zero, because $P(H_1\cap E_1)=0$. Under $H_2$, the joint probability is

$$ P(H_2,E_1,E_2) = 0.60(0.50)(0.40) = 0.12. $$

The total probability of the evidence is $0+0.12=0.12>0$. Thus $P(H_2\mid E_1,E_2)=0.12/0.12=1$. The posterior for $H_1$ is zero, so the posterior probabilities sum to one. The joint form reaches the result without assigning a value to an undefined conditional probability.

Common Mistakes and Checks

A frequent error is to use a marginal likelihood for a later observation where a history-specific likelihood is required. In the first example, the probability of $E_2$ given $E_1$, without specifying the line, is

P(E_2\mid E_1) = \frac{0.30(0.80)(0.25)+0.70(0.20)(0.50)} {0.30(0.80)+0.70(0.20)} = \frac{0.130}{0.380} \approx 0.3421.

Wrong calculation: use this same $0.3421$ as the second likelihood for both lines. The resulting weights would be $0.30(0.80)(0.3421)$ and $0.70(0.20)(0.3421)$, leaving the posterior for A at $0.24/(0.24+0.14)\approx0.6316$, the posterior after $E_1$ alone. This wrongly makes $E_2$ irrelevant to the comparison.

Correction: $P(E_2\mid E_1)$ is an average across the possible lines; it is not the likelihood within either line. Use $P(E_2\mid A,E_1)=0.25$ and $P(E_2\mid B,E_1)=0.50$ separately. This gives the correct posterior $0.4615$.

Do not confuse priors with posteriors. A prior $P(H_j)$ is not the probability of $H_j$ after observing evidence.
Do not silently assume conditional independence. Only replace a conditional likelihood with a simpler one when the model justifies that equality.
Check the conditioning history. The order of observations matters when the likelihood depends on earlier results.
Check that the evidence has positive probability. A posterior conditional on an event of probability zero is not defined.
Check normalization. Posterior probabilities over an exhaustive partition must sum to one, apart from any rounding.

The essential calculation is a comparison of prior-weighted likelihoods for the complete observed sequence. Keeping each hypothesis-specific likelihood separate preserves the effect of every observation and makes the final normalization transparent.

Check Your Understanding

For each problem, identify the prior-weighted likelihoods before normalizing. Conditional probabilities not stated as independent should be read as conditional on the evidence history shown.

A hypothetical parcel is sent by route A with prior probability $0.25$, or route B with probability $0.75$. A scan gives $E_1$, with likelihoods $0.80$ under A and $0.20$ under B. A second scan gives $E_2$, with $P(E_2\mid A,E_1)=0.10$ and $P(E_2\mid B,E_1)=0.40$. Find $P(A\mid E_1,E_2)$. (Answer: 0.2500)
Three hypothetical sources have priors $0.20,0.50,0.30$. Two observations $E_1,E_2$ have successive likelihoods $(0.50,0.40)$ under $H_1$, $(0.20,0.50)$ under $H_2$, and $(0.80,0.25)$ under $H_3$. Find $P(H_3\mid E_1,E_2)$. (Answer: 0.4000)
Two explanations have priors $0.40,0.60$. Under $H_1$, $P(E_1\mid H_1)=0$; under $H_2$, $P(E_1\mid H_2)=0.50$ and $P(E_2\mid H_2,E_1)=0.40$. If $P(E_1,E_2\mid H_1)=0$, find $P(H_2\mid E_1,E_2)$. (Answer: 1.0000)
Two hypothetical device types have priors $P(H_1)=0.40$ and $P(H_2)=0.60$. Three readings occur in order. Their successive likelihoods are $0.70,0.20,0.50$ under $H_1$, and $0.30,0.50,0.25$ under $H_2$. Find $P(H_1\mid E_1,E_2,E_3)$. (Answer: 0.5545)
A hypothetical alert can come from either $H_1$ or $H_2$, with priors $0.30$ and $0.70$. For two observations, the successive likelihoods are $0.60,0.50$ under $H_1$, and $0.20,0.25$ under $H_2$. Find the posterior odds $P(H_1\mid E_1,E_2)/P(H_2\mid E_1,E_2)$. (Answer: $18/7$, or approximately $2.5714:1$)

What You'll Learn