Conditioning on a Shared Latent Variable

Why a Shared Latent Variable Changes a Sum

Suppose several trials have the same success probability once a background condition is known. That condition might be an unobserved environment, a risk class, or a temporary operating state. If the condition varies from one group of trials to another, it is natural to represent its success probability by a random variable \(\Theta\). The outcomes can be independent when \(\Theta\) is fixed, yet dependent when it is not observed. To find the distribution of their total, first condition on \(\Theta\), then average over its distribution.

This distinction is useful whenever a shared factor affects a group of observations. The key is that the same value of \(\Theta\) applies to every trial in the group; drawing a separate success probability for each trial would describe a different model.

Definition: Let \(\Theta\sim\operatorname{Beta}(\alpha,\beta)\), where \(\alpha>0\) and \(\beta>0\). Given \(\Theta=\theta\), let \(X_1,\ldots,X_n\) be conditionally independent Bernoulli\((\theta)\) random variables, and define \(K=\sum_{j=1}^n X_j\). Then \(K\) has a beta-binomial distribution with parameters \(n,\alpha,\beta\). Conditional independence means that the \(X_j\) are independent after the value of \(\Theta\) is fixed; it does not assert their unconditional independence.

From Conditional Binomial to Beta-Binomial

The beta density is \(f_\Theta(\theta)=\theta^{\alpha-1}(1-\theta)^{\beta-1}/B(\alpha,\beta)\) for \(0<\theta<1\), where \(B(r,s)=\int_0^1\theta^{r-1}(1-\theta)^{s-1}\,d\theta\). Given \(\Theta=\theta\), the total \(K\) is binomial with \(n\) trials and success probability \(\theta\). Conditioning and integrating gives the unconditional probability of each possible count.

\begin{aligned} P(K=k) &=\int_0^1 P(K=k\mid\Theta=\theta)f_\Theta(\theta)\,d\theta\\ &=\binom{n}{k}\int_0^1 \frac{\theta^k(1-\theta)^{n-k}\theta^{\alpha-1}(1-\theta)^{\beta-1}} {B(\alpha,\beta)}\,d\theta\\ &=\binom{n}{k}\frac{B(\alpha+k,\beta+n-k)}{B(\alpha,\beta)}, \qquad k=0,1,\ldots,n. \end{aligned}

The binomial coefficient counts the different arrangements of \(k\) successes among \(n\) trials. For one specified arrangement, such as success, success, failure, the probability conditional on \(\Theta=\theta\) is \(\theta^k(1-\theta)^{n-k}\), with no binomial coefficient. Integrating that expression gives \(B(\alpha+k,\beta+n-k)/B(\alpha,\beta)\). Multiplying by \(\binom{n}{k}\) converts the probability of one arrangement into the probability of the total count \(K=k\).

When the parameters are positive integers, the beta function can be evaluated using \(B(r,s)=(r-1)!(s-1)!/(r+s-1)!\). For other positive parameters, the beta-function ratio remains the standard probability formula.

Fix the latent probability.
Given \(\Theta=\theta\), use the binomial model for the count, or the Bernoulli product for a specified sequence.

Average over the latent distribution.
Integrate the conditional probability against the beta density, combining the powers of \(\theta\) and \(1-\theta\).

Check the event being asked for.
Use the binomial coefficient for a count, but not for one specified ordering of outcomes. For a range of counts, add the relevant beta-binomial probabilities.

Worked Applications

Worked Example: An Exact Count Under Shared Heterogeneity

A hypothetical group of four devices shares an unobserved operating condition. The condition's success probability \(\Theta\) has a \(\operatorname{Beta}(2,3)\) distribution. Given \(\Theta\), the devices succeed independently with probability \(\Theta\). Find the probability that exactly two devices succeed.

Method. Apply the beta-binomial count formula with \(n=4\), \(\alpha=2\), \(\beta=3\), and \(k=2\). The event is a count, so include \(\binom{4}{2}=6\).

First evaluate the denominator beta function: \(B(2,3)=1!2!/4!=2/24=1/12\). For the numerator, \(B(4,5)=3!4!/8!=144/40320=1/280\). Therefore,

P(K=2)=\binom{4}{2}\frac{B(4,5)}{B(2,3)} =6\frac{1/280}{1/12} =6\frac{12}{280} =\frac{9}{35} =0.2571.

The probability is \(0.2571\) to four decimal places. As a check, the value is between zero and one, and it is the probability of the count \(K=2\), not merely of one particular ordering such as success, success, failure, failure.

Common mistake avoided: Do not replace the shared \(\Theta\) with its mean and then treat the four outcomes as unconditionally independent. That would use \(E[\Theta]=2/5\) and give \(\binom{4}{2}(2/5)^2(3/5)^2=216/625=0.3456\), which is not the model's probability.

Worked Example: Predicting Two Outcomes After Observations

A hypothetical testing process has a shared success probability \(\Theta\sim\operatorname{Beta}(2,2)\). Three conditionally independent outcomes are observed: two successes and one failure. Find the probability that the next two outcomes are both successes, assuming the same \(\Theta\) continues to apply.

Method. Update the beta density using the observed outcomes, then integrate the conditional probability of two further successes. The likelihood of the observed data, apart from a factor that does not depend on \(\theta\), is \(\theta^2(1-\theta)\). Multiplying by the prior density gives a density proportional to \(\theta^{3}(1-\theta)^2\), so the conditional distribution of \(\Theta\) after these observations is \(\operatorname{Beta}(4,3)\).

Given \(\Theta=\theta\), the probability that both next outcomes succeed is \(\theta^2\). Averaging over the updated beta distribution gives

\begin{aligned} P(\text{both next outcomes succeed}\mid\text{observations}) &=\frac{B(4+2,3)}{B(4,3)}\\ &=\frac{5!2!/8!}{3!2!/6!} =\frac{1/168}{1/60} =\frac{5}{14} =0.3571. \end{aligned}

Thus the requested probability is \(0.3571\) to four decimal places. The beta-function ratio averages \(\theta^2\), not \(\theta\); the two future successes share the same unknown probability.

Common mistake avoided: Updating the beta parameters is essential. Using the original \(\operatorname{Beta}(2,2)\) distribution would ignore the two observed successes and one failure. Also, the observed outcomes update the shared probability; they are not included in the requested count of the next two outcomes.

Worked Example: A Tail Probability for a Group Total

In a hypothetical batch of three inspections, the shared success probability has distribution \(\Theta\sim\operatorname{Beta}(3,1)\). Given \(\Theta\), inspections are independent Bernoulli\((\Theta)\). Find the probability of at least two successes.

Method. Since \(K\geq2\) means \(K=2\) or \(K=3\), calculate both beta-binomial probabilities and add them. The common denominator is \(B(3,1)=2!0!/3!=1/3\).

For \(K=2\), the numerator beta function is \(B(5,2)=4!1!/6!=1/30\). Hence \(P(K=2)=\binom{3}{2}(1/30)/(1/3)=3/10\). For \(K=3\), it is \(B(6,1)=5!0!/6!=1/6\), so \(P(K=3)=\binom{3}{3}(1/6)/(1/3)=1/2\). Adding the disjoint cases gives

P(K\geq2)=P(K=2)+P(K=3)=\frac{3}{10}+\frac{1}{2}=\frac{4}{5}=0.8000.

The probability is \(0.8000\) to four decimal places. The cases \(K=2\) and \(K=3\) are disjoint and together cover every way to have at least two successes, which checks that the tail calculation matches the stated event.

Common mistake avoided: “At least two” is not the same event as “exactly two.” Omitting \(P(K=3)\) would give \(0.3000\), not the requested tail probability.

Interpreting the Model and Avoiding Traps

The beta-binomial model is a mixture: conditional on the latent probability, the count is binomial; unconditionally, its probability mass function averages those binomial probabilities over the beta distribution. The beta distribution is a convenient model for uncertainty or variation in a probability between groups. The model requires a common latent \(\Theta\) for the trials in a group and conditional independence given that value. If the trials have different conditional probabilities or remain dependent even after conditioning, this formula need not apply.

A useful diagnostic is the event description. “Exactly \(k\) successes” asks for a total and requires the binomial coefficient. “The first \(k\) are successes and the rest are failures” describes one specified sequence and does not. If the question asks for a range of counts, sum the probabilities for the included values or use a complement when that is simpler.

Do not assume unconditional independence. Conditional independence does not make the shared latent probability disappear.
Keep the same latent variable throughout the group. Independent draws of a new probability for each trial produce a different construction.
Update only when observations are given. For a beta prior and Bernoulli observations, add the number of successes to \(\alpha\) and the number of failures to \(\beta\).
Match the formula to the event. Include the binomial coefficient for a count, not for one specified sequence.

The method is aligned with the SOA VEE Mathematical Statistics syllabus because it combines conditioning, discrete sums, and integration over a continuous distribution. Its main practical lesson is simple: write the conditional model first, identify what is random in the unconditional model, and only then calculate the probability of the requested total.

Check Your Understanding

For each problem, state whether the event concerns a count or a specified sequence before calculating.

Let \(\Theta\sim\operatorname{Beta}(1,2)\). Given \(\Theta\), three trials are conditionally independent Bernoulli\((\Theta)\). Find the probability of no successes. (Answer: \(2/5=0.4000\))
Let \(\Theta\sim\operatorname{Beta}(2,2)\), and let \(K\) be the number of successes in three conditionally independent trials given \(\Theta\). Find \(P(K=1)\). (Answer: \(3/10=0.3000\))
Let \(\Theta\sim\operatorname{Beta}(3,2)\). Given \(\Theta\), four trials are conditionally independent Bernoulli\((\Theta)\). Find the probability that all four succeed. (Answer: \(3/14=0.2143\))
A prior is \(\operatorname{Beta}(2,1)\). After two successes and one failure, two further conditionally independent outcomes are observed under the same \(\Theta\). Find the probability that exactly one of these two outcomes is a success. (Answer: \(8/21=0.3810\))
Let \(\Theta\sim\operatorname{Beta}(2,2)\), with two conditionally independent Bernoulli\((\Theta)\) trials. Find the probability that the two outcomes differ. (Answer: \(2/5=0.4000\))

What You'll Learn