Tutorials › VEE Mathematical Statistics › When Independent Bernoulli Variables Have a Binomial Sum

VEE Mathematical Statistics · Bernoulli and Binomial Models · Tutorial 97 of 880

When Independent Bernoulli Variables Have a Binomial Sum

Determine when independent Bernoulli outcomes produce a binomial count, and calculate probabilities when trial probabilities differ.

Beginner 11 min read

What You'll Learn

  • State the independence and common-probability conditions for a binomial sum.
  • Derive the binomial probability formula by counting success patterns.
  • Calculate exact and tail probabilities from a sum of Bernoulli variables.
  • Explain why unequal success probabilities generally do not give a binomial count.
  • Identify why a matching mean does not establish a binomial model.

Why the Conditions Behind a Binomial Count Matter

A binomial count records how many successes occur in a fixed number of trials. It is tempting to use a binomial model whenever the outcomes are independent and each is either a success or a failure. Independence alone, however, is not enough: the success probabilities must also be the same for every trial. This tutorial derives that result and shows what changes when the probabilities differ.

The distinction matters in practical applications. For example, a hypothetical system might contain several components that can each fail or operate. If the components have different failure probabilities, treating the total number of failures as binomial can give incorrect probabilities even when the component outcomes are independent.

Definition: A Bernoulli random variable \(X_i\) records the outcome of one trial: \(X_i=1\) for success and \(X_i=0\) for failure. If \(P(X_i=1)=p_i\), then \(X_i\) has a Bernoulli distribution with success probability \(p_i\). For a fixed number \(n\) of trials, the count of successes is \(S=\sum_{i=1}^{n}X_i\).

The sum \(S\) is always a count between 0 and \(n\). But its distribution is binomial with parameters \(n\) and \(p\) only under the appropriate conditions.

Theorem: Suppose \(X_1,\ldots,X_n\) are independent Bernoulli random variables, and each has the same success probability \(p\). Then \(S=\sum_{i=1}^{n}X_i\sim\operatorname{Binomial}(n,p)\), with \(P(S=k)=\binom{n}{k}p^k(1-p)^{n-k}\) for \(k=0,1,\ldots,n\).

The conditions are specific: there is a fixed number \(n\) of trials, every trial has two outcomes coded as success or failure, the trials are independent, and every trial has the same success probability \(p\). If \(p=0\), the count is always 0; if \(p=1\), it is always \(n\). These are the degenerate binomial cases.

Why Identical Probabilities Give the Binomial Formula

For exactly \(k\) successes, first consider one particular arrangement, such as successes on trials 1 and 3 and failures on all other trials. Independence means the probability of that arrangement is the product of its trial probabilities. With a common success probability \(p\), every such arrangement has the same probability, \(p^k(1-p)^{n-k}\). There are \(\binom{n}{k}\) different arrangements with exactly \(k\) successes. Adding their probabilities gives the binomial formula.

1
Fix a success pattern.
For a particular pattern with \(k\) successes and \(n-k\) failures, independence and the common value \(p\) give probability \(p^k(1-p)^{n-k}\).
2
Count the patterns.
Choose which \(k\) of the \(n\) trials are successes. There are \(\binom{n}{k}\) such choices.
3
Add the probabilities.
The patterns are disjoint and have equal probability, so \(P(S=k)=\binom{n}{k}p^k(1-p)^{n-k}\).

This argument shows why both assumptions matter. Independence lets us multiply probabilities within a pattern. A common probability makes the probability the same across patterns with \(k\) successes, so the count \(\binom{n}{k}\) can be factored out.

Worked Example: Exactly Two Positive Results

A hypothetical laboratory processes five independent samples. Each sample has probability \(0.2\) of producing a positive result, with the same probability for all five. Find the probability of exactly two positive results.

Method. Let \(X_i=1\) if sample \(i\) is positive and \(X_i=0\) otherwise. The five indicators are independent Bernoulli variables with common \(p=0.2\). Thus \(S=\sum_{i=1}^{5}X_i\sim\operatorname{Binomial}(5,0.2)\).

Substitute \(n=5\), \(k=2\), and \(1-p=0.8\):

$$ P(S=2)=\binom{5}{2}(0.2)^2(0.8)^3 $$

Evaluate each factor: \(\binom{5}{2}=10\), \((0.2)^2=0.04\), and \((0.8)^3=0.512\). Therefore \(P(S=2)=10(0.04)(0.512)=10(0.02048)=0.2048\).

The probability is \(0.2048\). The common mistake this method avoids is counting just one arrangement, such as positive results on samples 1 and 2, without accounting for all \(\binom{5}{2}=10\) possible pairs.

Using the Model for an Event, Not Just One Count

Once the binomial model is justified, an event involving the count can be translated into a set of possible values. For example, “at least three successes” means \(S=3\) or \(S=4\) when there are four trials. The probability is the sum of the probabilities for those disjoint values.

Worked Example: At Least Three Activations

A hypothetical monitoring system has four independent alarms. Each alarm activates during a specified test with probability \(0.25\). Find the probability that at least three alarms activate.

Method. Each activation is a success, and all four independent trials have the common success probability \(p=0.25\). Hence \(S\sim\operatorname{Binomial}(4,0.25)\). “At least three” means \(S=3\) or \(S=4\).

$$ P(S\geq 3)=P(S=3)+P(S=4) =\binom{4}{3}(0.25)^3(0.75)+(0.25)^4 $$

For three activations, \(\binom{4}{3}=4\), \((0.25)^3=0.015625\), and \(0.015625(0.75)=0.01171875\). Thus \(P(S=3)=4(0.01171875)=0.046875\).

For four activations, \(P(S=4)=(0.25)^4=0.00390625\). Adding the two terms gives \(P(S\geq3)=0.046875+0.00390625=0.05078125\).

The probability is \(0.05078125\). A common mistake is to add the two terms but report \(0.0625\); the terms are \(0.046875\) and \(0.00390625\), whose sum is \(0.05078125\), not \(0.0625\). Each term should be evaluated before the final addition.

What Changes When Success Probabilities Differ?

Suppose the trials are still independent, but their success probabilities are \(p_1,\ldots,p_n\), not all equal. A particular pattern with success set \(A\) has probability \(\prod_{i\in A}p_i\prod_{i\notin A}(1-p_i)\). Unlike the equal-probability case, patterns with the same number of successes can now have different probabilities. Therefore, one cannot generally multiply a single pattern probability by \(\binom{n}{k}\).

The sum is often called a Poisson-binomial count when the independent Bernoulli trials have different success probabilities. For a specified \(k\), its probability is found by adding the probabilities of all patterns with \(k\) successes. This is not the binomial formula with an average of the \(p_i\)'s substituted for \(p\).

Worked Example: Unequal Probabilities and a Non-Binomial Count

Three independent sensors report a signal with probabilities \(0.2\), \(0.5\), and \(0.8\), respectively. Let \(S\) be the number that report a signal. Find \(P(S=2)\), and determine whether \(S\) is binomial.

Method. The sensors are independent Bernoulli trials, but their success probabilities differ, so the binomial theorem above does not apply directly. For exactly two signals, add the probabilities of the three possible pairs:

$$ \begin{aligned} P(S=2) &=(0.2)(0.5)(1-0.8) +(0.2)(1-0.5)(0.8) +(1-0.2)(0.5)(0.8)\\ &=(0.2)(0.5)(0.2)+(0.2)(0.5)(0.8)+(0.8)(0.5)(0.8)\\ &=0.020+0.080+0.320=0.420. \end{aligned} $$

To check whether the count is binomial with \(n=3\), compare the endpoint probabilities. No signals has probability \(P(S=0)=(0.8)(0.5)(0.2)=0.08\), and three signals has probability \(P(S=3)=(0.2)(0.5)(0.8)=0.08\). If \(S\sim\operatorname{Binomial}(3,r)\), then these probabilities would be \((1-r)^3\) and \(r^3\). Their equality requires \(r=0.5\); that binomial model would give \(P(S=0)=(0.5)^3=0.125\), not \(0.08\). Thus this count is not binomial.

The answer is \(P(S=2)=0.420\), and the count is not binomial. The common mistake is to average the three success probabilities to get \(0.5\) and assume \(S\sim\operatorname{Binomial}(3,0.5)\). The average gives the same mean, \(1.5\), but does not make the probabilities of all success patterns equal or make the distribution binomial.

A Precise Check on the Equal-Probability Condition

For \(n\geq2\), if independent Bernoulli variables with probabilities strictly between 0 and 1 have a binomial sum with \(n\) trials, then their probabilities must in fact all be equal. One way to see this is to encode the probabilities of the count in the polynomial \(G(z)=E[z^S]\). Independence gives \(G(z)=\prod_{i=1}^{n}((1-p_i)+p_i z)\). If the sum is \(\operatorname{Binomial}(n,p)\), the binomial theorem gives \(G(z)=((1-p)+pz)^n\). The first expression has one linear factor for each trial; the second repeats the same linear factor \(n\) times. Equality of these polynomials requires each factor to be proportional to that repeated factor, which means \(p_i=p\) for every \(i\). If a trial has probability 0 or 1, the count has a restricted range; the degenerate endpoint cases also require all trials to have the same endpoint probability to give a binomial count with \(n\) trials.

This fact is stronger than saying unequal probabilities can cause trouble: with nondegenerate independent Bernoulli trials, a binomial distribution with the full trial count requires a common success probability. For a single trial, the condition is automatic because there is only one probability.

Common Mistakes and Practical Checks

  • Assuming independence is sufficient. Independent outcomes with different success probabilities do not generally produce a binomial count.
  • Using the average probability as if it were common. The average can help describe the mean count, but it does not determine the entire distribution.
  • Multiplying one pattern by \(\binom{n}{k}\) when probabilities differ. That shortcut requires all patterns with \(k\) successes to have the same probability.
  • Ignoring what “at least” includes. For \(S\geq k\), include the probability at \(k\) as well as all larger possible counts.
  • Forgetting the failure probability. A pattern with \(k\) successes and \(n-k\) failures has a factor \(p^k(1-p)^{n-k}\), not just \(p^k\).
Key takeaway: A sum of independent Bernoulli variables is binomial with parameters \(n\) and \(p\) when every trial has the same success probability \(p\). Independence supports multiplication within a pattern; the common probability makes all patterns with the same success count equally likely.

Check Your Understanding

For each problem, first decide whether a binomial model is justified before calculating.

  1. Six independent components each pass a test with probability \(0.1\). Find the probability that none pass. (Answer: \(0.9^6=0.531441\).)
  2. Five independent claims are each flagged for review with probability \(0.4\). Find the probability that at least one is flagged. (Answer: \(1-(0.6)^5=0.92224\).)
  3. Two independent trials have success probabilities \(0.3\) and \(0.7\). Find the probability of exactly one success, and explain why the common-\(p\) binomial formula does not apply. (Answer: \((0.3)(0.3)+(0.7)(0.7)=0.58\); the probabilities differ.)
  4. Three independent devices have success probabilities \(0.2\), \(0.2\), and \(0.5\). Find the probability of exactly two successes. (Answer: \((0.2)(0.2)(0.5)+2(0.2)(0.8)(0.5)=0.18\).)
  5. Four independent trials have success probabilities \(0.1\), \(0.3\), \(0.6\), and \(0.8\). Identify the appropriate type of count and find the probability of exactly one success. (Answer: A Poisson-binomial count; \(0.0056+0.0216+0.0756+0.2016=0.3044\).)
Clinical Biostats