Tutorials › Statistical Theory I - Probability and Distribution Theory › Deriving the Binomial Probability Formula

Home  /  Tutorials  /  Bernoulli and Binomial Families

Bernoulli and Binomial Families · Tutorial 106 of 994

Deriving the Binomial Probability Formula

Learn how independence and counting combine to give the probability of exactly \(k\) successes in \(n\) repeated Bernoulli trials.

Beginner 9 min read

What You'll Learn

  • Express a count of successes as a sum of Bernoulli indicators.
  • Find the probability of a particular ordered success-failure sequence.
  • Count the sequences containing exactly a specified number of successes.
  • Derive the binomial probability mass function from disjoint sequence events.
  • Apply the formula to repeated-trial examples and identify a common counting error.

Why Counting Is Part of the Probability Formula

A count of successes does not specify the order in which they occurred. For example, in four trials, exactly two successes could appear as success-success-failure-failure or as failure-success-failure-success, among other arrangements. Each particular arrangement has a probability, but the event “exactly two successes” includes all of them. To find the probability of the count, we must both calculate the probability of an individual arrangement and count how many arrangements belong to the event.

Suppose there are \(n\) independent trials, each with the same probability \(p\) of success, where \(0<p<1\). Let \(X_i=1\) if trial \(i\) is a success and \(X_i=0\) otherwise. The total number of successes is \(X=X_1+\cdots+X_n\). The derivation below identifies the probability that this total equals a specified integer \(k\).

Definition: In \(n\) independent Bernoulli trials with common success probability \(p\), the success count is \(X=\sum_{i=1}^{n}X_i\), where each \(X_i\) is 1 for a success and 0 for a failure. The binomial probability mass function gives \(P(X=k)\), the probability of exactly \(k\) successes.

Write \(q=1-p\) for the probability of failure. A trial sequence is an ordered list of \(n\) outcomes, each either success or failure. For instance, with \(n=4\), the sequence success-failure-success-failure has two successes. Order matters when distinguishing sequences, even though the count \(X\) does not retain that order.

Probability of One Specified Sequence

Consider a particular sequence of \(n\) trial outcomes that contains exactly \(k\) successes and \(n-k\) failures. Independence means that the probability of the entire sequence is the product of the probabilities of its individual outcomes. Each success contributes a factor \(p\), and each failure contributes a factor \(q\). Therefore the sequence has probability \(p^kq^{n-k}\), regardless of where its successes appear.

$$ P(\text{one specified sequence with }k\text{ successes}) =\underbrace{p\cdots p}_{k\text{ factors}} \underbrace{q\cdots q}_{n-k\text{ factors}} =p^kq^{n-k}. $$

The phrase one specified sequence is essential. This is not yet the probability of exactly \(k\) successes. That event can happen through many distinct sequences, and their probabilities must be added.

Worked Example: Probability of a Particular Sequence

A machine produces an acceptable item with probability \(0.8\) on each independent production run. Find the probability of the specific four-run sequence acceptable, unacceptable, acceptable, acceptable.

There are three acceptable outcomes and one unacceptable outcome. Independence allows us to multiply the probabilities in the stated order:

$$ (0.8)(0.2)(0.8)(0.8) =(0.8)^3(0.2) =0.512(0.2) =0.1024. $$

Thus this one ordered sequence has probability \(0.1024\). The probability of exactly three acceptable items is larger, because other sequences with three acceptable items are also possible.

Counting Sequences With Exactly \(k\) Successes

To form a sequence with exactly \(k\) successes, choose which \(k\) of the \(n\) positions contain successes. All remaining positions must contain failures. The number of such choices is the binomial coefficient

$$ \binom{n}{k}=\frac{n!}{k!(n-k)!}, $$

where \(m!=m(m-1)\cdots 2\cdot1\) for a positive integer \(m\), and \(0!=1\). One way to verify this count is to first arrange \(k\) success labels and \(n-k\) failure labels in \(n!\) orders. Permuting the success labels among themselves does not create a new outcome sequence, and neither does permuting the failure labels. Each distinct success-failure sequence is therefore counted \(k!(n-k)!\) times in \(n!\), giving \(n!/[k!(n-k)!]\) distinct sequences.

The event \(X=k\) consists of exactly these \(\binom{n}{k}\) sequences. Distinct sequences cannot occur simultaneously: one set of trial outcomes cannot be two different ordered lists. Thus the sequence events are disjoint. Each has probability \(p^kq^{n-k}\), so adding their probabilities gives the formula.

Theorem: If \(X\) counts successes in \(n\) independent trials, each with success probability \(p\), then for an integer \(k\) with \(0\leq k\leq n\), $$ P(X=k)=\binom{n}{k}p^k(1-p)^{n-k}. $$

Proof. Fix \(k\). Each ordered trial sequence with \(k\) successes has probability \(p^k(1-p)^{n-k}\), by independence and the common success probability. There are \(\binom{n}{k}\) such sequences, because specifying a sequence is equivalent to selecting the \(k\) positions occupied by successes. The event \(X=k\) is the disjoint union of these sequence events. By additivity of probability for a finite disjoint union,

$$ P(X=k) =\sum_{\text{sequences with }k\text{ successes}} p^k(1-p)^{n-k} =\binom{n}{k}p^k(1-p)^{n-k}. $$

This establishes the formula. \(\square\)

Worked Example: Exactly Two Heads in Four Tosses

A coin has probability \(p=0.6\) of landing heads on each independent toss. Find the probability of exactly two heads in four tosses.

Here \(n=4\), \(k=2\), and \(q=1-p=0.4\). First count the sequences with two heads by choosing which two of the four positions are heads:

$$ \binom{4}{2}=\frac{4!}{2!2!} =\frac{24}{4}=6. $$

Each such sequence has two heads and two tails, so its probability is \((0.6)^2(0.4)^2\). For example, H-T-H-T has that probability, as do the other five sequences with two heads. Summing their equal probabilities gives

$$ P(X=2) =6(0.6)^2(0.4)^2 =6(0.36)(0.16) =6(0.0576) =0.3456. $$

So the probability of exactly two heads is \(0.3456\). The factor 6 counts possible positions; it is not an additional trial probability.

Using the Formula in Repeated-Trial Settings

The same calculation applies whenever the situation genuinely consists of independent trials with a common probability of success. The success label depends on the question: it might mean a patient responds to treatment, a component is defective, or a free throw is made. Once the trial outcome is coded as success or failure, identify \(n\), \(p\), and the requested count \(k\), then interpret each factor in the formula.

Worked Example: Exactly Four Positive Results

Suppose five independent tests each have probability \(0.9\) of returning a positive result. Find the probability that exactly four tests are positive.

There are \(n=5\) tests, the success probability is \(p=0.9\), and the failure probability is \(1-p=0.1\). The four positive results can occupy any four of the five test positions, so there are

$$ \binom{5}{4}=\frac{5!}{4!1!}=5 $$

possible sequences. Each has probability \((0.9)^4(0.1)\). Therefore

$$ P(X=4) =\binom{5}{4}(0.9)^4(0.1) =5(0.6561)(0.1) =5(0.06561) =0.32805. $$

The result concerns exactly four positive tests, so the one remaining test must be negative. The factor \(0.1\) accounts for that failure.

Worked Example: Exactly One Made Free Throw

A player makes each free throw independently with probability \(0.7\). Find the probability of making exactly one shot in three attempts.

Here \(n=3\), \(k=1\), \(p=0.7\), and \(q=0.3\). The single made shot can occur in any one of three positions, so \(\binom{3}{1}=3\). Each corresponding sequence has one success and two failures:

$$ P(X=1) =\binom{3}{1}(0.7)(0.3)^2 =3(0.7)(0.09) =3(0.063) =0.189. $$

As a direct check, the three sequences are made-missed-missed, missed-made-missed, and missed-missed-made. Each has probability \(0.7(0.3)(0.3)=0.063\), and their total is \(3(0.063)=0.189\), agreeing with the formula.

Why the Formula Accounts for Every Outcome

The possible success counts divide all trial sequences into groups: one group has zero successes, another has one, and so on through \(n\) successes. These groups do not overlap, and together they include every sequence. The probabilities given by the formula must therefore sum to 1. The binomial theorem confirms this directly:

Result: Summing the binomial probabilities over all possible counts gives 1:
$$ \sum_{k=0}^{n}\binom{n}{k}p^k(1-p)^{n-k} =\bigl(p+(1-p)\bigr)^n =1^n =1. $$

The first equality is the binomial expansion of \((p+(1-p))^n\). It also reflects the sequence argument: every ordered list of \(n\) outcomes has been included in exactly one count group. This is a useful consistency check on the formula, not a substitute for checking that the trial assumptions are appropriate.

Common Mistake: Forgetting the Number of Arrangements

A frequent error is to calculate \(p^k(1-p)^{n-k}\) and report it as \(P(X=k)\). That quantity is the probability of one specified sequence with \(k\) successes. It is the probability of the count only when there is just one sequence with that count. In general, there are \(\binom{n}{k}\) distinct sequences, so their probabilities must be added.

The opposite error is to count the arrangements and then treat the factor \(\binom{n}{k}\) as though it were a trial probability. It is a number of possible sequences, not the probability of any one sequence. The multiplication works because the \(\binom{n}{k}\) disjoint sequences each have the same probability \(p^k(1-p)^{n-k}\).

The assumptions matter as well. If trial probabilities differ, or outcomes across trials are dependent, a particular sequence need not have probability \(p^k(1-p)^{n-k}\). In that case this derivation does not apply as written. First check the repeated-trial conditions; then use the formula for the count.

Check Your Understanding

Use the sequence-counting argument, not just a memorized formula.

  1. In six independent trials with success probability \(0.2\), how many ordered sequences have exactly two successes?
  2. For a specified sequence of five trials containing three successes and two failures, write its probability when the common success probability is \(p\).
  3. In four independent trials with success probability \(0.5\), explain why the probability of exactly one success is not just \(0.5(0.5)^3\).
  4. State which part of the derivation uses independence and which part uses the count \(\binom{n}{k}\).
  5. Why do the probabilities for all success counts from zero through \(n\) sum to 1?

Previous

Next

Back to Tutorials