Binomial and Hypergeometric Sampling Compared

Two Sampling Designs, Two Count Models

A count of successes in $n$ draws can follow different probability models depending on how the draws are made. If each draw is an independent trial with the same success probability, the count is binomial. If the draws come from a finite population and sampled items are not replaced, the count is hypergeometric. The success proportion on any one draw may be the same in both setups, but the joint behavior of the draws is not: without replacement, one draw changes the composition of the population left for the next.

To compare the models, suppose a finite population contains $N$ items, of which $K$ are successes. Let $n$ be the number of draws, with $0\leq n\leq N$, and define $p=K/N$. Sampling $n$ times with replacement gives independent success indicators, each with probability $p$. Sampling $n$ items without replacement gives a hypergeometric count.

Definition: For $0\leq K\leq N$ and $0\leq n\leq N$, the number $X$ of successes in $n$ draws without replacement from a population with $K$ successes and $N-K$ failures has a hypergeometric distribution: $$ P(X=k)=\frac{\binom{K}{k}\binom{N-K}{n-k}}{\binom{N}{n}}, \qquad \max(0,n-(N-K))\leq k\leq \min(n,K). $$ If instead $n$ independent draws are made with replacement, the success count $Y$ has a binomial distribution: $$ P(Y=k)=\binom{n}{k}p^k(1-p)^{n-k}, \qquad k=0,1,\ldots,n, \quad p=\frac{K}{N}. $$

The parameters $N,K,n$ describe a particular finite-population sampling design. The binomial model uses only the number of draws and the success probability $p$, treating each draw as independent. Using the same $p$ makes the two models comparable; it does not make them identical.

Supports: Which Counts Are Possible?

The binomial count has support $\{0,1,\ldots,n\}$ when $0<p<1$: every number from zero through $n$ can occur under independent trials. In the finite population, however, there may not be enough successes or failures to produce every count. The hypergeometric support begins at $\max(0,n-(N-K))$, since at most $N-K$ failures can be drawn, and ends at $\min(n,K)$, since no more than $K$ successes exist.

Consequently, when $0<K<N$, the hypergeometric support is a subset of the binomial support. The supports are equal exactly when $K\geq n$ and $N-K\geq n$. If either condition fails, the binomial model assigns positive probability to one or more counts that are impossible without replacement. Even when the supports are equal, the probabilities need not match.

Worked Example: A Count Impossible Without Replacement

A population has $N=12$ items, including $K=2$ successes. Draw $n=4$ items. Compare the possible counts under sampling without replacement and with replacement, using $p=2/12=1/6$ for the binomial model.

Without replacement, there are only two successes in the entire population, so $X$ can be $0,1,$ or $2$, but cannot be $3$ or $4$. The hypergeometric probabilities are $$ P(X=0)=\frac{\binom{2}{0}\binom{10}{4}}{\binom{12}{4}} =\frac{210}{495}=\frac{14}{33}, \qquad P(X=1)=\frac{\binom{2}{1}\binom{10}{3}}{\binom{12}{4}} =\frac{240}{495}=\frac{16}{33}, $$ $$ P(X=2)=\frac{\binom{2}{2}\binom{10}{2}}{\binom{12}{4}} =\frac{45}{495}=\frac{1}{11}. $$ These probabilities sum to $14/33+16/33+3/33=1$.

With replacement, $Y\sim\operatorname{Binomial}(4,1/6)$, so all counts from 0 to 4 have positive probability. For example, $$ P(Y=3)=\binom{4}{3}\left(\frac{1}{6}\right)^3\frac{5}{6} =\frac{20}{1296}=\frac{5}{324}, \qquad P(Y=4)=\left(\frac{1}{6}\right)^4=\frac{1}{1296}. $$ These outcomes are possible when an item is returned after each draw, but impossible in the four-item sample without replacement.

Comparing Probability Masses

For a count $k$ in the hypergeometric support, both models assign positive probability when $0<K<N$. Their ratio gives an exact comparison. Write $(a)_r=a(a-1)\cdots(a-r+1)$ for $r\geq1$, and $(a)_0=1$. This is the falling factorial; it counts ordered selections of $r$ distinct objects from $a$ objects.

Theorem: Suppose $0<K<N$, $1\leq n\leq N$, and $k$ is in the hypergeometric support. With $p=K/N$, $$ \frac{P(X=k)}{P(Y=k)} = \frac{(K)_k}{K^k} \frac{(N-K)_{n-k}}{(N-K)^{n-k}} \frac{N^n}{(N)_n}. $$ Thus the two masses at a feasible $k$ agree exactly when this ratio is $1$. At a $k$ outside the hypergeometric support, the hypergeometric mass is zero while the binomial mass is positive.

To verify the ratio, the hypergeometric mass can be written as $\binom{n}{k}(K)_k(N-K)_{n-k}/(N)_n$. This follows either by expanding the binomial coefficients in the hypergeometric formula or by counting ordered draw sequences: choose the $k$ success positions, then fill those positions and the failure positions without replacement. The binomial mass is $\binom{n}{k}K^k(N-K)^{n-k}/N^n$, because each draw has success probability $K/N$. Dividing cancels $\binom{n}{k}$ and gives the displayed ratio.

The ratio also clarifies why matching $p$ is not enough to match the distributions. The falling-factorial terms account for the decreasing numbers of each type remaining after draws without replacement. A particular feasible mass can coincide by chance, so the ratio should be checked for an individual $k$. But, except in special cases, the entire distributions differ.

Worked Example: Same Support, Different Probabilities

A population has $N=10$ items, half successes, so $K=5$ and $p=1/2$. Draw $n=2$. Both models have support $\{0,1,2\}$, but their masses are different.

For sampling without replacement, the hypergeometric probabilities are $$ P(X=0)=\frac{\binom{5}{0}\binom{5}{2}}{\binom{10}{2}}=\frac{10}{45}=\frac{2}{9}, \quad P(X=1)=\frac{\binom{5}{1}\binom{5}{1}}{\binom{10}{2}}=\frac{25}{45}=\frac{5}{9}, \quad P(X=2)=\frac{2}{9}. $$ For sampling with replacement, $Y\sim\operatorname{Binomial}(2,1/2)$, giving $$ P(Y=0)=\frac{1}{4}, \qquad P(Y=1)=\frac{1}{2}, \qquad P(Y=2)=\frac{1}{4}. $$ The supports agree, but none of the corresponding probabilities agree. The restriction against repeated items concentrates more probability on one success than the independent model does.

Means Agree; Variances Usually Do Not

Let $p=K/N$. Both counts have mean $np$: each draw has success probability $p$, and the expected total is the sum of the $n$ success indicators. The variance comparison captures the effect of dependence. For $N\geq2$, the binomial variance is $np(1-p)$, whereas the hypergeometric variance is $np(1-p)(N-n)/(N-1)$.

Theorem: For $N\geq2$, $0\leq K\leq N$, and $0\leq n\leq N$, let $p=K/N$. If $X$ is the count without replacement and $Y$ is the count with replacement, then $$ E[X]=E[Y]=np, \qquad \operatorname{Var}(Y)=np(1-p), \qquad \operatorname{Var}(X)=np(1-p)\frac{N-n}{N-1}. $$ In particular, $$ \operatorname{Var}(Y)-\operatorname{Var}(X) =np(1-p)\frac{n-1}{N-1}. $$

Here is a derivation of the variance comparison. For the sample without replacement, let $I_i$ equal 1 if the $i$th draw is a success and 0 otherwise, so $X=\sum_{i=1}^n I_i$. Each indicator has $E[I_i]=p$ and $\operatorname{Var}(I_i)=p(1-p)$. For distinct draws $i,j$, the chance that both are successes is $K(K-1)/(N(N-1))$. Therefore $$ \operatorname{Cov}(I_i,I_j) =\frac{K(K-1)}{N(N-1)}-\left(\frac{K}{N}\right)^2 =-\frac{p(1-p)}{N-1}. $$ Adding the $n$ individual variances and the $n(n-1)$ ordered covariance terms gives $$ \operatorname{Var}(X) =np(1-p)-\frac{n(n-1)p(1-p)}{N-1} =np(1-p)\frac{N-n}{N-1}. $$ Also $E[X]=\sum_i E[I_i]=np$. For the with-replacement count, the $n$ indicators are independent Bernoulli variables with probability $p$, so their means add to $np$, and their variances add to $np(1-p)$. Subtracting the two variance formulas yields the stated difference. $\square$

If $0<K<N$, the variance difference is zero for $n=1$, and positive for $n\geq2$. Thus, for a nondegenerate population and at least two draws, the two distributions cannot be identical: identical distributions would have identical variances. The hypergeometric count is less variable because successes and failures become slightly harder to repeat as the sample is drawn. When $n=N$, every population item is selected, so $X=K$ with probability one and its variance is zero.

Worked Example: Checking the Finite-Population Variance

Return to $N=10,\ K=4,\ n=3$, so $p=0.4$. Both means are $$ E[X]=E[Y]=np=3(0.4)=1.2. $$ The binomial variance is $$ \operatorname{Var}(Y)=np(1-p)=3(0.4)(0.6)=0.72. $$ For sampling without replacement, the variance is $$ \operatorname{Var}(X) =0.72\frac{10-3}{10-1} =0.72\frac{7}{9} =0.56. $$ The difference is $0.72-0.56=0.16$, which agrees with the comparison formula: $$ np(1-p)\frac{n-1}{N-1} =0.72\frac{2}{9}=0.16. $$ The common mean does not imply equal distributions; their variances, and their probability masses, differ.

When Do the Models Agree?

There are several precise agreement cases. If $n=0$, both counts are zero. If $n=1$, both counts are Bernoulli with success probability $K/N$, so their distributions agree exactly. If $K=0$, both counts are always zero; if $K=N$, both are always $n$. These are degenerate cases, and they agree for any allowed $n$. If $N=1$, the only possible sample sizes are 0 and 1, already covered by these cases.

Outside these cases, suppose $0<K<N$ and $2\leq n\leq N$. The variance difference is strictly positive, so the distributions differ. They can still have the same support when there are at least $n$ successes and at least $n$ failures, but equal support is not equal probability. The ratio formula identifies exact agreement or disagreement for any particular feasible count.

Common Mistake

Wrong reasoning: “Each draw has success probability $p=K/N$, so the success count is binomial.”
Correction: A binomial count requires independent trials with the same success probability. With replacement, those conditions hold. Without replacement, the chance of success on a later draw depends on earlier outcomes; the count is hypergeometric. The two models share a mean, but generally have different probabilities, supports, and variances.

Key takeaway: Match the model to the sampling design. Replacement gives independent draws and a binomial count; sampling a fixed finite population without replacement gives a hypergeometric count. With $p=K/N$, their means agree, while the hypergeometric variance is reduced by the factor $(N-n)/(N-1)$.

Check Your Understanding

Use the sampling design, not just the success proportion, to answer these questions.

A population has 30 successes among 100 items. What are $p$ and the possible values of a hypergeometric count for a sample of 5?
When $0<K<N$, what conditions make the binomial and hypergeometric supports equal?
For $N=20,\ K=8,\ n=4$, compare the two means and write the two variance formulas using $p=8/20$.
Why does sampling without replacement reduce the variance when $n\geq2$ and $0<K<N$?
Give the conditions under which the binomial and hypergeometric distributions agree exactly.