Hypergeometric Probability Ratios and the Mode

Why Compare Adjacent Probabilities?

A hypergeometric random variable counts successes in a sample drawn without replacement from a finite population. Its possible counts form a consecutive range of integers, but the probabilities need not increase or decrease throughout that range. Comparing the probability at one count with the probability at the next gives a direct way to find where the distribution reaches its maximum—and to detect when the maximum is shared by two counts.

The method is especially useful when individual probabilities involve large combinations. Rather than calculate every probability, we can derive a ratio, determine where that ratio is greater than, equal to, or less than 1, and identify the mode. This tutorial is aligned with the SOA VEE Mathematical Statistics syllabus.

Definition: Suppose a population contains $N$ objects, of which $K$ are classified as successes. A simple random sample of size $n$ is drawn without replacement, and $X$ is the number of successes in the sample. Then $X$ has a hypergeometric distribution with parameters $N$, $K$, and $n$, and $$ P(X=k)=\frac{\binom{K}{k}\binom{N-K}{n-k}}{\binom{N}{n}}. $$ Its support is $L\leq k\leq U$, where $L=\max(0,n-(N-K))$ and $U=\min(n,K)$. A mode is any support value $k$ at which $P(X=k)$ is maximal.

The support matters: a ratio of probabilities is useful only when both adjacent counts are possible. In what follows, compare $P(X=k+1)$ with $P(X=k)$ for $L\leq k<U$. The population size, number of successes, and sample size are fixed; the draws are without replacement and every sample of size $n$ is equally likely.

Deriving the Consecutive-Probability Ratio

Write $p_k=P(X=k)$. For two consecutive values within the support, divide the probability at $k+1$ by the probability at $k$. The common denominator $\binom{N}{n}$ cancels. Applying the identities for ratios of consecutive binomial coefficients gives

\frac{p_{k+1}}{p_k} = \frac{\binom{K}{k+1}}{\binom{K}{k}} \frac{\binom{N-K}{n-k-1}}{\binom{N-K}{n-k}} = \frac{K-k}{k+1} \frac{n-k}{N-K-n+k+1}.

All factors in this expression are positive when both $k$ and $k+1$ belong to the support. Thus the ratio is greater than 1 exactly when the probability increases from $k$ to $k+1$; it equals 1 exactly when those probabilities tie; and it is less than 1 exactly when the probability decreases.

Derivation: To locate the change from increasing to decreasing probabilities, compare the ratio's numerator and denominator. Since both are positive on the relevant support, $$ \frac{(K-k)(n-k)}{(k+1)(N-K-n+k+1)} \gtreqless 1 \quad\Longleftrightarrow\quad (K-k)(n-k)\gtreqless(k+1)(N-K-n+k+1). $$ Expanding and collecting terms gives $$ (K-k)(n-k)-(k+1)(N-K-n+k+1) =(n+1)(K+1)-(N+2)(k+1). $$ Therefore $p_{k+1}$ is greater than, equal to, or less than $p_k$ according as $$ k+1 \quad\text{is less than, equal to, or greater than}\quad q=\frac{(n+1)(K+1)}{N+2}. $$ This linear comparison explains why the probability ratios cross 1 at most once.

The resulting mode rule is compact. If $q$ is not an integer, the unique mode is $\lfloor q\rfloor$, the greatest integer less than $q$. If $q$ is an integer, there are two modes: $q-1$ and $q$. The support constraints ensure these candidates are possible counts. In the integer case, the ratio between those two counts is exactly 1; probabilities rise up to the tie and fall afterward.

Theorem: For a hypergeometric count with parameters $N$, $K$, and $n$, let $q=(n+1)(K+1)/(N+2)$. If $q$ is noninteger, the mode is $\lfloor q\rfloor$. If $q$ is an integer, both $q-1$ and $q$ are modes.

Applying the Ratio and Mode Rule

Worked Example: Finding a Unique Mode

A hypothetical kit contains 20 components, 7 of which have a designated coating. A quality analyst selects 5 components uniformly at random without replacement. Find the most probable number of coated components. Verify the result using adjacent probabilities.

Method. This is a hypergeometric model with $N=20$, $K=7$, and $n=5$. Its support is $L=\max(0,5-13)=0$ through $U=\min(5,7)=5$.

Calculate the mode threshold: $$ q=\frac{(5+1)(7+1)}{20+2}=\frac{48}{22}=\frac{24}{11}\approx2.1818. $$ Because $q$ is not an integer, the mode is $\lfloor q\rfloor=2$.

Check the probabilities on either side. At $k=1$, $$ \frac{p_2}{p_1}=\frac{(7-1)(5-1)}{(1+1)(13-5+1+1)} =\frac{24}{20}=\frac65>1. $$ At $k=2$, $$ \frac{p_3}{p_2}=\frac{(7-2)(5-2)}{(2+1)(13-5+2+1)} =\frac{15}{33}=\frac5{11}<1. $$ The probabilities are still increasing when the count goes from 1 to 2, then decreasing from 2 to 3. The ratio comparison shows the same increase-before and decrease-after pattern throughout the support.

For a numerical check, $\binom{20}{5}=15{,}504$, and $$ p_2=\frac{\binom72\binom{13}{3}}{\binom{20}{5}} =\frac{21\cdot286}{15{,}504} =\frac{6{,}006}{15{,}504}\approx0.3875. $$ Also, $p_1=5{,}005/15{,}504\approx0.3228$ and $p_3=2{,}730/15{,}504\approx0.1761$. The values confirm that count 2 is more probable than either adjacent count. Common mistake avoided: choosing the mode by rounding the mean; the mode rule comes from the probability ratios, not simply from rounding an expected value.

Worked Example: Identifying Two Tied Modes

A hypothetical collection has 8 cards, 3 of which carry a particular symbol. Four cards are drawn uniformly at random without replacement. Find every modal value of the number of symbol cards.

Method. Here $N=8$, $K=3$, and $n=4$. The support is $L=\max(0,4-5)=0$ through $U=\min(4,3)=3$. The threshold is $$ q=\frac{(4+1)(3+1)}{8+2}=\frac{20}{10}=2. $$ Since $q$ is an integer, the modes are $q-1=1$ and $q=2$.

Verify the tie directly. There are $\binom84=70$ possible samples. For $k=1$, $$ p_1=\frac{\binom31\binom53}{\binom84} =\frac{3\cdot10}{70}=\frac37. $$ For $k=2$, $$ p_2=\frac{\binom32\binom52}{\binom84} =\frac{3\cdot10}{70}=\frac37. $$ The adjacent ratio at $k=1$ is also $$ \frac{p_2}{p_1} =\frac{(3-1)(4-1)}{(1+1)(5-4+1+1)} =\frac66=1. $$ At $k=0$, the ratio is $12/2=6>1$; at $k=2$, it is $2/12=1/6<1$. Thus probabilities rise to the tie at 1 and 2, then decline. Both counts are modes. Common mistake avoided: reporting only one mode when the threshold is an integer.

Worked Example: Handling a Restricted Lower Support

A hypothetical shipment contains 18 units, including 5 flagged units. An inspector selects 14 units without replacement. Find the modal number of flagged units and verify it despite the fact that zero flagged units is impossible.

Method. Use $N=18$, $K=5$, and $n=14$. At most 13 unflagged units exist, so at least $14-13=1$ flagged unit must be selected. The support is $L=\max(0,14-13)=1$ through $U=\min(14,5)=5$.

The threshold is $$ q=\frac{(14+1)(5+1)}{18+2}=\frac{90}{20}=4.5. $$ It is noninteger, so the unique mode is $\lfloor4.5\rfloor=4$. The adjacent ratios around that candidate are $$ \frac{p_4}{p_3} =\frac{(5-3)(14-3)}{(3+1)(13-14+3+1)} =\frac{22}{12}=\frac{11}{6}>1, $$ and $$ \frac{p_5}{p_4} =\frac{(5-4)(14-4)}{(4+1)(13-14+4+1)} =\frac{10}{20}=\frac12<1. $$ So the probabilities increase up to 4 and decrease after 4. As a numerical check, $\binom{18}{14}=3{,}060$, and $$ p_4=\frac{\binom54\binom{13}{10}}{\binom{18}{14}} =\frac{5\cdot286}{3{,}060} =\frac{143}{306}\approx0.4673. $$ The adjacent probabilities are $p_3=(6/11)p_4=13/51\approx0.2549$ and $p_5=(1/2)p_4=143/612\approx0.2337$, both smaller. Common mistake avoided: assuming the support begins at zero just because a count cannot be negative.

Common Ratio Errors and a Reliable Procedure

A frequent algebra error is to omit the $+1$ in the factor $N-K-n+k+1$. That term comes from taking the ratio of two consecutive binomial coefficients; it is not optional. For the card example above, incorrectly using $N-K-n+k$ at $k=1$ gives a denominator of $2\cdot2=4$ and a false ratio $6/4=1.5$. The correct factor is $5-4+1+1=3$, giving denominator $2\cdot3=6$ and ratio $6/6=1$. The incorrect calculation would miss the tie.

A second error is to apply the ratio at a count outside the range where both probabilities are positive. At an endpoint, first check the support; do not infer a ratio for an impossible count. A sound solution follows the same order each time:

State the model and support.
Identify $N$, $K$, and $n$, and calculate $L$ and $U$.

Calculate the threshold.
Evaluate $q=(n+1)(K+1)/(N+2)$ without rounding it to an integer prematurely.

Report the mode or modes.
Use $\lfloor q\rfloor$ if $q$ is noninteger; if $q$ is integer, report both $q-1$ and $q$.

Check the transition.
Evaluate the ratio just before and after the candidate mode, when those adjacent support values exist. Confirm an increase before the maximum and a decrease after it, or equality at a tie.

The mode describes the single most probable count, not the mean and not necessarily a count with a large probability in absolute terms. The ratio method is valuable because it isolates the location and possible tie of the maximum without requiring a full table of probabilities.

Key takeaway: For adjacent hypergeometric probabilities, the ratio crosses 1 according to whether $k+1$ is below or above $(n+1)(K+1)/(N+2)$. A noninteger threshold gives one mode, its floor; an integer threshold gives two tied consecutive modes.

Check Your Understanding

For each hypothetical sampling-without-replacement situation, identify the support and use the threshold or adjacent ratios to determine the mode or modes.

A set contains 16 items, 6 of which are marked. A sample of 3 is drawn uniformly without replacement. Find the support and the mode. (Answer: Support $0$–$3$; unique mode $1$.)
A population contains 12 records, 5 flagged. A sample of 5 is selected without replacement. Calculate the threshold and find the mode. (Answer: $q=18/7$; unique mode $2$.)
A population contains 14 objects, 3 of which are designated successes. A sample of 3 is drawn without replacement. Find the support and every mode. (Answer: Support $0$–$3$; modes $0$ and $1$.)
A shipment contains 20 units, 17 of them tagged. Eight are selected without replacement. Find the support and mode, taking account of the minimum possible success count. (Answer: Support $5$–$8$; unique mode $7$.)
A population of 25 components contains 9 with a particular feature. A sample of 10 is taken without replacement. Find the mode and verify the transition by calculating the ratios on either side. (Answer: Unique mode $4$; $p_4/p_3=21/20$ and $p_5/p_4=6/11$.)

What You'll Learn