Multinomial Probability Calculations in Categorical Sampling

From a Sampling Description to a Probability

In categorical sampling, a question often describes a sample by its final category counts: for example, how many observations fell into each response group. The order in which the observations appeared is usually irrelevant. An exact calculation therefore has two parts: count the ordered samples that produce the specified counts, and multiply by the probability of each such ordered sample.

Suppose \(n\) independent trials each produce one of \(k\) categories, with fixed category probabilities \(p_1,\ldots,p_k\), where \(p_i\geq 0\) and \(\sum_{i=1}^k p_i=1\). Write \(X_i\) for the number of observations in category \(i\). The count vector must have nonnegative integer entries summing to \(n\). For any such vector \((x_1,\ldots,x_k)\), the multinomial probability formula is

P(X_1=x_1,\ldots,X_k=x_k) = \frac{n!}{x_1!\cdots x_k!}\prod_{i=1}^k p_i^{x_i}.

The fraction is the multinomial coefficient. It counts the distinct ways to arrange \(x_1\) outcomes of category 1, \(x_2\) of category 2, and so on, among \(n\) trial positions. Each arrangement has the same probability \(\prod_i p_i^{x_i}\), so their probabilities add.

Calculation checklist: Confirm that the category probabilities sum to 1, the proposed counts are nonnegative integers summing to \(n\), and the coefficient and probability factors both match those counts. For any category with count zero, its factorial is \(0!=1\), and its probability factor is \(p_i^0=1\), including when \(p_i=0\).

Worked Calculations

Worked Example: One Specified Count Vector

A response is classified as A, B, C, or D with probabilities \(0.50, 0.25, 0.15,\) and \(0.10\), respectively. In \(n=8\) independent responses, find the probability of exactly 3 A responses, 2 B responses, no C responses, and 3 D responses.

The count vector is \((3,2,0,3)\), and its entries sum to \(3+2+0+3=8\), as required. The number of ordered sequences with these counts is

\frac{8!}{3!\,2!\,0!\,3!} = \frac{40320}{6\cdot2\cdot1\cdot6} = 560.

The probability of any one such sequence is \(0.50^3(0.25)^2(0.15)^0(0.10)^3\). Thus

$$ P(X_A=3,X_B=2,X_C=0,X_D=3) = 560(0.50)^3(0.25)^2(1)(0.10)^3 = 560(0.0000078125) = 0.004375. $$

The factor for C is 1 because there are no C responses; it is not an additional probability of observing C. The count vector itself specifies that C is absent.

Worked Example: Two Zero-Count Categories

A laboratory records six independent specimens as positive, inconclusive, negative, or invalid, with probabilities \(0.40, 0.30, 0.20,\) and \(0.10\). Find the probability of exactly 2 positive, 0 inconclusive, 4 negative, and 0 invalid results.

The counts \((2,0,4,0)\) sum to 6. The coefficient is

\frac{6!}{2!\,0!\,4!\,0!} = \frac{720}{2\cdot1\cdot24\cdot1} = 15.

The probability is therefore

$$ 15(0.40)^2(0.30)^0(0.20)^4(0.10)^0 = 15(0.16)(1)(0.0016)(1) = 0.00384. $$

Both zero-count categories remain part of the sampling model, but neither contributes a factor smaller than 1 to this particular count-vector probability. They still matter when checking that the full category probabilities sum to 1.

Worked Example: An Event With Several Count Vectors

Each of five independent observations falls in A, B, or C with probabilities \(0.50, 0.30,\) and \(0.20\). Find the probability of exactly 2 observations in A and at least 1 in B.

The number in C is not specified, so list the allowed count vectors. Since the two A observations leave three positions, and B must occur at least once, the possibilities are \((2,1,2)\), \((2,2,1)\), and \((2,3,0)\). These are disjoint events, so their probabilities add.

For \((2,1,2)\), the coefficient is \(5!/(2!1!2!)=30\), giving probability \(30(0.50)^2(0.30)(0.20)^2=0.0900\). For \((2,2,1)\), the coefficient is \(5!/(2!2!1!)=30\), giving \(30(0.50)^2(0.30)^2(0.20)=0.1350\). For \((2,3,0)\), the coefficient is \(5!/(2!3!0!)=10\), giving \(10(0.50)^2(0.30)^3=0.0675\).

Adding the disjoint cases gives

P(X_A=2,\ X_B\geq 1) = 0.0900+0.1350+0.0675 = 0.2925.

A useful check is that there are 10 ways to choose which two of the five positions are A. For any such choice, the other three positions must be B or C, with at least one B. That probability is \(0.50^3-0.20^3=0.117\). Hence \(10(0.50)^2(0.117)=0.2925\), agreeing with the count-vector sum.

Two Useful Summation Results

A count-vector probability is for one exact outcome of the count vector. An event that allows multiple count vectors requires a sum over all allowed vectors. When some counts are unrestricted, the multinomial theorem can collapse that sum into a single expression.

Theorem: Fix the counts \(x_1,\ldots,x_r\) for \(r<k\) categories, and let \(m=n-\sum_{i=1}^r x_i\). If \(m\) is a nonnegative integer, then

P(X_1=x_1,\ldots,X_r=x_r) = \frac{n!}{x_1!\cdots x_r!m!} \left(\prod_{i=1}^r p_i^{x_i}\right) \left(\sum_{j=r+1}^k p_j\right)^m.

The remaining \(m\) observations can be allocated among the unrestricted categories in every possible way. To prove the formula, sum the full multinomial probabilities over all nonnegative integers \(x_{r+1},\ldots,x_k\) summing to \(m\). Each summand is

\frac{n!}{x_1!\cdots x_r!} \left(\prod_{i=1}^r p_i^{x_i}\right) \frac{\prod_{j=r+1}^k p_j^{x_j}}{\prod_{j=r+1}^k x_j!}.

Factor out the terms that do not depend on the remaining counts. The multinomial theorem gives

\sum_{\substack{x_{r+1}+\cdots+x_k=m\\x_j\geq0}} \frac{\prod_{j=r+1}^k p_j^{x_j}}{\prod_{j=r+1}^k x_j!} = \frac{1}{m!}\left(\sum_{j=r+1}^k p_j\right)^m.

Substitution yields the stated expression. If any fixed count is incompatible with \(n\), or \(m<0\), the event is impossible and its probability is zero. \(\square\)

A particularly useful special case concerns a single specified category. Summing over all allocations among the other categories gives

P(X_i=x) = \binom{n}{x}p_i^x(1-p_i)^{n-x}, \qquad x=0,\ldots,n.

Indeed, take \(r=1\) in the theorem and use \(\sum_{j\ne i}p_j=1-p_i\). This is the binomial marginal of a multinomial count, and provides a convenient check when the event concerns only one category.

Corollary: The probability that a specified category with probability \(p_i\) has zero observations in \(n\) trials is \((1-p_i)^n\).

To verify this directly from the theorem, set \(x_i=0\). Then \(m=n\), the fixed-category factor is \(p_i^0=1\), and the sum of the other category probabilities is \(1-p_i\). The resulting probability is \((1-p_i)^n\). This calculation sums all possible allocations among the other categories, rather than treating any one allocation as the whole event. \(\square\)

How to Calculate Event Probabilities Reliably

For a general event, first translate its wording into constraints on the counts. Then enumerate the count vectors that satisfy those constraints, calculate each exact probability, and add them. If a category count is unrestricted, the summation theorem may reduce the work. If the event refers only to one category's count, its binomial marginal may be simpler than listing every full vector.

Record the model.
Write \(n\), the category probabilities, and the labels corresponding to the categories.

Translate the event.
Write down the required count equalities and inequalities, remembering that all counts must sum to \(n\).

Choose the calculation.
Use one multinomial term for a single vector, add terms for several allowed vectors, or sum unrestricted allocations with the theorem.

Check the result.
Verify the count total, coefficient, probability factors, and that the final probability lies between 0 and 1.

Common Mistakes

Using only the probability of one ordering. A count vector usually corresponds to many ordered sequences. Multiply the probability of one sequence by the multinomial coefficient.
Forgetting that counts must sum to \(n\). If they do not, the proposed vector is impossible; do not apply the formula as though it were valid.
Dropping an absent category from the model. A zero count contributes \(0!=1\) and \(p_i^0=1\), but the category still belongs in the probability list, whose entries must sum to 1.
Using one vector for an event with several possibilities. “At least one B” can include multiple count vectors. Add their probabilities, or use a justified summation shortcut.
Confusing “zero observed” with “probability zero.” A category with positive sampling probability can still have zero observations in a finite sample. Its zero-count probability is \((1-p_i)^n\).

Multinomial calculations are exact bookkeeping: the coefficient accounts for how many arrangements produce a count vector, and the product accounts for the probability of each arrangement. Keeping those roles distinct makes both point probabilities and more complicated count events easier to verify.

Check Your Understanding

Work out each question by identifying the relevant count vector or set of vectors.

For \(n=5\) trials with category probabilities \(0.60\) and \(0.40\), what is the coefficient for the count vector \((3,2)\)?
Why does a category with count zero contribute a factor of 1 to the multinomial probability formula?
With three categories and counts \((2,1,0)\) in \(n=3\) trials, how many ordered sequences produce that count vector?
For a category with probability \(0.25\) in \(n=4\) trials, calculate the probability of zero observations in that category.
In an event where one count is fixed and all other categories are unrestricted, what quantity replaces the probabilities of the unrestricted categories in the summation formula?