Aggregation and Conditioning in Multinomial Samples

Why Aggregation and Conditioning Matter

A multinomial sample records counts in several mutually exclusive categories. Often, the question concerns only whether an observation falls in a particular group of categories, or how the counts are distributed after a group total is known. These are not approximations: both questions have exact sampling laws. The key is to distinguish the probability of entering a group from the conditional probabilities of landing in its individual categories.

We begin with the multinomial model for $n$ independent trials. Each trial falls in exactly one of $k$ categories, with category probabilities $p_1,\ldots,p_k$, where $p_i\geq 0$ and $\sum_{i=1}^k p_i=1$. Write $X_i$ for the count in category $i$. For nonnegative integers $x_i$ summing to $n$, the joint probability mass function is

P(X_1=x_1,\ldots,X_k=x_k) =\frac{n!}{\prod_{i=1}^k x_i!}\prod_{i=1}^k p_i^{x_i}.

As usual, a factor $0^0$ in this mass function is interpreted as $1$. The multinomial coefficient counts the trial outcomes with those category totals; the product gives the probability of each such outcome.

Aggregating Categories

Let $A$ be a nonempty subset of the categories and define its total count and total probability by

X_A=\sum_{i\in A}X_i, \qquad p_A=\sum_{i\in A}p_i.

On each trial, the event of landing in any category in $A$ has probability $p_A$. Thus $X_A$ counts the number of successes in $n$ independent trials, where “success” means landing somewhere in $A$.

Theorem: For $0\leq p_A\leq 1$, the aggregate count has the binomial distribution $X_A\sim\operatorname{Bin}(n,p_A)$. More generally, if the $k$ categories are partitioned into groups $G_1,\ldots,G_g$, their group totals $Y_j=\sum_{i\in G_j}X_i$ have a multinomial distribution with size $n$ and probabilities $q_j=\sum_{i\in G_j}p_i$.

To see the general grouping result, consider any nonnegative group totals $y_1,\ldots,y_g$ summing to $n$. For a fixed allocation of the counts within each group, the original multinomial mass has the common factor $\prod_j q_j^{y_j}$ after summing over allocations, while the sum of the within-group multinomial coefficients is $y_j!^{-1}(\sum_{i\in G_j}p_i)^{y_j}$ for group $j$, by the multinomial theorem. Consequently the grouped mass is

P(Y_1=y_1,\ldots,Y_g=y_g) =\frac{n!}{\prod_{j=1}^g y_j!}\prod_{j=1}^g q_j^{y_j}.

This is the multinomial mass function with group probabilities $q_1,\ldots,q_g$. Taking two groups, $A$ and its complement, gives the binomial law for $X_A$.

Worked Example: Combining Two Categories

Suppose $n=12$, and each observation falls into one of four categories with probabilities $0.10,0.20,0.30,0.40$. Let $A$ consist of categories 1 and 2. Then $p_A=0.10+0.20=0.30$, so $X_A\sim\operatorname{Bin}(12,0.30)$. The probability that exactly four observations fall in those two categories is

P(X_A=4)=\binom{12}{4}(0.30)^4(0.70)^8 =495(0.0081)(0.05764801) \approx 0.23114.

The separate probabilities $0.10$ and $0.20$ do not enter this aggregate probability except through their sum $0.30$. They matter, however, if we ask how the four observations are split between categories 1 and 2; that is a conditional question.

Allocation Within a Group Given Its Total

Suppose the total $X_A$ is known to equal $m$, where $0\leq m\leq n$. If $p_A>0$, the conditional probability that a trial belongs to category $i$, given that it belongs somewhere in $A$, is $p_i/p_A$. The counts within $A$ therefore have a multinomial law of size $m$ with these renormalized probabilities.

Theorem: Assume $0<p_A<1$. For nonnegative integers $x_i$, $i\in A$, with $\sum_{i\in A}x_i=m$, $$ P\bigl((X_i)_{i\in A}=(x_i)_{i\in A}\mid X_A=m\bigr) =\frac{m!}{\prod_{i\in A}x_i!} \prod_{i\in A}\left(\frac{p_i}{p_A}\right)^{x_i}. $$ The complement counts, conditional on $X_A=m$, have a multinomial distribution of size $n-m$ with probabilities $p_j/(1-p_A)$, $j\notin A$. These two conditional count vectors are independent.

Here is a direct derivation that also verifies the coefficient. For specified counts $x_i$ in $A$, let $m=\sum_{i\in A}x_i$. Sum the original multinomial mass over all possible allocations to the complement. The complement counts sum to $n-m$, so the multinomial theorem gives

P\bigl((X_i)_{i\in A}=(x_i)_{i\in A},X_A=m\bigr) = \frac{n!}{\left(\prod_{i\in A}x_i!\right)(n-m)!} \prod_{i\in A}p_i^{x_i}(1-p_A)^{n-m}.

The denominator contains $(n-m)!$, because the unrecorded complement consists of $n-m$ observations. Divide by $P(X_A=m)=\frac{n!}{m!(n-m)!}p_A^m(1-p_A)^{n-m}$. Since $\sum_{i\in A}x_i=m$, the quotient is exactly the conditional mass in the theorem.

For the independence assertion, specify also complement counts $z_j$, $j\notin A$, summing to $n-m$. Divide their joint multinomial mass together with the specified $x_i$'s by $P(X_A=m)$. The result factors as

\left[ \frac{m!}{\prod_{i\in A}x_i!} \prod_{i\in A}\left(\frac{p_i}{p_A}\right)^{x_i} \right] \left[ \frac{(n-m)!}{\prod_{j\notin A}z_j!} \prod_{j\notin A}\left(\frac{p_j}{1-p_A}\right)^{z_j} \right].

Each bracket is a conditional multinomial mass function, and their product establishes conditional independence. The assumptions $0<p_A<1$ ensure that both conditional probability vectors are defined. If a group has probability zero, its total is necessarily zero; if it has probability one, its complement total is necessarily zero.

Proof: The aggregation formula follows by summing the multinomial mass over allocations within each group and applying the multinomial theorem. The conditional formula follows by dividing the specified-count-and-total probability by the binomial mass of the total. Factoring the conditional joint mass into the two displayed multinomial masses proves conditional independence. $\square$

Worked Example: Splitting a Known Group Total

Use the four-category model with probabilities $0.10,0.20,0.30,0.40$, and suppose the combined total in categories 1 and 2 is known to be $4$. Within this group, the conditional probabilities are

\frac{p_1}{p_A}=\frac{0.10}{0.30}=\frac13, \qquad \frac{p_2}{p_A}=\frac{0.20}{0.30}=\frac23.

Thus $X_1\mid X_A=4\sim\operatorname{Bin}(4,1/3)$. For example, the probability that two of the four are in category 1 and two in category 2 is

\binom{4}{2}\left(\frac13\right)^2\left(\frac23\right)^2 =6\cdot\frac19\cdot\frac49 =\frac{8}{27}.

The complement has probability $0.70$, split between categories 3 and 4 in proportions $0.30/0.70=3/7$ and $0.40/0.70=4/7$. Given $X_A=4$, its total is $8$, and its conditional count vector is multinomial with size $8$ and probabilities $3/7,4/7$, independently of the allocation within $A$.

Conditioning on Specified Cell Counts

A related calculation fixes counts in individual categories rather than fixing only their combined total. Let $S$ be a subset of categories, and suppose $X_i=x_i$ is specified for every $i\in S$. Write $r=\sum_{i\in S}x_i$. If $p_S=\sum_{i\in S}p_i<1$, then $n-r$ observations remain to be allocated among categories outside $S$.

Theorem: If the conditioning event $X_i=x_i$ for all $i\in S$ has positive probability and $p_S<1$, then for nonnegative $z_j$, $j\notin S$, with $\sum_{j\notin S}z_j=n-r$, $$ P\bigl((X_j)_{j\notin S}=(z_j)_{j\notin S}\mid (X_i)_{i\in S}=(x_i)_{i\in S}\bigr) =\frac{(n-r)!}{\prod_{j\notin S}z_j!} \prod_{j\notin S}\left(\frac{p_j}{1-p_S}\right)^{z_j}. $$

To derive this, the joint probability of the specified counts in $S$ and the particular remaining counts is $\frac{n!}{\prod_{i\in S}x_i!\prod_{j\notin S}z_j!}\prod_{i\in S}p_i^{x_i}\prod_{j\notin S}p_j^{z_j}$. Summing over all remaining allocations gives the probability of the conditioning event:

P((X_i)_{i\in S}=(x_i)_{i\in S}) =\frac{n!}{\left(\prod_{i\in S}x_i!\right)(n-r)!} \prod_{i\in S}p_i^{x_i}(1-p_S)^{n-r}.

Dividing the joint probability by this event probability cancels the fixed-cell factors and leaves the stated multinomial mass. Thus fixed cell counts use up $r$ observations, while the remaining $n-r$ observations are allocated according to the original probabilities renormalized over the categories not fixed.

Worked Example: Fixing Two Cell Counts

Let $n=10$ and let four categories have probabilities $0.10,0.20,0.30,0.40$. Condition on $X_1=2$ and $X_2=3$. There are $5$ remaining observations, and the probability mass outside the fixed categories is $1-(0.10+0.20)=0.70$. The remaining conditional category probabilities are therefore

P(3\mid\text{outside }\{1,2\})=\frac{0.30}{0.70}=\frac37, \qquad P(4\mid\text{outside }\{1,2\})=\frac{0.40}{0.70}=\frac47.

Consequently $X_3$, conditional on $X_1=2,X_2=3$, has distribution $\operatorname{Bin}(5,3/7)$, and $X_4=5-X_3$. In particular,

P(X_3=2,X_4=3\mid X_1=2,X_2=3) =\binom52\left(\frac37\right)^2\left(\frac47\right)^3 =10\cdot\frac9{49}\cdot\frac{64}{343} =\frac{5760}{16807}\approx 0.3427.

The conditioning event has positive probability: its complement total is $5$, and the original complement probability is $0.70>0$. This check matters; a conditional distribution is defined only when the event conditioned on has positive probability.

Common Pitfall: Keeping the Original Probabilities

A frequent error is to condition on a group total and then use the original category probabilities without renormalizing. For instance, in the four-category example, after learning that an observation is in categories 1 or 2, its conditional probability of being in category 1 is not $0.10$; it is $0.10/(0.10+0.20)=1/3$. The conditional probabilities must sum to one over the categories still under consideration.

A second useful check is to track the count left to allocate. If $X_A=m$, the complement contains $n-m$, not $m$, observations. In the joint mass for specified counts within $A$ and the aggregate total $m$, the factorial for the aggregated complement is therefore $(n-m)!$. The conditional factorization provides an independent check: the within-group multinomial coefficient uses $m!$, while the complement coefficient uses $(n-m)!$.

Key takeaway: Aggregation adds category probabilities and produces an exact binomial or multinomial law. Conditioning fixes a total number of observations, after which the allocation among the relevant categories is multinomial with probabilities renormalized to sum to one.

Check Your Understanding

Work from the multinomial model, keeping track of both the remaining sample size and the conditional probabilities.

If $n=20$ and a group of categories has total probability $0.15$, what is the distribution of its count?
For category probabilities $0.2,0.3,0.5$, find the conditional probability of category 2 given that an observation is in category 1 or 2.
When conditioning on $X_1+X_2=m$, how many observations remain in categories outside $\{1,2\}$?
In a four-category multinomial model, what distribution governs the remaining two counts after the first two cell counts are fixed?
Why must the probability of the conditioning event be positive before its conditional distribution is defined?